In 1610, James McGrath received these lands as a grant; he was the son of Archbishop Myler McGrath of Cashel. The following year, Lord Carew noted that the Archbishop had begun building "a prittie castle at Termon Magragh", with plans to quickly complete it alongside a fortified enclosure, or bawn, as all his building materials were already on site. The tower house would have been an impressive statement of the McGrath family's status in early 17th century Donegal.

The castle's active life proved brief. When Cromwell's forces swept through Ireland in 1649-50, they bombarded Termon McGrath, destroying the north wall of the tower house. The structure has stood as a ruin ever since, its broken walls a monument to that violent period of Irish history. The Down Survey map of 1655-6 shows "Tarmon McGragh Castle & Towne" in the Parish of Cam, Barony of Tyrehugh, indicating that a small village had developed around the castle by this time.

Archaeological evidence suggests the site was more extensive than just the castle itself. A network of drains can still be traced east and west of the roadway leading to the castle entrance, whilst aerial photography from the St. Joseph collection reveals what may be the layout of the former town, or possibly formal gardens that once surrounded the tower house. These ghostly outlines in the landscape hint at a thriving settlement that grew up in the shadow of the McGrath stronghold, only to fade after the castle's destruction.