Yet even in this reduced state the building rewards attention. The west wall was never bonded to the adjacent tower house, sitting some 1.65 metres to its west, which suggests the two structures were built at different times and perhaps conceived with different purposes. Later alterations, including the insertion of brick into the original stonework and the blocking of several windows, further complicate any reading of what the building once looked like. Of all its original features, only a single rectangular window with a chamfered surround, a neatly angled cut to the stone edge, has survived intact.

The Bryan family held the castle of Scar from at least the early seventeenth century. In 1640, Nicholas Bryan was recorded as holding the castle and 180 acres at Skarr, a considerable landholding by any measure. His tenure did not survive the upheavals of the Cromwellian settlement: by 1656 he had been displaced and awarded 80 acres in Connacht as compensation, part of the broader policy of transplanting Catholic landowners westward. The fortified house itself, two or three storeys in its original form and measuring roughly 16.5 metres north to south, would have stood alongside the tower house on the south-facing slope of a broad, low hill. A stone-lined circular well, now covered, lies to the northeast of the tower house and may have served both structures, even though there is no evidence of a bawn, the enclosed courtyard typically associated with defended settlements of this type, linking them formally.