Today, only fragments remain of this once impressive structure; a roughly D-shaped tower that rises just one storey high, though it still boasts an remarkable groin-vaulted ceiling over its ground floor. The tower, measuring 5.3 metres in external diameter with walls about a metre thick, shows evidence of a destroyed window on its northeast face and what appears to be a blocked doorway to the south. Portions of bawn wall attached to the northeast and southwest faces suggest this tower may have served as a flanking tower along a larger defensive curtain wall.

Historical records paint a picture of the castle's gradual decline. When documented in the OS Letters, it was described as an 'old castle which is now nearly destroyed', with only 'one small apartment with a stone arch over it' and isolated bits of wall remaining. By the time of the Civil Survey in 1654-6, the castle had deteriorated further, recorded as a 'ruined old castle, Irrepayreably demolished'. The raised platform on which the tower sits is believed to be the location of the original bawn, now defined only by a drainage feature.

The castle's stones have found new life in the surrounding area, though not in their original form. A modern laneway now cuts through the site just south of the tower, with a 19th-century building on the other side that appears to have been constructed using medieval stone, possibly salvaged from the demolished castle. Another nearby 19th-century house features a roughly dressed chamfered and rebated door jamb, likely medieval in origin, protruding from its gable wall; a tangible reminder of Kilfadda Castle's former grandeur, now scattered across the local landscape.