Where cattle now graze on a west-southwest facing slope, 130 metres from the shoreline, once stood a castle whose very foundations have vanished into the boggy Mayo landscape. The site offers no dramatic ruins or crumbling towers; instead, visitors find only pasture stretching towards expanses of bog to the east, with perhaps the faintest suggestions in the earth where walls once stood.

The castle's story survives mainly through fragments of historical record and fading local memory. When antiquarian John O'Donovan visited in 1838 whilst compiling information for the Ordnance Survey, he found locals who could point to where the foundations lay, and elderly residents who recalled seeing substantial portions still standing in their youth. Yet even then, no one could tell him who built the castle or who last occupied it before its destruction. The 1838 Ordnance Survey map optimistically marked out a rectangular area of about 10 to 12 metres, accompanied by a small cluster of three farmhouses; by 1920, even these modest dwellings had disappeared, leaving only the notation "Castle (site of)" on updated maps.

Local tradition holds that fire destroyed the castle, though when this happened remains unclear. Historical documents from 1585 and 1612 suggest it may have been an early Butler family property, connecting this remote Mayo site to one of Ireland's most powerful Anglo-Norman dynasties. Today, the only tangible link to the area's ancient past stands 70 metres to the south: a cross-inscribed pillar stone, weathered but enduring, keeping silent watch over the place where Castlehill's lost castle once commanded the approaches to Blacksod Bay.