The site's history stretches back to the late 16th century, when various members of the mac Brien family held this fortification. Records from 1578 show Tirlagh mac Brien in possession of Cloghdallatowne castle, whilst between 1583 and 1586, it passed to Conor mac Brien of Coonagh. The mac Brien connection continued through the 17th century; in 1655, another Connor mac Brien held Clogdalton, maintaining the family's grip on this strategic location.

The 1657 Down Survey map of Coonagh Barony provides a tantalising glimpse of what once stood here, depicting a tower house castle with two accompanying dwellings at 'Cloughdalton'. This architectural arrangement was typical of Irish tower houses, where the main defensive structure would be supplemented by additional buildings for everyday living and agricultural purposes. The castle's importance in the local landscape is evident from its consistent appearance in historical documents, though various spellings; Cloghdallatowne, Cloghtallon, Clogdalton, and Coldalton; reflect the fluid nature of place names in early modern Ireland.

Following the upheavals of the 17th century, the castle changed hands from its traditional Irish owners. In 1668, the lands were granted to Harding and Henry as part of the wider redistribution of property following the Cromwellian conquest. By 1688, the site appeared on the Trustees' Map as 'Coldalton in Ulloe', one of the final documentary references to this once-prominent stronghold. Today, whilst the castle itself has disappeared, the annotated rock serves as a modest memorial to centuries of local history and the families who once commanded this corner of Limerick.