These structures, which survived until the early 19th century before vanishing completely, weren't grand fortresses but rather fortified town houses; similar examples still stand today in Fethard, County Tipperary, and in Ardee and Carlingford, County Louth.

Historical records from the early 20th century document these four castles, though their exact locations remain somewhat elusive. The Ordnance Survey maps mark one castle site to the south-east of the churchyard and another to the north of the village, though no physical remains survive at either location. A third castle possibly stood where Caherconlish House was later built, to the south-west of the village, according to 19th-century historian Maurice Lenihan. The town also apparently housed a "college", as noted by Samuel Lewis in his 1837 topographical survey, adding another layer to the settlement's medieval significance.

The Urban Survey of County Limerick, compiled by Bradley and colleagues in 1989, provides the most comprehensive modern assessment of these lost structures. Drawing on earlier work by antiquarian Thomas Johnson Westropp from 1906-7, the survey confirms that whilst traces of these castles lingered into the 1800s, the modern landscape has completely absorbed any remaining evidence. Today, only the annotated map references and historical documentation preserve the memory of Caherconlish's fortified past, leaving historians and archaeologists to piece together the story of what was clearly once a well-defended medieval town.