What it is not, despite a persistent local tradition, is the legacy of the Knights Templars, who were said to have built a house here in 1184. Scholars have long since dismissed that claim; the Normans had not yet managed to establish themselves in Limerick at all by that date, making the story an attractive fiction rather than a historical fact.

The actual founders of the settlement were the Geraldines, the great Anglo-Norman dynasty also known as the Earls of Desmond, who were already present in this part of south-west Limerick by 1215. They held the castle here as one of their principal strongholds for the better part of four centuries, until the fall of the Desmonds in 1584. A rental document from 1297 to 1298 records a hundred court at the site, which has led some historians to classify it as a medieval borough, though the evidence remains ambiguous. By 1586, when the manor was formally surveyed following the Desmond collapse, the records reveal a surprisingly detailed landscape around the castle: a great orchard of four acres called Owlorde, a great garden known as Gardenmoore, two water mills, and a cluster of named streets including Shradegower, which translates roughly as Goat Street, and Shrade netona. The manor at that point ran to seven acres of core demesne land, with additional parcels stretching into the neighbouring parish of Monaghe Adare. By 1654 to 1656, when the Civil Survey recorded the lands then held by a Colonel Francis Courtney, the castle still came with a bawn (a walled enclosure typically used for livestock and defence), a weekly market, an annual fair, and the use of the Arra river running beside it.

The town sits on the Limerick to Tralee road, and the Arra river, which feeds into the Deel and eventually the Shannon, still runs close to the castle grounds. The present layout of the town, with its central square, appears to be a post-medieval arrangement; the medieval street pattern has not survived in any recoverable form, though the old street names preserved in sixteenth-century documents offer some sense of what once existed. The castle itself remains the most tangible remnant of the Geraldine period, and the surrounding demesne gives some indication of the scale of what was once a substantial lordly complex.