From the late 13th century, when Thomas Hay served as Sheriff of Wexford in 1284, the family maintained their grip on these lands through turbulent times. Records from 1324 show Nicholas Hay and others holding five carucates, or townlands, in what was then called Thahtcomesan. The family's prominence continued through the medieval period; in 1417, Dionysius Hay inherited three carucates from Magina Neville, widow of Philip Hay, and by 1593, Mathew Haie died still in possession of the manor of Tacumshane.

The Hays weathered the political upheavals of the 17th century remarkably well. According to the Book of Survey and Distribution from the 1660s, Robert Hay held 121 acres at Tacumshin and another 75 acres at nearby Reedstown in 1640, just before the Confederate Wars would transform Irish landholding forever. Their castle, however, has fared less well than the family's fortunes once did.

Today, only the base of the tower survives, measuring roughly 8 metres square and standing about 2 metres high. The structure is completely overgrown and inaccessible, though earlier surveys noted it once featured a vault and two doorways before its eastern wall was removed. What remains is a tantalising glimpse of medieval power; a moss-covered foundation that once anchored centuries of local authority in this quiet corner of Wexford.