The castle's history stretches back to at least 1582, when the Prendergasts of Ballyfarnoge leased it to George Dormer, a merchant from New Ross. The Dormers eventually became outright owners, and by 1641, Nicholas Dormer, described as an Irish papist in the Civil Survey of 1654-6, held 500 acres here along with what was termed a 'faire castle', complete with a watermill and weir.

The tower house itself is a relatively compact structure, measuring seven metres east to west and 6.6 metres north to south, rising three storeys to its stepped crenellations. Unlike many Irish tower houses, it lacks the typical defensive features such as base batter, vaulting, or dressed quoins. The original entrance, a lintelled doorway, sits towards the eastern end of the north wall. Much of the interior has been altered over the centuries; the internal walls are all later insertions, leaving few original features beyond some blocked window lights and a notable mullion and transom window with square hood mouldings on the second floor, now also blocked. Curiously, there's no evidence of the wall chambers or mural stairs typically found in such structures, nor any signs of wall plates that would have supported the original ceiling timbers.

The tower continued to serve as a dwelling well into the modern era, with Jos. Deane residing there in the 1830s according to Lewis's Topographical Dictionary, and it likely remained occupied into the 20th century. Today, the medieval tower forms part of the south range of an 18th or 19th century courtyard, standing vacant but still offering a tangible link to the centuries of merchant families, political upheavals, and everyday life that shaped this corner of County Wexford.