At its peak the structure covered roughly 3.5 acres, a square enclosure with an internal courtyard of around 65 metres a side, ringed by a limestone rubble wall and a moat nearly eleven metres wide. At each corner projected a pointed bastion, the angular design characteristic of early modern European fortification, where angled flanking towers allowed defenders to cover the full length of each wall without leaving blind spots. By the time the first Ordnance Survey six-inch map was published in 1839, virtually the whole thing had been dismantled or built over. Today it sits beneath the Avalon House hotel, about 120 metres north-west of the bridge over the river Dinin.

The fort's moment in the historical record comes with the 1641 Rising, when around 400 English Protestant settlers, brought to Castlecomer as part of the Wandesforde plantation, took shelter in what sources describe as the old castle on "the Garrison." The castle held out for more than three months before surrendering in March 1642 to Catholic Confederate forces under Captain John Bryan and Captain Phillip Purcell. A separate reference to the "Castle and fort" of Castlecomer in orders issued shortly afterwards to Richard Butler by the Confederate commander Mountgarret may be the only surviving mention of the bastioned fort as a distinct structure. A 1653 petition from William Wandesforde to the Commonwealth government recalls how his family had fled the estate during the Rising and could only return once Parliamentary forces had re-established an English garrison there. After that the fort recedes from the record entirely. When United Irishmen fought Crown forces through the streets of Castlecomer in 1798 in what became known as the Battle of Comer, burning the town and the Wandesforde residence in the process, the bastioned fort is not mentioned at all, suggesting it had long since ceased to serve any military function.

An archaeological excavation carried out in 2018 by Cóilín Ó Drisceoil recovered the north-east corner bastion and a section of a southern flank, allowing a detailed reconstruction of the fort's original dimensions and construction. The moat, more than 0.8 metres deep and possibly once wet, fed by a channel drawn from the nearby river Cloghogue, is still partially traceable. A 1812 town plan by James Healy had already recorded substantial remains surviving at that date; by 1839 nearly all of them were gone. One angular wall at the rear of what was formerly a coal-yard on the north side of High Street may preserve a fragment of the north-west bastion, though heavy ivy makes it difficult to say with any certainty.