The pink brick walls laid from 1670 onwards by George Mathew, half-brother to the Duke of Ormond, were designed in an unusually refined manner for their time and place: a great chamber or gallery set above a rusticated arcade, projecting wings, and a massive oak staircase rising from the arcade to the first floor. The arcade may have been designed by Sir William Robinson, the same architect associated with the Royal Hospital Kilmarnock in Dublin, and the builders are thought to have been the same craftsmen who worked for the Duchess of Ormond at Dunmore House near Kilkenny. That original house, one room deep and two storeys tall, now amounts to a single upstanding brick-faced wall, 6.4 metres long and 0.75 metres wide, at the eastern end of the ruins, identifiable in part because the seventeenth-century bricks are measurably thinner and shorter than those used in later phases.

The building was substantially renovated from 1718 onwards, and then again in 1812, when the architect Richard Morrison was brought in to apply a mock-gothic façade, at which point the original pink brick was resurfaced in cement. The mock-gothic style involved adding pointed arches, castellated features, and other medievalising details that were fashionable among the Anglo-Irish gentry of the period. After Lady Elizabeth Mathew died in 1841, the house passed to the Viscount Rohan de Chabot and then to his son the Comte de Jamac, who used it as a summer residence until his death in 1872. It then went to Lord Dunsandle, who rarely came near the place. Caretakers managed the estate until the 1920s, when the Land Commission distributed the land and the house was left to deteriorate. In 1938 the ruin was acquired by Archbishop David Mathew, a descendant of the original family, but by 1973 part of the main structure had collapsed, taking the Gothic hall and dining room with it. Close inspection of the remaining walls reveals earlier bricks reused within the later stonework, a quiet record of each successive attempt to remake the building into something new.