He counted twenty-four rooms on a single floor, marvelled at a staircase wide enough for twenty men to walk abreast, and noted that, had the building possessed one additional wing, it would have formed a perfect H-plan. Within a decade of that account, the house was gone, or at least emptied and abandoned. Today, the site on a north-south terrace above the Nore and Dinin river valleys is occupied by Dunmore Cottage, a modest replacement built at the beginning of the nineteenth century after the ruins were cleared away.

The story of Dunmore House is one of almost operatic reversals, centred on the Butler family and their Ormond estate. The manor had belonged to the Butlers for generations; a lease of 1593 even required that twelve couples of rabbits be delivered each week from Dunmore to the Earl's houses at Kilkenny and Gowran. In 1618, Elizabeth Butler, daughter of Thomas Butler the 10th Earl of Ormond, and her husband Richard Preston, Lord Dingwall, were granted a large portion of the contested Ormond estate. By 1623, Elizabeth, by then Countess of Desmond, was living at Dunmore and writing to arrange furnishings for the great chamber. She left Ireland in 1624 and never came back. Her daughter, also Elizabeth, married her cousin James Butler, the 1st Duke of Ormond, in 1629, reuniting the fractured estate. After the Cromwellian confiscations, Parliament ordered the return of Dunmore and its lands, valued at £2,000 a year, to Elizabeth and her children in 1653. A survey that same year found a castle, a stone house adjoining in repair, another small old castle, thatched houses, and some 2,126 acres. Elizabeth took up residence in 1657, and after the Restoration of 1660 she launched an ambitious building programme. Whether she refitted the existing structure or built entirely anew is debated; one view holds that a three-storey return was added to an earlier structure of similar height, and the house was said to have been built of red brick. A letter from her agent in 1665 describes marble steps, a carved staircase, wainscoted apartments, a columned frontispiece, and rooms carefully locked up to stop the newly fitted glass from breaking. The Duchess of Ormond died in 1684, and the decline was swift. By 1709, an observer recorded that the house had been pulled down, or at least dismantled, its furniture and pictures removed to Kilkenny Castle.