A cluster of earthworks, roughly 150 metres by 180 metres in extent, once spread out to the north, east, and south-east of Ballydoyle Castle. They were the physical remains of a small community that had grown up around a tower house, probably from the fifteenth or sixteenth century through to the seventeenth. In the 1960s, the earthworks were levelled, and the ground was returned to farmland.

The Civil Survey of 1654 to 1656 offers a vivid, if brief, picture of what once stood here. The surveyors recorded that in 1640 the lands of Ballydwille held a castle, about a dozen cabins, and an orchard, all under the proprietorship of James, Lord Baron of Dunboyne, described in the survey's characteristically blunt terminology as an "Irish Papist." That designation reflects the confiscatory logic of the Cromwellian period, during which Catholic landowners across Ireland were being catalogued with a view to dispossession. The Dunboyne Butlers were a prominent Catholic noble family, and this snapshot of their Tipperary holding, a modest tower house surrounded by tenant cabins and a working orchard, is one of the clearer documentary glimpses of how such settlements were actually organised in early modern rural Ireland. A tower house, in this context, was a fortified stone residence of the kind built widely across Ireland between the fourteenth and seventeenth centuries, typically by lords of middling rank, with a small dependent community clustered nearby for labour and mutual protection.

Before the earthworks were cleared in the 1960s, a rapid survey was carried out and published by Rynne in 1966, preserving at least some record of what was there. That survey is now the principal evidence for a settlement that the landscape itself no longer shows.