What makes the site quietly puzzling is that when the Down Survey mapped this part of Ireland in 1655 and 1656, no house was recorded here at all, neither in the written survey nor on the barony map of Gowran nor the parish map of Graige. The building that now stands in ruin apparently did not yet exist, or had not registered as worth noting, when those surveyors were at work.

The house was almost certainly built in the latter half of the seventeenth century, after the Down Survey was completed. The Down Survey itself was a remarkable undertaking ordered by Oliver Cromwell's administration to map confiscated Irish land, and it recorded that in 1640 the proprietor of Ballyogan was Sir Edward Butler. Whether the Butlers, one of the great Anglo-Norman dynasties of Kilkenny, had anything to do with the eventual construction is not stated, but the name Galmoy connects to a Butler title, the Viscounts Galmoy, which lends a plausible thread. The structure as built was a rectangular three-storey house, roughly eight metres east to west and six and a half metres north to south, composed of roughly coursed sandstone with traces of plaster render surviving on both internal and external faces. The chimney arrangement is particularly telling about the building's domestic ambitions: an external chimney projection over three metres long dominates the east wall, a large ground-floor fireplace nearly two and three-quarter metres wide sits centrally in that same wall, and further fireplaces appear on both the first and second floors. String course trim, a horizontal decorative band of moulded stonework, once ornamented both gable stacks. These are the details of a household that expected comfort and a degree of formality.

The south and east walls survive to their full original height, while the upper-floor windows are poorly preserved, their flat-headed two-light openings partly collapsed. A nineteenth-century house has been built extending westward from the south wall, and modern farm buildings press against the east end of the same wall, so the ruin has been absorbed into a working farmyard rather than standing isolated in open ground. The surrounding hills close in the views, giving the whole site a contained, inward quality.