Beneath and around it, however, the ground holds considerably more: foundations, possible cellars, and the buried evidence of a building whose history stretches back to the medieval period and forward to a fire around 1940 that ended its life as a standing structure.

The site passed through some of the most prominent hands in Irish history. The castle at Westcourt, which may have taken the form of a tower house with a bawn, a walled enclosure attached to an Irish tower house for defence and livestock, was recorded as the old manor house of Callan, and passed from the descendants of Earl Marshall to the Earl of Ormond in 1391. It remained with the Ormond family until around 1700, when it passed to Lord Desart, and later to George Agar, Lord Callan, during the second half of the eighteenth century. The historian Carrigan, writing in 1905, relayed a suggestion from a Thomas Shelly that this was the strong Butler castle of Callan referenced during the Cromwellian siege of the town in 1650. By the time of the Down Survey of 1655 to 1656, a large house was already recorded in the townland then spelled 'Whitescourte', likely incorporating the earlier castle. Following the death of the owner John Galwey in 1793, Lord Callan demolished that older structure and replaced it with a mansion. That mansion survived until around 1940, when a catastrophic fire left it abandoned and, eventually, demolished.

Archaeological excavations carried out in 2000 probed the vicinity of the house and found scorched mortar and slab flooring tentatively associated with the medieval phase of the building. More striking was a wall-footing roughly 1.5 metres wide, built with a batter, the slightly sloping face typical of defensive masonry, and accompanied by an adjacent fosse, or ditch. The excavator interpreted these as possibly the remains of the bawn surrounding the original medieval castle, suggesting that despite centuries of rebuilding and eventual destruction, something of the earliest structure still survives, just out of sight.