It is, if the record is accurate, the last surviving physical remnant of the medieval castle of Castlegregory, a structure that was once considered notable enough to be called "famous" by those who remembered it.

By 1899, the castle had already vanished almost entirely. Writing in that year, Prendergast recorded that only one stone remained of the building, and that even this had been removed and reused. The culprit, or perhaps the unlikely preserver, was Archdeacon Rowan, who took the stone away and had it set into the wall of Edenburn House. The phrasing Prendergast uses, "we believe, inserted in the wall", carries a faint note of uncertainty, as though even in 1899 the exact location within the house was not quite confirmed. What the stone looks like, whether it bears any carved detail or moulding typical of medieval masonry, is not recorded. It survives less as an artefact than as a rumour in stone, absorbed into a domestic building and largely forgotten.