Historical maps provide the best evidence of its existence; the Down Survey maps from 1656-8 show a castle positioned near the parish boundary between Mullary, Priestowne and Tynure. By the time Taylor and Skinner surveyed the area in 1777-8, the structure was already recorded as being in ruins, sitting on the western side of the road to Ardee.

Today, the castle exists more in local memory than in stone. The area known as 'Castle Field' preserves the name, if not the structure, of this lost fortification. During nineteenth-century ploughing, farmers reportedly uncovered sections of paving, suggesting the presence of substantial buildings or courtyards. However, despite these tantalising clues, the exact location of the castle remains uncertain.

This disappearance of Castlelumney reflects a common fate for many of Ireland's smaller castles and tower houses. Without continuous occupation or maintenance, these structures gradually succumbed to weather, stone robbing for local building projects, and the relentless march of agricultural improvement. What remains is a puzzle for local historians; a castle important enough to appear on multiple historic maps, yet ephemeral enough to leave barely a trace on the modern landscape.