Stone steps cut into the exterior of the south gable lead down to its entrance, and a finely carved blocking stone guards what lies within. Above ground, a four-storey tower thick with ivy rises from the west end of the roofless nave, its defensive proportions sitting at an odd angle to the modest rubble walls of the church beside it. The whole ruin sits quietly in the northwest corner of a graveyard, barely 190 metres from the village's market square, yet it carries several centuries of accumulated complexity in its fabric.

The settlement now called Castlepollard was known in the medieval period as Rathyoung, a name visible on the Down Survey map of Rathgraff parish from 1659. The castle built there by Nicholas Pollard gave the town the name it carries today. The church on this site may have deeper roots still; according to nineteenth-century scholarship, the earlier church of Killafree, possibly associated with a monastery, once stood here. A datestone set into the south wall records that the building was constructed by Walter Pollard of Castlepollard in 1679, the same year that also appears on a datestone in the transept chapel above the family vault. Whether the 1679 structure replaced the medieval church entirely or absorbed some of its fabric is not certain. In 1676, the Union of Rathgraffe had merged several local parishes, including Rathgraffe, Faughalstown, Mayne, and St Fechin's of Fore, into a single administrative unit, and the Pollard church served this combined congregation until the arrangement became strained. By 1820, the building was described in print as too small and too old for its congregation, and a new church was already under construction elsewhere in the town.

What remains today is a rectangular nave with a semi-circular apse at the east end, walls of uncoursed rubble limestone with rough undressed quoins, and blocked or grilled window openings that record successive alterations across the eighteenth and nineteenth centuries. In one section of the north wall, a later round-arched opening cuts directly across an earlier flat-headed arch still visible in the stonework, a small collision of phases that the ivy has not yet covered. The transept chapel on the south side retains a cut stone pointed doorway with punch-dressed chamfered jambs, though it is now closed off with a metal grill. The interior of both the church and the tower are inaccessible, but the exterior repays careful attention: a string course hinting at a former wall-walk can be glimpsed at the northeast angle of the tower where the ivy has partly fallen away.