St Selskar's may derive from a corruption of 'Sepulchre', connecting it to the Holy Sepulchre tradition common among Augustinian houses, or it may come from the Norse word 'skar', meaning a rock, which would root the name in the Viking settlement that once shaped this part of the town. Neither origin has been settled definitively, and the uncertainty suits a site that has accumulated layers of use, ruin, and reuse across roughly eight centuries of continuous occupation.

The priory was founded by the Roche family as a house for the Augustinian Canons Regular, a religious order following the Rule of St Augustine, and it was certainly established by 1240. There are grounds, however, for thinking it older still. The curved alignment of the surrounding streets and the nearby town wall suggests the site may predate the Norse and Norman presence in Wexford entirely, pointing to an early Christian foundation later absorbed into the medieval monastic system. The abbey was suppressed in 1541 under the Dissolution, though by that point it had long been serving the parish of St Selskar rather than functioning as a priory in any strict sense. A visitation in 1615 by Thomas Ram, the Protestant bishop of Ferns, found a curate named David Browne in post; the chancel was reportedly in reasonable repair, but the rest of the church was not. A Church of Ireland building was erected just to the east of the medieval structure in 1818 in a neo-Gothic style, but that too is now roofless.

What survives is genuinely varied. The medieval church has a double nave, probably thirteenth century in date, with an arcade of four pointed arches dividing the two aisles; the western windows, now mostly destroyed, were built in the decorated Gothic style and still show their glazing grooves. The later fortified tower, standing 15.1 metres to the parapet, is detailed in unusual ways: its quoins are of dressed Dundry stone, a limestone quarried near Bristol and frequently imported to Ireland's east coast during the medieval period, while the dressed stonework of the upper floors is granite. The vault on the first floor retains traces of wicker-centring, a construction technique in which woven wicker was used as a mould to shape the wet mortar of the arch. Notably, the tower has no fireplaces, no garderobes, and was never used as a belfry; it was purely a defensive structure. Among the monuments inside the church are a thirteenth-century graveslab, a seventeenth-century armorial crest, a seventeenth-century memorial, and a sarcophagus. The whole site sits within a small triangular graveyard bounded partly by the town wall itself, which forms the south-western edge of the enclosure.