It stands within a D-shaped graveyard enclosed by masonry walls, on a low triangular island at the northern end of a semi-saltwater lagoon on the Wexford coast. The lough, known as Lough Togher in the seventeenth century, is separated from the sea by a bar that salt water seeps through and occasionally breaches entirely, giving the whole setting an in-between quality, neither fully freshwater nor fully marine. The church itself reached the island by causeway, guarded at the mainland end by a tower house and a double curtain wall, which survives as a separate monument.

The site's older name offers a small puzzle. It was recorded as Insula Barry, which may be a corruption of Insula Mhuire, meaning Island of Mary, a connection that would align with the long Marian pilgrimage tradition here. Augustinian Canons, a religious order known for administering major shrines, are said to have regulated pilgrimages to the island from the fourteenth century. A plenary indulgence, a full remission of the temporal punishment due to sin, was granted to pilgrims in the early seventeenth century. Cromwell is said to have destroyed the shrine during the siege of Wexford in 1649, though no corroborating evidence has been found for this. What is clear is that by 1682 the pilgrimage was very much alive; Colonel Solomon Richards described it in that year as involving a full circuit of the island. It still takes place each August, centred on the Feast of the Assumption on the 15th. During conservation work on the surviving walls, four dressed stones came to light that may all belong to the same three-light cusped ogee-headed window, a late medieval style in which the arched heads of the lights curve outward before coming to a point, possibly the original east window of the church.

The pilgrimage route incorporated two holy wells: Our Lady's Well, about five hundred metres to the east on the mainland, and a second well on the western shore of the island, roughly seventy metres from the church ruins. Visitors approaching across the causeway pass the tower house before reaching the graveyard, which is still in use and surrounds the fragmentary church walls. The dressed window stones recovered during conservation give some sense of how elaborate the building once was, even if almost nothing above the west gable now survives.