What makes the site unusual is not simply its ruined condition, but its origin: this Augustinian priory was founded, at least in part, as a refuge. Before the mid-eleventh century, the monks of Skellig Michael, whose monastery clung to a rock eight miles offshore in the Atlantic, abandoned it due to the hazardous conditions there and relocated to this coastal site. The priory continued to hold possession of the Great Skellig for centuries afterwards, appearing in the Papal Taxation Lists of 1300 under the name 'Redditus Prioris de Rupe Beati Michis', meaning roughly 'revenues of the Prior of the Rock of St Michael'. The famous island, in other words, was for a long time an outlying property of this now-forgotten shoreline community.

The priory was founded in 1210, or shortly afterwards, by the Arroasian Canons of the Order of St Augustine, a reformed congregation of Augustinian canons that had spread into Ireland from France via Arrouaise. It originated from Rattoo in north Kerry. The community survived well into the sixteenth century, still occupied by Arroasian canons in 1555 and possibly in use as late as 1569. By 1578, however, it had been disbanded; a fiant of that year granted a twenty-one year lease on 'the site of the late monastery of Canons of Ballinskilligge' to a Gyles Clinsher. The site then passed through several hands, to John Blake in 1585, then Richard Harding, and to the Sigerson family in 1615. The buildings span architectural details from the thirteenth to the fifteenth centuries, and include a nave-and-chancel church, the remains of a cloister, a refectory, and the relatively well-preserved Prior's House. A ruined structure noted on Ordnance Survey maps as a castle stood 13 metres south-west of the refectory; its foundations were still intact in 1841 but by 1902 only its north-west wall remained.

The sea has been the site's most persistent adversary. Coastal erosion documented since at least the eighteenth century has destroyed several buildings and much of the south-eastern side of the monastery and its graveyard. A nineteenth-century observer recorded the angle of a building with the tide washing around it, located just beyond the sacristy wall; it was gone before 1902. A substantial sea-wall, reinforced with groynes, now holds the line. Inside the church nave, graves still fill the interior, many marked with reused architectural fragments. A limestone font recorded in an old photograph once stood near a now-vanished cross-wall; by 1938 it had been knocked half sideways, and today only a rough perforated block remains in its place.