Hervey wasted little time establishing his presence in the region, founding the Cistercian abbey of Dunbrody just 1.6 kilometres east of the island between 1171 and 1175. The island itself became the heart of a thriving manor, with detailed accounts from the 1280s and 1290s revealing a well-managed estate that included regular repairs to a castle and the digging of a defensive moat.

The remains of what may be Hervey's original ringwork castle can still be traced on Great Island today. An impressive arc of earthen bank curves from north through east to southeast, best preserved on the eastern side where it stands 11 metres wide at its base and rises 1.2 metres on the interior. Outside this bank runs a fosse, or defensive ditch, measuring 12 metres across at its widest point. If the defensive circuit were complete, it would have enclosed a D-shaped area roughly 250 metres east to west and 200 metres north to south, though much of this would have been positioned on the steep slope facing the river.

Great Island itself has changed dramatically since medieval times. Once a true island measuring approximately 2.5 kilometres north to south and up to 1.8 kilometres east to west, it sat in the River Barrow/Nore with water channels on all sides. The channel that once flowed around the northeast and eastern edges, spanning 500 to 600 metres wide, has long since silted up and been reclaimed as farmland. Within the old enclosure, local tradition places two castle sites and a leper hospital, whilst another ringfort lies about 100 metres to the southeast, all testament to the island's long and layered history.