Built around 1490 by Domhnall MacConmara, this multi-storey fortification features a distinctive wicker vaulted basement, a construction technique that involved weaving branches or timber to create a centring for the stone arch above. The castle changed hands multiple times over the centuries; by 1570 it had passed to the O'Briens, though the MacNamara clan had reclaimed it by 1618, carrying out substantial renovations that included adding fireplaces, chimneys, and larger windows during the following three decades.

The castle's turbulent history continued through the Cromwellian wars of the 1640s, after which it was granted to Philip Bigoe, one of its last recorded owners. Today, three of the castle's walls remain standing at full height, though they're completely covered in ivy from basement to chimney level, obscuring most architectural features. The northeast wall collapsed in 1981, and its rubble now lies scattered down the rock outcrop, a mass of random stone and roughly hewn limestone covered in thick moss. Historic Ordnance Survey maps from the 19th century already depicted the structure as 'Milltown Castle (in ruins)', suggesting its decline began long before the modern era.

Access to the castle is extremely limited and potentially dangerous; the collapsed northeast wall has created a barrier that nearly seals off the basement just below the apex of the pointed wicker vault, leaving only a narrow gap through which visitors can glimpse the interior. The southwest basement window has been stripped of all its cut stone, whilst the northwest window's surround is completely hidden beneath the ivy. Despite its ruined state, the castle remains an evocative reminder of the medieval tower houses that once dotted the Irish landscape, serving as both defensive structures and symbols of power for the Gaelic and Anglo-Norman families who controlled them.