Today, only grass-covered foundations remain of what was once a square tower house, its walls now reduced to mounds barely half a metre high. The 9-metre square foundations rest on a natural rock outcrop, surrounded by an extensive complex of earthworks that likely formed the original bawn and settlement. These defensive features are still visible from aerial photographs, hinting at the castle's former strategic importance overlooking the waters of Lough Ree to the north and west.

The castle's origins are somewhat disputed, though local tradition suggests it was first built in 1401 by Geróid Dalton, son of Lucas Dalton. The townland's name, Ballynacliffy, may derive from the Irish 'Baile na Cloiche', meaning 'town of the stone house', a fitting description for this once-formidable structure. By the 17th century, ownership had passed through various hands; the Ordnance Survey Letters attribute it to the Dillons, whilst records from 1641 show the Earl of Kildare as the landowner. The site's proximity to Bethlehem nunnery, just 1.8 kilometres north, placed it within a network of important medieval buildings in this part of Westmeath.

During the Irish Rebellion of 1641, the castle gained new significance as Ballineclossy Castle, serving as a garrison for English forces. Sir Henry Piers, writing in 1682, described it as 'a strong castle and dwelling house of a gentleman of antient family and good estate, situated in the promontory'. In 1642, two companies of English foot soldiers were quartered here under Captain Bertie, brother to the Earl of Lindsey. When the English forces arrived, they found the castle abandoned by its inhabitants but well stocked with provisions, including 'excellent strong beer and aquavitae'. From this strategic position jutting into Lough Ree, the garrison could control movements along the Shannon and maintain watch over the surrounding countryside, including nearby Portenure across the River Inny.