This imposing four-storey tower house, built by Thomas Comerford who died in 1588, stands as a remarkably well-preserved example of late medieval defensive architecture. The castle underwent significant remodelling under Thomas's son Richard, who married into the influential Purcell family and transformed the fortress into a more comfortable residence by inserting fashionable two-light mullioned windows and grand fireplaces in the upper chambers; clear signs of the transition from purely military stronghold to aristocratic home.

The tower house measures approximately 8.3 metres north to south and 10.9 metres east to west, with walls nearly two metres thick built from roughly coursed rubble and dressed quoinstones. Entry through the pointed doorway on the eastern wall reveals a sophisticated defensive system: visitors first encounter a lobby protected by a guardroom, then pass through a second lobby before accessing either the ground floor chamber or the spiral staircase in the southeast corner. The ground floor, lit only by narrow defensive slits, contrasts sharply with the upper halls where Richard Comerford's renovations are most evident; the second floor public hall and third floor private hall both feature large stone fireplaces and those distinctive paired windows that replaced earlier, narrower openings.

Following the confiscation of John Comerford's lands in 1656 and his transplantation to County Clare, the castle fell into decline, losing its roof by the 19th century. However, restoration work between 1979 and 2006 has preserved many original features, including the spiral stairs that wind up to the wall-walk, multiple garderobes cleverly built into the northeast angle, and even a secret mural chamber that likely served as the castle's treasury. The machicolation over the main entrance and the circular bartizan on the northwest corner remind visitors that despite its later comforts, Ballybur remained fundamentally a fortress, built to protect the Comerford family's interests in turbulent times.