The castle appears as a rectangular building with an adjacent lime kiln on the 1840 Ordnance Survey map, but by 1897, only the north gable remained standing. The structure was completely levelled sometime between 1827 and 1840, leaving no visible traces above ground by the time antiquarian O'Kelly visited in 1943.

Historical records trace the castle's ownership back to at least 1251, when it was known as Catherussoc or Cathyryssok. In 1287, Thomas de Clare leased the property to H. son of John, and by 1302, Adam son of Philip held it. The castle changed hands several times throughout the medieval period; in 1325, Richard and Avelina de Midia were involved in legal proceedings concerning the property. By 1583, Maurice Mac Shane controlled both Ballinard Castle and the surrounding village, but the FitzGerald family had taken ownership by 1657 when Captain Garrott FitzGerald held the estate.

According to Lewis's historical accounts, the last structure on the site was built in the 15th century by the Geraldines, though the FitzGeralds maintained possession until late in the 18th century. The castle stood to the north of the local church and remained a prominent feature of the landscape until its demolition in the early 19th century. Today, while the physical castle has vanished, Digital Globe aerial photographs still reveal the ghostly outlines of its foundations in the pasture, a subtle reminder of nearly 600 years of Irish history.