This substantial fortification measures 44 feet by 35½ feet at its base, with walls an impressive 7 feet thick; a testament to the turbulent times in which it was built. The four-storey battlemented tower has witnessed centuries of Irish history, from medieval murder to Civil War siege.

The castle's early history is marked by tragedy. In 1487, James, the ninth Earl of Desmond, met his untimely end here when he was murdered by his own servants within these very walls. The property, variously recorded as Caslan Mattras, Courtmattress, or Matrascourte, remained significant throughout the Tudor period. By 1580, it was held by one N.f. William, and following the Desmond Rebellions, the Crown granted it to Billingsley in 1588. The castle changed hands again in 1608 when Sir J. Dowdall of Pilltown received it along with the Courts-Barron and the nearby abbey.

Through the centuries, Castle Matrix continued to play its part in Ireland's story. The Southwell family held it from at least 1616, and during the Confederate Wars it endured a siege in 1642 by M. Herbert before surrendering. By 1655, Edmund Southwell maintained the castle along with its orchard and grist mill on the Deel. The tower underwent repairs in the 1840s under J. Browne's ownership, preserving this remarkable Desmond stronghold for future generations to explore and appreciate.