Though it appears on the 1838 Ordnance Survey map as a T-shaped building with formal gardens stretching to the south, no trace of it remains visible today. The map shows it as a rectangular structure running east to west, with a small projection extending northward from its northern side.

This wasn't just any castle; it was the official residence of the Protestant Bishop of Killala and Achonry. Records show the building was already standing in 1646 when defensive walls were constructed around it, likely a response to the turbulent times. The palace weathered considerable damage during the Williamite Wars, but Bishop William Lloyd undertook extensive repairs in 1691, restoring both the See-house and its gardens which had been, in his words, 'miserably despoiled during the wars under James II'. Earlier repairs are also documented from 1663, suggesting the building required regular maintenance throughout its lifetime.

There's some historical ambiguity about the site's earlier history. Local historian Knox mentions a 16th-century castle in Killala owned by Richard Barrett, though it remains unclear whether this refers to the same building that later became the bishop's palace or an entirely different structure. What is certain is that by the time later Ordnance Survey editions were produced, the palace had vanished from the maps, its stones likely repurposed for other buildings in the growing town.