The name is a corruption of a working term: a fish palace, or pilchard palace, was an industrial facility where pilchards were pressed for their oil, a common feature of the southern Irish and Cornish coastlines during the centuries when pilchard shoals ran reliably through these waters. The pressing process involved stacking the fish in stone or timber vats under weighted beams, extracting the train oil used for lamps and leather-curing, and packing the remaining fish in barrels for export. At Scilly, even that utilitarian infrastructure has largely vanished. There is no visible surface trace of the pilchard press recorded here.

What survives, at least in partial ruin, points to two phases of industry in the same small area. A map surveyed in 1842 for the Ordnance Survey at six-inch scale shows two parallel rectangular structures oriented roughly north-east to south-west within the area called the palace, and a ruined residential structure on the ground may be the remains of those buildings. Immediately to the south-west stands something more legible: a roofless two-storey structure of eleven bays, oriented north to south, with a date plaque reading 1840. This is known locally as the fish factory, and its scale, eleven bays across a single facade, suggests a building of some ambition for its time, even if its ambition was purely commercial. The 1840 date places it in the period when Kinsale's fisheries were still a significant local industry, before the collapse of the pilchard fishery later in the nineteenth century reduced much of this coastal infrastructure to the condition it is in today.