Local tradition holds that a castle once stood atop this prominent eminence, though no visible traces remain today to confirm these stories. The site's strategic position would have made it an ideal location for a medieval fortification, overlooking approaches from multiple directions.

Archaeological surveys conducted in the 1980s and early 1990s found no surface evidence of the rumoured castle, yet the persistence of local memory suggests something significant once occupied this spot. Adding weight to the area's historical importance, archaeologists have identified what appears to be a souterrain nearby; these underground passages were typically associated with early medieval settlement sites throughout Ireland, used for storage, refuge, or both.

The site was first documented by Corcoran in 1932, who recorded the local traditions surrounding the supposed castle. While the physical structure may have vanished, whether through stone robbing for later buildings or simple decay, Carrick Philip remains an intriguing reminder of how landscape and memory intertwine in Irish local history. The combination of the prominent hilltop location, persistent oral tradition, and the presence of the nearby souterrain all point to this area having played some role in the medieval landscape of County Louth, even if the exact nature of that role remains tantalisingly uncertain.