Despite its obvious age, no founding charter survives, no town walls were ever built, and the boundaries of the medieval settlement remain, in the words of those who have studied it, indeterminate. What exists instead is a layered sequence of ownership and upheaval stretching back to the earliest decades of the Anglo-Norman presence in Ireland, with a castle on the hill and the ghost of a friary to show for it.

The land on which the town grew was once the caput, or administrative centre, of the manor of Duffry, a territory reaching westward from the River Slaney to the Blackstairs Mountains, roughly matching the old parish of Templeshanbo. Before 1172, Strongbow granted Duffry to Robert de Quincy, who had married Strongbow's sister Basilia. After Robert's death, Basilia married the Norman commander Raymond le Gros in 1175, and it is Raymond who may have thrown up the original earthwork castle on the site where the present stone structure now stands. The manor passed through several hands: to the de Prendergasts by marriage in 1198, then to the Rochfort family from 1251 onwards, though by 1326 the manor was recorded as destroyed by the Irish. The MacMurroughs appear to have held possession during the fifteenth century, and in 1460 Donal Reagh MacMurrough founded a Franciscan friary in the town. The Franciscans were an order of mendicant friars who typically established themselves in or near urban centres, and the friary's founding here points to Enniscorthy's significance as a local hub even in a period of contested control. Then in 1585 Sir Henry Wallop acquired the lease, set about rebuilding the town, and developed industries in timber and iron. He almost certainly gave the castle its current form, a structure modelled on Ferns Castle with corner towers, but smaller in scale and carrying the distinctive architectural details of the late sixteenth century.

The town spread unevenly across both banks of the Slaney, with the greater part on the steep western bank. Plans for a bridge were drawn up in the late sixteenth century, but one was not actually built until the end of the seventeenth. Archaeological investigation beneath the modern streets has turned up relatively little: a well, and two tanneries that may pre-date 1700. The identifiable medieval monuments that remain are the castle, the site of the Franciscan friary, the church of Templeshannon on the eastern bank, and the site of St Mary's parish church, each one a fixed point in an otherwise hazy urban outline.