Near Crawford's residence, Sir George Merbury founded what would become the market town of Letterkenny, initially comprising fifty thatched houses; thirteen of these had more substantial clay and stone walls, along with a single watermill. By 1654, the Civil Survey of Donegal painted a picture of a thriving settlement with a weekly Friday market, two annual fairs, and several notable structures including a large stone dwelling house protected by a bawn with four flanker towers, a church, and a bridge spanning the River Swilly at the town's edge.

The fortified house and its defensive bawn appear to have stood near the present site of Conwal Church of Ireland, with Castle Street originally connecting the church to this imposing structure. The castle likely occupied ground near the southern angle of Mount Southwell, where the modern Ordnance Survey maps still mark areas called Castle Gardens and Fort Well; telltale placenames that preserve the memory of this seventeenth century stronghold. A substantial two storey house now stands beside Castle Gardens, possibly built directly atop the castle's foundations, and surrounded by a large stone wall that may incorporate remnants of the original plantation fortifications.

These physical traces offer tantalising glimpses into Letterkenny's plantation era origins, when fortified houses like Crawford's served dual purposes as both grand residences and defensive structures in a newly colonised landscape. The evolution from a modest settlement of fifty thatched houses to today's bustling town began with these stone walls and flanker towers, markers of a turbulent period when comfort and security went hand in hand.