The Canons Regular of the Augustinian Abbey of St Mary founded an abbey here in the mid 12th century. Unfortunately nothing now remains, but it is remembered in name only because the field west of the street is known as Abbey field. A mural tower, part of the town’s medieval wall, survives on the south of Canon Street.