Built in 1128 and the years following, Kelso Abbey was one of the finest examples of Romanesque architecture. Finally finished, it was dedicated to the Blessed Virgin and St John in 1243. It was soon one of the largest and richest in Scotland, having a superb library in medieval times. In the 12th and 13th centuries, the Abbot of Kelso was granted the right to wear a mitre, which gave him a precedence higher than any other Scottish abbot. The abbey's wealth came from its vast lands, its churches, schools, farms and its granges in the Cheviot Hills.
James III was crowned in the Abbey, and Prince Henry, son of David I, was buried there in 1152.
With the Reformation, and Henry VIII's determination to wipe out the Border Abbeys, by 1550 Kelso Abbey along with those at Melrose, Dryburgh and Jedburgh had been reduced to rubble by the English forces under Hertford. Despite attempts at rebuilding, all the abbeys went into decline, and soon few, if any, monks remained. A Parish Church used part of the site about 1650 with a new school alongside. This seems to have continued in use until about 1770, with the abbey ruins being used as a source of ready-hewn stone for buildings elsewhere in the town.