The abbey was founded sometime in the 630s in what was once a great tract of uninhabited land in the Vicoigne Forest between the Scarpe and the brook called the Elnon, from which the monastery took its first name, Elnon(e) Abbey. The founder was Saint Amand of Maastricht, under the patronage of Dagobert I. The name of the saint eventually became applied both to the abbey and the village that grew up round it.
Apart from its considerable effect on the landscape, the abbey became a major centre of study during the Carolingian Renaissance. Notable members of the community included Milo of Saint-Amand, author of a Life of Saint Amand, and his nephew, Hucbald of Saint-Amand, a noted music theorist and composer.
The abbey was totally destroyed by the Normans at the end of the 9th century. Although rebuilt, it was frequently destroyed by fire and the incidents of war, and was not completely restored until the 17th century, to an ambitious and much-admired plan implemented by Abbot Nicolas du Bois. In 1616-1617 Peter Paul Rubens painted a new high altarpiece for the monastery church, the Saint Stephen Triptych.
In 1672, Dom Mabillon discovered that, at the end of a manuscript of works of Gregory Nazianzen, there is a praise poem of the late 9th century in Old German, the Ludwigslied, which commemorates the victory of the Frankish army of Louis III over the Vikings on 3 August 881 at the Battle of Saucourt-en-Vimeu. The same manuscript, now held at the municipal library of Valenciennes, was found to contain one of the earliest literary texts in vernacular French, the poem called Sequence of Saint Eulalia. The Annales sancti Amandi, a set of annals of the Frankish kingdom, also originate from Saint-Amand.
The abbey was declared national property in 1789, and mostly demolished between 1797 and 1820. The former courthouse (échevinage) and the exuberantly decorated church tower, which now accommodates a faience museum, survive and can still be visited.