Born in Aaronsburg, Pennsylvania, Stover completed preparatory studies at Bellefonte Academy. He studied law. He was admitted to the bar in 1857 and commenced practice in Bellefonte, Pennsylvania. Held several local offices. He served as district attorney of Center County 1860-1862. Enlisted in the Union Army in 1861 as a private, and was successively a captain and a major. In May 1864, he was commissioned as colonel of the 184th Pennsylvania Infantry. He after the war moved to Versailles, Missouri, and resumed the practice of law. He served as district attorney of Morgan County from 1866 to 1868.
Stover was elected as a Republican to the Fortieth Congress to fill the vacancy caused by the resignation of Joseph W. McClurg and served from December 7, 1868, to March 3, 1869. He was not a candidate for renomination in 1868. He resumed the practice of his profession. He engaged in the real estate business and was also interested in mining pursuits at Versailles, Missouri. He served as delegate to the Centennial Exposition at Philadelphia in 1876. He was an unsuccessful candidate for election in 1876 to the Forty-fifth Congress. He died at Aurora Springs, Missouri, October 27, 1889. He was interred in the City Cemetery, Versailles, Missouri.