He was born into a Boston family as one of six boys with one sister; his brothers were David Coit Scudder and Samuel Hubbard Scudder. He graduated from Boston Latin School in 1854 and from Williams College in 1858, taught school in New York City, and subsequently, returned to Boston and devoted himself to literary work.
He is now best known for his children's books and the editorship he held of The Atlantic Monthly. He published the Bodley Books (1875-87) and was also an essayist, and produced large quantities of journalism that was printed anonymously. He was a correspondent of Hans Christian Andersen and biographer of James Russell Lowell. He edited The Riverside Magazine for Young People (1867 to 1870), where several Andersen fairy tales were published for the first time.
Scudder also prepared, with Mrs. Taylor, the Life and Letters of Bayard Taylor (1884) and was series editor for the extensive "American Commonwealths Series" as well as the "Riverside Literature Series" for Houghton Mifflin, where he also worked as literary advisor for several years.
Scudder may have been most famous for his 1884 work A History of the United States of America Preceded By a Narrative of the Discovery and Settlement of North America and of the Events Which Led to the Independence of the Thirteen English Colonies for the Use of Schools and Academies, which long set the standard for American history textbooks.