Levin was born in Minneapolis, the son of Beatrice Hirshler (née Tuchman) and Isadore Henry Levin. His family was Jewish. Levin was educated at Harvard University (where he was a contemporary of M. H. Abrams). According to a biographical memoir by Walter Jackson Bate:
After graduating Summa Cum Laude in 1933 he was appointed Junior fellow in then new Society of Fellows, where he pursued in depth what were to become his three major interests: Shakespeare and the English Renaissance; modern literature generally; and the relation of English and American to other literatures, from Greek and Latin antiquity to the present, all of which are reflected in his early publications, giving him a perspective lacking in the ordinary specialist and scarcely matched in his later years by more than three or four scholars here or abroad. In the 1930s, junior fellows did not normally take a Ph. D., so that Harry, like his noted predecessor, George Lyman Kittrdge, remained an A.B., though he was in time to receive six honorary degrees, including ones from Oxford and the Sorbonne, and though he was, over the years, to supervise over ninety doctoral theses.
Levin began teaching at Harvard in 1939 and that same year he married Elena Zarudnaya. He was named Irving Babbitt Professor of Comparative Literature at Harvard in 1960 and retired in 1983. He continued to live near campus in Cambridge, Massachusetts, until his death in 1994. He was survived by his widow Elena and their daughter Marina.
Levin's course in "Comedy on the Stage" inspired Leonard Lehrman to write the paper, "The Threepenny Cradle," comparing the Brecht-Weill Threepenny Opera to Marc Blitzstein's The Cradle Will Rock. In the fall of 1969, in a production of Cradle directed by Lehrman, Levin was the sole patron. In 1970-1971 he encouraged, advised, and became a patron for two other Harvard productions by Lehrman: the U.S. premiere of Brecht's The Days of the Commune, and a triple-bill in memory of Blitzstein, which was attended by Leonard Bernstein. It was at that production that Levin invited Bernstein to become Norton Lecturer at Harvard, which he did, a year later.
In 1985, the American Comparative Literature Association began awarding the Harry Levin Prize for books on literary history or criticism and in 1997, Harvard University endowed the new chair (position) of Harry Levin Professor of Literature.