Born in New York City, he gained fame as a comic in vaudeville and claimed that he created two of vaudeville's more renowned pieces of business, "Floogle Street" (a.k.a. "Susquehana Hat Company") and "Slowly I Turned". In addition to an active career in vaudeville and the legitimate theater, he appeared in many movies and TV shows.
The Republic Theatre was the site of Faye's New York stage debut at age 21. During World War II, he entertained Allied military personnel in Africa and Europe as part of a troupe headed by Marlene Dietrich.
Faye played second banana (the second-ranking comedian in a show) to Phil Silvers in two Broadway shows, High Button Shoes and Top Banana. He also appeared in the 1954 film. In a Broadway career that stretched between the late 1930s and the early 1990s, he appeared in 17 shows altogether, including Room Service (his Broadway debut) and The Tender Trap. He also appeared in the 1955 movie adaptation), the 1965 revival of Guys and Dolls, and Neil Simon's musical Little Me.
He appeared as a guest in many TV shows from 1949 through 1984 and a series of short subject films, including Mack & Myer for Hire (1963), about two bumbling plumbers, who rode around in a motorcycle with a sidecar, attempting repairs, but producing chaos.