Palmer began his career on tuba, playing with Oliver Cobb late in the 1920s; Cobb's group recorded for Paramount and Brunswick. From 1931-34 he played with Eddie Johnson, then joined Dewey Jackson until 1941. Following this he played sporadically with George Hudson up to 1948.
Toward the end of the 1940s, Palmer began to get higher-profile performing and recording dates, including with Clark Terry in 1947 and Jimmy Forrest in 1948. Thereafter he played in Count Basie's orchestra. Additionally, he played bass on record with blues musicians such as Big Joe Williams and Sonny Boy Williamson.
In 1950 he left Basie's group and started his own band, the Dixieland Six; Robert Carter was the trombonist for the sextet. This Dixieland jazz ensemble continued playing in St. Louis into the 1980s. Palmer became a source for jazz historians late in his life, offering oral history testimonies of his early years in the music industry.