Martin played guitar in his youth before settling on tuba as his main instrument. He played with Papa Jack Laine's Reliance Brass Band around 1910, and worked in various other brass bands in the city in the 1910s. In 1923, he traveled to Chicago and played with the New Orleans Rhythm Kings with whom he made his first records. He also recorded guitar duets with Leon Roppolo, but these unfortunately were never issued. He returned to New Orleans with the Rhythm Kings in 1925, and made further recordings with them. He also played with the Halfway House Orchestra (with which he recorded on both tuba and string bass), the New Orleans Harmony Kings, and the New Orleans Swing Kings. In the 1930s, Martin worked as a staff musician at WSMB radio. He continued to play tuba for his entire career, though he also played and recorded on the double-bass (like many New Orleans tubists) from at least the 1920s onward. He played with dozens of noted New Orleans jazz musicians, appearing on record with Sharkey Bonano, Santo Pecora, Pete Fountain, Al Hirt, and others, and released one album under his own name on Southland Records in 1963.
Martin's son, Martin "Little Chink" Abraham, is a jazz bassist.