Rubel was born in western Ukraine and was educated in law and philosophy in Vienna and Chernivtsi National University, the town of his birth and was influenced by the Austro-Marxist Max Adler. He moved to France in 1931 to study sociology at the Sorbonne, from which he received his Licence-ès-lettres in 1934. He became a French citizen in 1937, and shortly after began publishing the literary magazine Verbe-Cahiers humains, before being drafted into the French Army. Due to his Jewish origins, Rubel lived semi-secretly in Paris under the German occupation of France.
In his encounters with Marxist members of the resistance movement in this milieu Rubel was reputedly astonished by the incoherence and confusion that surrounded Karl Marx and so-called "scientific" socialism. In difficult circumstances Rubel then set to work to gain a thorough understanding of Marx's life and work. It was Rubel who originally coined the term "marxologie" to refer to a systematic scholarly approach to the understanding of Marx and Marxism, which he saw as quite distinct.
After the war, Rubel continued with his research, first publishing on Marx in 1946, and receiving a Doctorat-ès-lettres from the Sorbonne in 1954. He joined the Centre d'études sociologiques in the Centre national de la recherche scientifique in 1947, and retired as Maître de recherche honoraire in 1970. Rubel published widely on Marx--more than 80 titles--and had a very active academic career. He combined controversial readings of Marx with rigorous scholarship, and was frequently polemical in his criticisms of the ideologies of "Marxism", which he often contrasted with a view of working class liberation that emphasised the "self-movement" of the working class above all else.
In the 1950s a political discussion circle gathered around Rubel, which eventually adopted the title Council Communist Group. It included the Vietnamese former Trotskyist Ngo Van. Van recalls that, in addition to a re-reading of Marx, Rubel introduced the group to other "incorruptible and pitiless judges of their era" such as Soren Kierkegaard and Friedrich Nietzsche, thinkers who espoused "new sets of values, new reasons for living, new norms for acting, a new ethic."