Kleist was born on a farm in the Town of Eaton, Manitowoc County, Wisconsin on September 29, 1860. Six years later his parents moved to the Town of Rantoul in Calumet County. He attended public schools as a boy in the winter, working on the farm in summer, and later worked in the woods in winter. After the 1888 death of his father, he and his brother, Charles, operated the homestead farm, now known as Kleist Brothers' Grain and Dairy Farm, and continued to live with their mother. By 1918, he was president of the Calumet County branch of the American Society of Equity (at that time "the greatest farmer organization that the State had ever known") and the Farmers Advancement Association of Brillion. He had a long history of activism in both farmers' organizations. He was also a member of the Brillion Masonic and Odd Fellow lodges.
In 1918, he was elected to the State Senate as a Socialist for the 15th district (Calumet and Manitowoc counties) to succeed Henry Rollman, a Democrat. Kleist received 4,393 votes to 3,611 for Democrat Leo P. Fox. He was assigned to the standing committees on the judiciary, and on "Reconstruction and Readjustment".
In 1920, he was the Socialist nominee for Lieutenant Governor of Wisconsin, losing to Republican (and fellow dairy farmer) George Comings. As a result of the poor performance of the Socialist candidates in that election, he found himself Wisconsin's sole non-Milwaukee Socialist legislator from 1921 to the end of his term.
The Senate was redistricted after the 1920 census, with Calumet and Manitowoc counties split between the 19th and 1st districts; Kleist did not run for re-election in either of these new districts. In the new 15th District (Rock County), he was succeeded by Alva Garey, a Republican.