He was a son of Berthold II, Duke of Carinthia, and a brother of Berthold II, Duke of Swabia. For some time, he was provost of Kanten, then entered the Benedictine monastery in Hirschau and, on 22 December 1084, was consecrated Bishop of Constance by the cardinal-legate, Otto of Ostia, the future Urban II. The see of Constance was then occupied by the imperial anti-Bishop Otto I, who, though excommunicated and deposed by Pope Gregory VII in 1080, retained his see by force of arms. At an imperial synod held in Mainz, in April, 1085, Gebhard and 14 other German bishops who remained faithful to Gregory VII were deposed, and Otto I was declared the lawful Bishop of Constance. However, the latter died in the beginning of 1086, and Gebhard was able to take possession of his see. One of his first acts as bishop was the reform of the Benedictine monastery of Petershausen near Constance, which he recruited with monks from Hirschau. In 1089, he consecrated the new cathedral of Constance, to replace the old one which had fallen into ruins in 1052.
On 18 April 1089, Pope Urban II appointed him and Bishop Altmann of Passau Apostolic-vicars for Germany. Arnold, a monk of St. Gall, whom Henry IV, Holy Roman Emperor appointed anti-Bishop of Constance on 28 March 1092, tried in vain to eject Gebhard from the See of Constance. The latter was brothers with Bertold II, Welf I, Duke of Bavaria, and was known among the monks of Hirschau and Petershausen, and the citizens of Constance. In 1094, Gebhard held a synod of reform at Constance, and in 1095, he attended the Council of Piacenza. In the same year, he participated in the consecration of Arnulf III, Archbishop of Milan. Soon, however, the influence of Henry IV began to increase in Germany. In 1103, Gebhard was driven from his see, and the imperial antibishop, Arnold, usurped the bishopric. With the assistance of Henry V, Gebhard regained his see in 1105, freed the king from the ban by order of Pope Paschal II, and accompanied him on his journey to Saxony. Gebhard attended the Synod of Nordhausen on 27 May 1105, the diet at Mainz on Christmas, 1105, was sent as imperial legate to Rome in the spring of 1106, and was present at the Council of Guastalla in October of the same year. In the fresh dispute that arose between Paschal II and Henry V, Holy Roman Emperor, Gebhard seemed to side with the emperor, but, after being severely reprimanded by the pope, withdrew from public life, and devoted his whole attention to the welfare of his diocese.