William was a son of William III, Count of Nevers and Ida of Sponheim, and the older brother of his successor Guy, Count of Nevers. Their paternal grandparents were William II of Nevers and his wife Adelais. Their maternal grandparents were Engelbert, Duke of Carinthia and Uta of Passau.
A younger brother named Renaud of Nevers joined the Third Crusade and died in Acre on 5 August 1191. Their sister Adelaide of Nevers, married Renaud IV, Count of Joigny. Ermengarde of Nevers, another sister, is only mentioned in documents recording her donations to the Benedictine monastery of Molesme.
In 1164, William married Eléonore de Vermandois, later Eléonore, Countess of Vermandois in her own right from 1183 to 1214. His wife was a daughter of Raoul I, Count of Vermandois and his second wife Petronilla of Aquitaine, who was a daughter of William X of Aquitaine and Aenor of Châtellerault; therefore Eléonore was a niece of Eleanor of Aquitaine.
His wife was previously married to Godfrey of Hainaut, Count of Ostervant, son of Baldwin IV, Count of Hainaut and Alice of Namur, who died on 7 April 1163, while preparing for a journey to Palestine. Eléonore went on to marry Matthew of Alsace, Matthew, Count of Boulogne and (possibly) Etienne II of Blois. She never had children and her designated heir to her realms was Philip II of France, a paternal second cousin, once removed.
William was knighted in 1159, only two years prior to the death of his father. He and his brothers are considered to have been quite young at the time of William III's death. His younger brother Guy was still mentioned as underage in 1164. William IV, Count of Nevers, resided in the chateaux of Nevers and of Clamecy (present day department of the Nièvre, Burgundy, France). The next nearest town to the East of Clamecy is Vezelay, which, in the early medieval period, was the marshalling point for the start of several crusades to the holy land.
According to the Catholic Encyclopedia, Vézelay Abbey was often in conflict with the counts of Nevers. William IV had his provost Léthard force the monks to take flight and abandon the abbey. In 1166, Louis VII of France arranged a reconciliation between William IV and Guillaume de Mello, abbot of Vézelay. On 6 January 1167 (Epiphany), Louis VII attended the celebration over the reconciliation. In atonement for his supposed crimes against the church, William set out for the Crusader states. In 1168, William of Tyre records the arrival of the Count of Nevers in Jerusalem. In the meantime, Amalric I was preparing for the invasion of Egypt. However, William died shortly afterwards, and he was buried in Bethlehem. Later on, most of his knights participated in Amalric's campaign, and were probably responsible for the massacre of the population of Bilbeis.
Before his death in 1168, he promised the bishop of Bethlehem that if Bethlehem should ever fall into Muslim hands, he would welcome him or his successors in Clamecy. After the capture of Bethlehem by Saladin in 1187, the bequest of the now deceased count was honoured and the Bishop of Bethlehem duly took up residence in the hospital of Panthenor, Clamecy, which remained the continuous in partibus infidelium seat of the Bishopric of Bethlehem for almost 600 years until the French Revolution in 1789.