Melville was the third son of Sir John Melville, laird of Raith in the county of Fife, who was executed for treason in 1548. One of his brothers was Robert, 1st Baron Melville of Monimail (1527-1621). James Melville in 1549 went to France to become page to Mary, Queen of Scots. Serving on the French side at the Battle of St. Quentin in 1557 Melville was wounded and taken prisoner. He subsequently carried out a number of diplomatic missions for Henry II of France. On Mary's return to Scotland in 1561 she gave Melville a pension and an appointment in her household, and she employed him as special emissary to reconcile Queen Elizabeth to her marriage with Henry Stuart, Lord Darnley. After the murder of Darnley in February 1567, Melville joined Lord Herries in boldly warning Mary of the danger and disgrace of her projected marriage with Bothwell, and was only saved from the latter's vengeance in consequence by the courageous resolution of the queen. During the troubled times following Mary's imprisonment and abdication Melville conducted several diplomatic missions of importance, and won the confidence of James VI when the king took the government into his own hands.
Melville was knighted at the coronation of Anne of Denmark on 17 May 1590, and entered her service as a Gentleman of the Chamber. In his memoir he records how he overcame her initial suspicions of him.
Having been adopted as his heir by the reformer Henry Balnaves, he inherited from him, at his death in 1579, the estate of Halhill in Fife; and he retired there in 1603, refusing the request of James to accompany him to London on his accession to the English throne. By his wife, Christina Boswell, he had one son and two daughters; the elder of these, Elizabeth, who married John Colville, de jure 3rd Baron Colville of Culross, has been identified with the author of a poem published in 1603, entitled Ane Godlie Dreame. Sir James Melville died at Halhill on the 13 November 1617. He was buried in Collessie churchyard.
At Halhill, Melville wrote the Memoirs of my own Life, a valuable authority for the history of the period, first published by his grandson, George Scott of Pitlochie, in 1683, from a manuscript discovered at Edinburgh Castle in 1660. The most complete edition of the Memoirs is that prepared by Thomas Thomson for the Bannatyne Club (Edinburgh, 1827), based on a manuscript discovered in 1827. Some eighteenth-century Scottish historians doubted the authenticity of Scott's publication. Gordon Donaldson notes in Scott (1683) some editing errors and suppression of the more sinister dealings of English government before Mary's condemnation.