The son of eminent dermatologist Sir Archibald Gray KCVO, John Gray was born in London and attended Cheltenham College before studying medicine at Clare College, Cambridge and University College Hospital. He began physiological research into the conditions faced by military personnel in battle conditions at the MRC's Armoured Fighting Vehicle Training School in 1943. After a spell at the National Institute for Medical Research, he became a surgeon-lieutenant in the Royal Navy during World War II, researching the physiological effects on personnel working inside tanks and naval gun turrets. He returned to the National Institute for Medical Research from 1946 until 1952 to work on neurophysiology, then became professor of physiology at University College London, where he researched sensory systems.
He became second secretary at the MRC in 1966 before succeeding Harold Himsworth as secretary in 1968. He was made a Fellow of the Royal Society in 1972, and was knighted in 1973. After finishing his term as secretary of the MRC he returned to research, working on neurophysiology of the sensory system in fish at the Marine Biological Association Laboratories in Plymouth.
In 1946 he married Vera Mares, who died the year before him in 2010. They had a son and a daughter.