Cousin to Horatio Nelson. Goulty had a keen interest on improving the education in the town and along with others founded Middle Street School, the Royal British School on Eastern Road and also a school for the fishermen's children at the bottom of West Street.
He was one of the four founders of the Extra Mural Cemetery in 1850, after Government Health Inspectors were concerned of the sanitary conditions of Brighton. They recommended that the burials in churchyards in the town should cease. Together with John Cordy Burrows, Amon Henry Wilds and his son Horatio Nelson Goulty they established the Brighton Extra Mural Company, acquiring 13 acres of land, kindly donated by the Marquis of Bristol.