Johvi was first mentioned as a village in 1241 in Liber Census Daniae when it was ruled by Denmark. Historical names of Johvi were Gewi and Jewi. In the 13th century a church was built here and Johvi became the centre of the local church parish.
On 1 May 1938 Konstantin Pats renamed the Johvi borough a town along with nearly all Estonian boroughs. Up to 1991, Johvi was a district of Kohtla-Jarve. In 2005 the town of Jõhvi was united with the parish of Johvi
