During 14th and 15th centuries, the cape that would become Vũng Tàu was a swamp which European trading ships visited regularly. The ships' activities inspired the name Vũng Tàu, which means "anchorage". French colonizers called it "Cap Saint Jacques" ("Cap Xanh Giắc", in Vietnamese). The cliff of Vũng Tàu is now called Mũi Nghinh Phong (literally meaning "Cape of breeze welcome").
Vũng Tàu was originally referred to as Tam Thắng ("Three Boats") in memory of the first three villages in this area: Thắng Nhất, Thắng Nhị, Thắng Tam, within the province of Biên Hòa under the Nguyễn Dynasty. Under the reign of king Gia Long (1761–1820), when Malay pirates built a base here and subsequently became a danger to traders in Gia Định city, the king his army to crack down on the pirates. The pirates were ousted and the troops were given the land as a reward. 10 February 1859 marked the first use of cannons by Nguyễn's army, when they fired at French battleships from the fortress of Phước Thắng, located 100m from Vũng Tàu's Front Beach. This marked an important period in Vietnam's war against French invaders in South Vietnam (then called Cochin china). In 1876, according to a decree by the French colonialists, Vũng Tàu was merged in Bà Rịa county per Saigon's administration. During the 1880s there talks about moving Saigon's port facilities to Vũng Tàu, but this came to nothing due to Saigon's better infrastructure.
On 1 May 1895, the governor of Cochin china established by decree that Cap Saint Jacques would thereafter be an autonomous town. In 1898, Cap Saint Jacques was merged with Bà Rịa county once again, but re-divided in 1899. In 1901, the population of Vũng Tàu was 5,690, of which 2,000 persons were immigrants from North Vietnam. Most of the town's population made their living in the fishing industry. On 4 April 1905, Cap Saint Jacques was made an administrative district of Bà Rịa province. In 1929,
