According to the Landnámabók (the book of settlement), Skutulsfjördur was first settled by Helgi Magri Hrólfsson in the 9th century. In the 16th century the town grew as it became a trading post for foreign merchants. Witch trials were common around the same time throughout the Westfjords, and many people were banished to the nearby peninsula of Hornstrandir, now a national nature reserve. The town of Ísafjörður was granted municipal status in 1786.
The local folk museum contains the oldest house in Iceland, built in 1734. The largest collection of old timber frame houses in Iceland is in this area. The houses were mostly constructed by foreign traders in the late 18th century. These include Tjöruhús (completed in 1742),Krambúð (1761), and Turnhús (1744) which now contains a maritime museum
