From the ferry stop, go straight ahead and up a little cobble stop path until you reach the street. Cross the street (if you look hard enough you will see a little yellow sign to the castle) and continue on the cobblestone path. You won’t feel like it is a correct path but if you make a sharp turn on it to the left and continue to climb up you will eventually reach a very shady path in the middle of small forest. The path goes right into the small town at the top of the hill and you have to continue through the town (following those yellow signs) to the castle. The walk up should take about 45 minutes
