At the base of the Oštarije Pass, the lowest crossing over the Velebit mountain range, opposite the island of Pag, lies the town of Karlobag. From an elevation of 927 meters, there is a breathtaking and unforgettable view of the island of Pag. The aerial distance from the sea coast to this elevation is less than 2.5 kilometers, highlighting the immense ascent that needs to be overcome. Supporting this is the existing paved road that steeply descends or ascends through switchbacks over 17 kilometers. Due to the short aerial distance and elevation difference, a strong northeasterly wind, famously known as the Baška Bura, rushes down to the sea coast.

Along the sea coast, between two bays—Baška Draga to the east and Draga Tatinja to the west—the town is situated in the central part of this area. On the flatter part above the settlement stands a small town fortress, from which the entire settlement as well as the maritime route between the coast and the island of Pag used to be monitored. Given the ruggedness of the entire area, there were very few cultivated plots of several dozen square meters, and what existed was barely arable due to the lack of water.

The Velebit Channel, about one and a half nautical miles wide, separates Karlobag from the island of Pag. On the northern side, opposite Karlobag, the island of Pag is completely barren and without any vegetation. Along the entire sea coast, among all the settlements in this area, only Karlobag is a larger and ancient town.