As you make more contacts (friend or enemy) you'll find that keeping track of them all will be easier from the command building interfaces than by scrolling on the map. Alternately, you can perform most interactions from your Naval and Command buildings. Go to the other Captain's Haven, click 'Actions' on the right menu and choose the action you want to perform.
You can also interact by visiting their Haven. You can interact with your fellow Captains from the Map interface by going to the Map and positioning your mouse over another Captain's Haven and choosing an action (Trade, Raid, etc.) from their Haven's pop-up menu. To succeed you'll need to master all the different ways of influencing and engaging other Captains. Interaction between Captains will be the key feature of your gameplay experience. You can help Pirates: Tides of Fortune Wiki by expanding it.