There is no universal answer to this. How bad is the most undesirable party that might win if you don't vote against them? If the consequences of them gaining power are dire, which they often are, there is a strong motivation to vote for the other "least bad" option.
The question then becomes the exact nature of the motivation. Money in your pocket is clearly quite different to the undesirable party being opposed to your existence or remaining in the country, for example. This is an existential point that is difficult to generalize about because it depends so much on circumstance.
Based on the assumption that nothing too terrible will happen if the most undesirable party gets in, you would then be more concerned with how well the democracy is working.
It could be the case that the two mainstream parties genuinely appeal to the majority of voters, and yours is a minority viewpoint. In that case, and considering that you are not being oppressed or put in danger, you may simply have to accept that no matter what you do it is unlikely to have much effect. Given that there seems little point not voting with your conscience, so at least that can be clear.
You could take action beyond just voting. Join your preferred party, campaign for it and try to increase its popularity. If the democratic system is broken and makes this unlikely to work you could consider fighting the system. Fighting can range from low level civil disobedience to a full blown civil war. You asked to add the assumption that the democracy was "western style" and therefore not too bad, but we have seen that even western democracies can result in extreme things happening. Staring or joining wars, or setting up a vast secret surveillance network, for example. One of the key ways that such systems avoid citizens rising up against them is by making life generally "okay" for the majority, so they don't feel strongly enough to act.
With that in mind we come back to an existential problem. In fact Sartre mentioned a similar example in Existentialism and Humanism, where a young French man was trying to decide if he should look after his elderly mother or join the war effort (in the 1940s). Each individual must decide for themselves how bad things are, how much they care about what is happening, and what the appropriate action to take is.
Sadly we have reached a point where mass protest is fairly ineffective, but on the other hand the rise of fairly far right parties in Europe (UKIP, Le Penn) and the US (Tea Party) suggests that movements can grow into influential political forces. It's notable that all these groups are based on anger and often hatred though.
You could also decide to simply accept that you are a minority and that true democracy dictates the majority rules. This is rarely the case of course, otherwise we would suffer the tyranny of the majority.