This is a matter of both practical and theoretical law.
In theory, a sovereign state can choose its own laws by its own internal procedures. A sovereign state can therefore pass laws that criminalise actions taken in foreign countries, and in principle actions taken in foreign countries by foreigners against foreigners.
But so what? Suppose the UK decides that "child rape" is such a horrid crime that anybody who rapes a child anywhere in the world is guilty of an offence. That would not in practice permit the UK police to enter the USA and arrest suspected child rapists.
I choose this example because the law in the UK "Anyone who commits an offence against children abroad will face the prospect of prosecution for the same offence here even though it may not have been offence in that country." (Home Office Minister Vernon Coaker). But in practice that means they may be arrested on their return to the UK.
You state "as soon as they land in a country where they can arrest them". That is correct. However, the only country where the UK police can arrest people is the UK. Even though someone has broken UK law by abusing a child in a foreign country, there cannot practically be arrested until they land in the UK. In practice this power is only used when there is no prospect of arrest or conviction in the foreign country and only against people who have entered the UK.
In the Alscom case, it does seem odd to prosecute a French company for actions undertaken outside the US. However, Alscom seeks to operate in the USA, and in doing so, it chooses to put itself under US jurisdiction. It would be equally odd if a business could, merely by relocation of a central office, evade the law. Or if a French company doing business in the USA could steal an advantage over its American competitors by the use of bribery.
So. Yes, a country could prosecute anyone for any reason, and arrest them as soon as they enter the country. In practice, this power is only used by responsible powers for the most heinous of crimes, and only when local jurisdictions seem unable to criminalise the behaviour.