API version is defined in different different ways depending on what you are doing with it.
- SOAP API calls
- REST API calls
- Apex
- Visualforce
All of these (and other features that use API) are fixed in the code you write against them, allowing you to upgrade/update when it fits your company/project lifecycle. Integration uses different end points. Apex and Visualforce use metadata to define the version.
Essentially there is only ever one version of Salesforce in production. Previous versions of the API are emulated for the purpose of integration and code resilience.
So when you write your REST request (as that's what you've been using), you will use a URL like this, for instance:
HTTP:GET
/services/data/v26.0/sobjects/account/001B0000003nJhJIAU
The v26.0 segment of the URL path is what stipulates the version. You can substitute any other previous or later version up to the current version (v33.0 for Spring 15, the current release).
The thing is, there is no guarantee it will be supported. For instance if you were to make a GET request to this URL:
HTTP:GET
/services/data/v20.0/sobjects/FeedItem
It would fail stating that this "service does not exist". The FeedItem object didn't exist in that version. It's predecessor FeedPost was used for that at the time.
In some instances you will get a response, but a different result. For instance these two GET endpoints:
/services/data/v26.0/sobjects/account/001B0000003nJhJIAU
/services/data/v33.0/sobjects/account/001B0000003nJhJIAU
The second returns a few extra fields that were added in the ensuing versions (I forget exactly which version...somewhere around 30, I think) for geolocation called longitude and latitude.
And another example:
HTTP:POST
/services/data/v33.0/composite/batch
In this instance, it won't work, but that's because it will not be available until version 34 of the API. But if you get a pre-release org, you can do the same with the updated API version (although you need to post some JSON in your request body for it to work):
HTTP:POST
/services/data/v34.0/composite/batch
As for considerations for when to upgrade...I wrote this answer about that. It pertains to Apex code, but the same principle applies: upgrading your integrations/code arbitrarily because Salesforce did a major release is just giving yourself extra work.
New releases bring new features. When your integration needs to make use of a new feature, that's the time to upgrade, and only the integrations that need the new feature.
Salesforce still has working integrations against every version of the API. If it is new build, always build against the newest version. If you have to write code against an existing integration, evaluate an upgrade at that time.
Finally, I don't know what tool you are using for your testing, but it sounds like it might be defaulting to a previous version of the API and maybe overriding your requests? If so, it is the tool. Salesforce always takes the requested API version and responds in kind. If you want to get to know the REST API, I'd suggest spending some time using the workbench.
The login page needs to be updated to use the latest version, but if you go to the REST Explorer, you can override the login version for request testing. In this screen shot, you can see I'm using v34.0 against a pre-release org. Production orgs will only work up to version 33!

/services/data/v26.0which I haven't explicitly set - so I'm assuming this is keyed off some aspect of my instance / object. – Huw May 13 '15 at 11:57