In that specific case, there isn't much difference (or no difference at all) between "I've only been there once." and "I've been there only once." That is not always the case, though.
For example, the following sentences have different meanings.
Only I was waiting for her.
I was only waiting for her.
I was waiting only for her.
The first sentence means there was nobody but me waiting for her; the second one means I was not doing anything else than waiting for her, or that was my only reason for staying there; the third one means I was waiting for her and not other people.
Being in a place doesn't mean you are not doing anything, or you are just waiting; for example, if I say "I have been at work since 6 AM." I don't mean I didn't do anything, or I was waiting for some event to happen. If you mean that you were doing something, and only that, you need to explicitly say that.