"Been" is, as you say, the past participle of "to be". It is not the same as "was", which is the past tense.
A participle is one of those very common words that's hard to explain. "To be" is another one; because it's used in many places in many (related, but separate) ways, it can be difficult to explain clearly. So your confusion is understandable!
Basically, a participle is a form of a verb, but it's used as a noun (for the action itself that the verb describes) or an adjective (to describe the state of someone/something that is doing the action). There are two participles for each verb: the present and the past.
An example that isn't "to be" might help here. Let's use "to eat":
eating (present participle):
1. (noun) The action you do when you eat. "Eating can be messy."
2. (adj.) The state you're in while you eat. "Can't you see that I'm eating?"
eaten (past participle):
3. (adj.) The state of being something that someone ate. "This apple is half eaten."
4. (adj.) The state you're in after you eat. "I have eaten dinner, but I'm still hungry."
Participles are also used in compound tenses. (In fact, that has happened in examples 2 and 4.) In this role, they always follow another verb, either "to have" or "to be".
You asked whether "been" can ever follow something other than "have" (or its other forms: "had", "having", etc.). The short answer is: no.
The long answer is: yes, in theory, but in practice it almost never happens. All of these are legal English sentences.
The rabbit was tired.
The rabbit was seen.
The rabbit was eaten.
The rabbit was been.
But the last one doesn't make sense. It says that something else was being the rabbit. But the rabbit is the rabbit. Nothing else is.
So while past participles are allowed to follow either "to have" or "to be", it doesn't make sense to put "been" after "to be". So, it only comes after "to have".