I ran into this situation in a gaming session last week and didn't want to make a big deal of it at the time, but I am curious.
The components of the banishment spell are:
V, S, M (an item distasteful to the target)
(emphasis mine)
In my session, we were trudging along through a swamp and came to an island with a mysterious altar, and 2 statues to either side of the altar. The statues turned out to be iron golems. Our wizard managed to banish one of them, which kept the fight reasonable for our party of 13th-level characters.
What I'm wondering is what is distasteful to a golem? I've always envisioned golems as mindless. If there is nothing distasteful to a golem, can you replace it with a spellcasting focus or component pouch?