Let's say that I want to cast Soul Cage. I know that I need the tiny silver cage worth 100gp. Can I just have it attached to my belt and cast the spell using my Spellcasting focus, or do I need to hold specifically the tiny silver cage?
I'm a Hexblade Warlock with a glaive and the Improved Pact Weapon feature, which makes my summoned glaive my Spellcasting Focus. I wonder if there's a way to kill an enemy with my glaive then use my reaction to cast Soul Cage.
Anytime a material component has a price, you must use that component for the spell, and cannot replace it with a spellcasting focus (PHB):
But if a cost is indicated for a component, a character must have that specific component before he or she can cast the spell.
[...]
A Spellcaster must have a hand free to access a spell’s material components—or to hold a Spellcasting focus—but it can be the same hand that he or she uses to perform somatic Components.
Remember that specific beats general. Generally you are able to replace holding a spell's material components by holding a spellcasting focus. But specifically material components with a cost cannot be replaced by a spellcasting focus, so you cannot cast those spell by holding the spellcasting focus in their place.
However for your situation, you can still cast the spell without dropping your weapon. Two-handed weapons require two hands to attack, but they do not require you to always have two hands on the weapon. As such, you can swing the weapon, then release the weapon with one hand and grab the tiny silver cage to cast Soul Cage. This would not work if you were wielding a one-handed weapon and a shield, however, as you would need to drop the weapon in order to grab the soul cage.
Like most rules in 5e, there is ambiguity here.
Casting some Spells requires particular Objects, specified in parentheses in the component entry. A character can use a Component pouch or a Spellcasting Focus (found in “Equipment”) in place of the Components specified for a spell. But if a cost is indicated for a component, a character must have that specific component before he or she can cast the spell.
If a spell states that a material component is consumed by the spell, the caster must provide this component for each casting of the spell.
This makes it clear that the silver cage requirement of Soul Cage cannot be substituted away with a Spellcasting focus. You must have that silver cage.
A Spellcaster must have a hand free to access a spell’s material components—or to hold a Spellcasting focus—but it can be the same hand that he or she uses to perform somatic Components.
The last paragraph is the most interesting one. The "same hand as somatic" can be dropped.
A Spellcaster must have a hand free to access a spell’s material components—or to hold a Spellcasting focus.
This can be read in two ways. Either
or
Which one applies requires you to talk to your DM, who may rely on rulings by various people at WotC or not.
Fluff wise, the second would be doing something like touching the component with your focus (you have to have it, but not hold it).
Regardless,
I'm a Hexblade Warlock with a glaive and the Improved Pact Weapon feature, which makes my summoned glaive my Spellcasting Focus. I wonder if there's a way to kill an enemy with my glaive then use my reaction to cast Soul Cage.
Using a glaive as a weapon requires 2 hands, but using it as a Spellcasting focus only requires 1. So you can attack, kill, drop one hand, grab the silver cage and use it with your now-free hand. The other hand can hold the glaive and use it as a spellcasting focus (assuming you care; some spellcasting focuses provide advantages to spellcasting).
Removing your hand from holding something is not considered an object interaction. Possibly putting your hand back on it might be. The "1 free object interaction" might be taken up by putting your hand back on the Glaive. You should discuss this with your DM.
The description of the Spellcasting Focus states:
A character can use a component pouch or a spellcasting focus in place of the components specified for a spell. But if a cost is indicated for a component, a character must have that specific component before he or she can cast the spell.
Furthermore
A spellcaster must have a hand free to access a spell's material components -- or to hold a spellcasting focus -- but it can be the same hand that he or she uses to perform somatic components.
So we know that a spellcasting focus can be used in place of material components and that for costly components the spellcaster must have them to cast the spell.
Nowhere is it stated, however, that the spellcaster must have the costly components in hand to cast the spell.
Whether your suggestion works depends an an interpretation of what "have" means in "must have that specific component". Does "have" mean you must be able to hold it in your hand? I suggest that simply possessing the component on one's person is sufficient to fulfill the common reading of "have".
So the requirements for casting a with costly components spell are:
By holding a focus and possessing the component you fulfill both criteria so you can cast the spell.
This can also work narratively. In the example of Soul Cage perhaps you perform the spell using the sword then touch your sword to the cage hanging on your belt, transferring the soul to a secure container.
If the costly components are consumed in casting then they are consumed regardless of whether you use a spellcasting focus.