There is some precedent for this
(Includes some information from Austin's answer)
There are two main considerations that involve your pact weapon.
Thirsting Blade Prerequisite: 5th level, Pact of the Blade feature
You can Attack with your pact weapon twice, instead of once, whenever
you take the Attack action on Your Turn.
.
Lifedrinker Prerequisite: 12th level, Pact of the Blade feature
When you hit a creature with your pact weapon, the creature takes
extra necrotic damage equal to your Charisma modifier (minimum 1).
If you have the Polearm Master feat, you can already get extra mileage out of the Lifedrinker invocation since the bonus action attack uses the pact weapon too.
There is no other unbalancing aspect if you have the Two-Weapon Fighting option extend to Lifedrinker (it won't extend to Thirsting Blade regardless since two weapon fighting doesn't use the Attack action for the second weapon attack).
As such, if you want it to be equivalent in balance, consider making a custom feat for Warlock with dual pact weapons. Some small benefit to compare to the opportunity attack of Polearm Master could also be added to increase the allure of the feat.
Note: It is debatable whether Thirsting Blade would let you use two attacks with one of those pact weapons or have the option to attack once with each since the feature dictates a singular pact weapon