There's no rule that requires "be" before adjectives.
If you're comparing the meaning of similar adjectives, then it's the words themselves we are comparing, so it's more natural to omit "be":
... the difference between burgundy and maroon.
... the difference between complicated and complex.
On the other hand, if you're comparing the state of being different adjectives, particularly the consequences of those states, then "being" is required:
... the difference between being married and being single.
... the difference between being lazy and being hardworking.
In the context of your example, the meanings of "exasperated" and "agitated" are quite close, so they're more likely referring to the meaning of the words rather than the state of being.