Jesus related his authority to being a Son of Man (no definite article):
[Jhn 5:26-27 NKJV] (26) "For as the Father has life in Himself, so He has granted the Son to have life in Himself, (27) "and has given Him authority to execute judgment also, because He is [the] Son of Man.
The OT background of the Son of Man motif that seems to be in view here is that of Daniel's vision:
[Dan 7:13-22 YLT] (13) 'I was seeing in the visions of the night, and lo, with the clouds of the heavens as a son of man was [one] coming, and unto the Ancient of Days he hath come, and before Him they have brought him near. (14) And to him is given dominion, and glory, and a kingdom, and all peoples, nations, and languages do serve him, his dominion [is] a dominion age-during, that passeth not away, and his kingdom that which is not destroyed. (15) 'Pierced hath been my spirit -- I, Daniel -- in the midst of the sheath, and the visions of my head trouble me; (16) I have drawn near unto one of those standing, and the certainty I seek from him of all this; and he hath said to me, yea, the interpretation of the things he hath caused me to know: (17) 'These great beasts, that [are] four, [are] four kings, they rise up from the earth; (18) and receive the kingdom do the saints of the Most High, and they strengthen the kingdom unto the age, even unto the age of the ages. (19) 'Then I wished for certainty concerning the fourth beast, that was diverse from them all, fearful exceedingly; its teeth of iron, and its nails of brass, it hath devoured, it doth break small, and the remnant with its feet it hath trampled; (20) and concerning the ten horns that [are] in its heads, and of the other that came up, and before which three have fallen, even of that horn that hath eyes, and a mouth speaking great things, and whose appearance [is] great above its companions. (21) 'I was seeing, and this horn is making war with the saints, and hath prevailed over them, (22) till that the Ancient of Days hath come, and judgment is given to the saints of the Most High, and the time hath come, and the saints have strengthened the kingdom.
Now, the NLT above translates verse 22 as saying that God judged in favor of his holy people:
[Dan 7:22 NLT] (22) until the Ancient One--the Most High--came and judged in favor of his holy people. Then the time arrived for the holy people to take over the kingdom.
This is how it is understood by the NKJV (but not the KJV), the NIV, the CSB, the NASB and others. However, a more literal reading can be understood as the saints receiving the authority to preside in judgement:
[Dan 7:22 YLT] (22) till that the Ancient of Days hath come, and judgment is given to the saints of the Most High, and the time hath come, and the saints have strengthened the kingdom.
The LXX seems to support that idea as well:
Daniel 7:22 Brenton(i) 22 until the Ancient of days came, and he gave judgment to the saints of the Most High; and the time came on, and the saints possessed the kingdom.
Such verses are cited as evidence that the Kingdom of God had arrived:
[Luk 11:20 NKJV] (20) "But if I cast out demons with the finger of God, surely the kingdom of God has come upon you.
Please see this strongly related question and the accepted answer.
This explains the reference to the giving of the authority to "men" in Matthew 9:8. The authority is given not only to the Son of Man but also to [designated] saints of the Most High.
Below are some background on "Binding and Loosing":
For a complete treatment of the background of the subject I recommend this article in the Jewish Encyclopedia (one of my favorite resources for a great many subjects):
http://jewishencyclopedia.com/articles/3307-binding-and-loosing
I recommend considering the comments as well.
The heart of the background of the passages is the authority that was enjoyed by the Pharisees not only in parsing the Torah into "you must do this but you don't have to do that" but "binding days" and other forms of control over Jewish life, with Jesus appearing to say that his disciples should submit to their edicts, despite the hypocrisy of their edicts:
Mat 23:1 Then spake Jesus to the multitude, and to his disciples, Mat 23:2 Saying, The scribes and the Pharisees sit in Moses' seat: Mat 23:3 All therefore whatsoever they bid you observe, that observe and do; but do not ye after their works: for they say, and do not. Mat 23:4 For they bind heavy burdens and grievous to be borne, and lay them on men's shoulders; but they themselves will not move them with one of their fingers.
This power is then assigned to the apostles. The Catholic power-grab was to pronounce a segue from the apostles to the Popes saying that the Bishop of Rome would inherit the authority from the apostles by some undocumented "handing off" of the keys of the kingdom by some alleged overlap between the apostle Peter and their own power structure.
In Matthew 5 Jesus says that he did not come to parse the Torah into "you must do this but you don't have to do that" and instead came to restore the integrity (unity) of the Torah:
Mat 5:17 "Do not think that I have come to abolish ["parse"] the Law or the Prophets; I have not come to abolish ["parse"] them but to fulfill them ["restore their integrity"]. Mat 5:18 For truly, I say to you, until heaven and earth pass away, not an iota, not a dot, will pass from the Law until all is accomplished. Mat 5:19 Therefore whoever relaxes one of the least of these commandments and teaches others to do the same will be called least in the kingdom of heaven, but whoever does them and teaches them will be called great in the kingdom of heaven. Mat 5:20 For I tell you, unless your righteousness exceeds that of the scribes and Pharisees, you will never enter the kingdom of heaven. Mat 5:21 "[For example,] You have heard that it was said to those of old, 'You shall not murder; and whoever murders will be liable to judgment.' Mat 5:22 But I say to you that everyone who is angry with his brother will be liable to judgment; whoever insults his brother will be liable to the council; and whoever says, 'You fool!' will be liable to the hell of fire.
So while Jesus seems to allow Pharisaic (and also Scribal(?)) authority (saying that they "sit in Moses' seat") and should be obeyed, grants the apostles the same authority he does not practice or condone "loosing" any of the commands of scripture.
I'm undecided but perhaps the following is an example of apostolic "binding":
1Co 5:4 When you are assembled in the name of the Lord Jesus and my spirit is present, with the power of our Lord Jesus, 1Co 5:5 you are to deliver this man to Satan for the destruction of the flesh, so that his spirit may be saved in the day of the Lord.