MYSQL8 unknown runaway memory

Question

First thank you, I am not very knowledgeable in MYSQL and am purely self taught.

I moved from metal to AWS(EC2, with maxed out IOPS:3000) and in the process we also upgrade from MYSQL5.6 to MYSQL8.0.28.

We have had major memory issues with MYSQL8 and cannot seem to track it down. It takes several days (~5-10) to exhaust memory on our production server (~1k requests per second) MYSQL is takes up over 100% of our 16GB of memory.

It slowly creeps up with a few random spikes.

we have 16GB of memory, the old metal server also had 16GB.

all tables are innodb.

I reduced innodb_buffer_pool_size from 8GB to 4GB to try and stabilize the server with no joy.

any advise would be a lot of help for me to understand what is going on.

---

UPDATE 4-25-2022

I wanted to let this run for a few days to let it get back to an add state. I changed a bunch of settings based on another post recommendation, so i removed the previous information for this thread.

mysqlc.cnf https://pastebin.com/FRuh55Lx

The paste bins request by Wilson below:

A)439

B) https://pastebin.com/86Tj6YBe

C) https://pastebin.com/Jn19dyu7

D)

mysql> SHOW FULL PROCESSLIST;
+---------+-----------------+----------------------+-------+---------+--------+------------------------+-----------------------+
| Id      | User            | Host                 | db    | Command | Time   | State                  | Info                  |
+---------+-----------------+----------------------+-------+---------+--------+------------------------+-----------------------+
|       5 | event_scheduler | localhost            | NULL  | Daemon  | 357369 | Waiting on empty queue | NULL                  |
| 2208882 | root            | localhost            | NULL  | Sleep   |    330 |                        | NULL                  |
| 2210947 | root            | localhost            | NULL  | Sleep   |    144 |                        | NULL                  |
| 2211462 | calypso         | <removed>:33362 |  <removed>| Sleep   |     52 |                        | NULL                  |
| 2211998 | freeradius      | localhost            |  <removed>| Sleep   |      0 |                        | NULL                  |
| 2212151 | calypso         |  <removed>:33992 |  <removed>| Sleep   |     52 |                        | NULL                  |
| 2212502 | freeradius      | localhost            | ikeja | Sleep   |      0 |                        | NULL                  |
| 2212641 | root            | localhost            | NULL  | Query   |      0 | init                   | SHOW FULL PROCESSLIST |
+---------+-----------------+----------------------+-------+---------+--------+------------------------+-----------------------+
8 rows in set (0.00 sec)

E)

mysql> STATUS;
--------------
mysql  Ver 8.0.28-0ubuntu0.20.04.3 for Linux on x86_64 ((Ubuntu))
Connection id:          2212641
Current database:
Current user:           root@localhost
SSL:                    Not in use
Current pager:          stdout
Using outfile:          ''
Using delimiter:        ;
Server version:         8.0.28-0ubuntu0.20.04.3 (Ubuntu)
Protocol version:       10
Connection:             Localhost via UNIX socket
Server characterset:    utf8mb4
Db     characterset:    utf8mb4
Client characterset:    utf8mb4
Conn.  characterset:    utf8mb4
UNIX socket:            /var/run/mysqld/mysqld.sock
Binary data as:         Hexadecimal
Uptime:                 4 days 3 hours 18 min 15 sec
Threads: 12  Questions: 37878605  Slow queries: 35  Opens: 1647  Flush tables: 3  Open tables: 1501  Queries per second avg: 105.955