I am learning PyQt5 and was reading about it an PySide and I was wondering about something regarding the licensing.
PyQt is available under the GNU GPLv3 and a commercial license. From my understanding, if I use PyQt under the GPL license, anything and everything I publish onto which I used PyQt, regardless if I changed PyQt's code, must have its source code published, right?
Yes, your understanding is mostly correct.
If you use PyQt under the terms of the GPLv3 license, then the license terms dictate that you use effectively the same license also for the code you wrote.
Regarding the actual publishing of the source code, you only need to give the source code to the people that received the binary from you. So, if you are only creating applications for yourself (for your amusement and/or education) that you don't share with anyone, then you also don't have to publish the source code.