Okay, so I found a few answers elsewhere on Stack Overflow that answers my question. Very few programs can perform this process on a PDF, so you'll need to do it by hand in a plain-text editor that can handle files tens of megabytes large and won't choke on or corrupt the binary data in the embedded images. I used vim for this.
Search for the string Catalog in the PDF, then in the opening double angle brackets << add the following:
/PageLabels << /Nums [
0 << /P (Front Cover) >>
1 << /P (Front Cover, Inner) >>
2 << /S /D >>
26 << /P (Back Cover, Inner) >>
27 << /P (Back Cover) >>
] >>
Making the relevant changes for your PDF. These were just the particular page numbers and options relevant to me. The first number is the page number, starting from 0. The /P says that I want the text in parentheses to be the page title. The /S /D says that I want to start the numbering in Arabic numerals with page 1 being the third page of the PDF. There are a lot of options to use here. More information about this process and the available options can be found at these links:
https://askubuntu.com/a/347338
https://superuser.com/questions/232553/how-to-change-internal-page-numbers-in-the-meta-data-of-a-pdf
https://www.w3.org/TR/WCAG20-TECHS/PDF17.html#PDF17-ex2
https://newbedev.com/renumber-pages-of-a-pdf