Why does my hexadecimal to base64 converter break down on large numbers?

Question

I'm working on a cryptopals problem. Specifically, the first one. I have a, what I feel, decent solution for it, in that it works for given inputs, and for the example they give, but I've been looking at further testing to see if it holds up, and it doesn't seem to. Here's my code:

hex="0123456789abcdef"
base64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"

def hexNumeralToBinary(hexNumeral):
    index = hex.index(hexNumeral)
    binaryNumber = []
    while index > 0:
        if (index % 2) == 0:
            binaryNumber = ["0"] + binaryNumber
            index = index / 2
        else:
            binaryNumber = ["1"] + binaryNumber
            index = (index-1)/2

    while len(binaryNumber) < 4:
        binaryNumber = ["0"] + binaryNumber
    return ''.join(binaryNumber)


def hexToBinary(hexNumber):
    length = len(hexNumber) + 1

    binaryNumber = []

    for i in range(1, length):
        hexNumeral = hexNumber[-1*i]
        binaryNumeral = hexNumeralToBinary(hexNumeral)
        binaryNumber = [binaryNumeral] + binaryNumber

    return ''.join(binaryNumber)

def splitString(binaryNumber):
    while (len(binaryNumber) % 6) != 0:
        binaryNumber = "0" + binaryNumber

    binaryNumberSplit = []

    while binaryNumber != "":
        binaryNumberSplit.append(binaryNumber[0:6])
        binaryNumber = binaryNumber[6:]

    return binaryNumberSplit

def hexToBase64(hexNumber):
    base64Number = []

    binaryNumber = hexToBinary(hexNumber)
    binaryNumberSplit = splitString(binaryNumber)

    for sixBitNum in binaryNumberSplit:
        #Convert 6 bit binary number into the base64 index
        index = 0
        for i in range(1, 7):
            index += int(sixBitNum[-1*i]) * (2**(i-1))
        base64Digit = base64[index]


        base64Number = base64Number + [base64Digit]

    return ''.join(base64Number)


hexNumber = input("Please enter a hexadecimal number to convert to base64: ")

print(hexToBase64(hexNumber))

It seems like the hexadecimal number needs to have a number of digits divisible by 3, or it just gets stuck while running. I don't know where, and I'm really stumped. I'm a total beginner to programming, and this is easily the most complex thing I've done, just trying to work it out as I go, and this isn't coming to me.