The base damage dice for the weapon
For Mundane weapons, this is the standard die shown on the Weapons table on PHB p. 146 or here in the basic rules (thanks FenrirG!):
The Weapons table shows the most common weapons used in the worlds of d&d, their price and weight, the damage they deal when they hit, and any special properties they possess.
(emphasis mine)
If it is a magical weapon that has additional damage (like 2d6 fire from Flame Tongue) then that damage is included.
This is supported by Jeremy Crawford here in a discussion on Savage Attacks:
Savage Attacks: add 1 of a weapon's dice to a crit. again. Frost brand is a wpn. w/ 1d6 cold as 1 of its dice
So what doesn't count?
Pretty much everything else that isn't the above. If the die isn't included in the base stats of the weapon, then it's not included. This includes race/class/spell features that are above and beyond the base weapon stats that don't explicitly state a damage die is replaced (like with a monk (PHB, 78))
You can roll a d4 in place of the normal damage of your unarmed strike or monk weapon. This die changes as you gain monk levels, as shown in the Martial Arts column of the Monk table.
(emphasis mine)