Cause Fear says:
Target one living creature with 5 or fewer HD
This implies the caster cannot target anybody with Hit Dice bigger than 5. Does that mean they actually knows if the creature had 5 or fewer HD? Or they actually can target such a creature, thus, waste the spell?
A few rare special abilities can determine a creature's hit points, but determining a creature's Hit Dice is rarer still, usually coming down to just guesswork. (Perhaps a generous GM will allow a creature's typical Hit Dice to be learned as a hint with a successful Knowledge skill check to gain some monster lore about the creature, but that's subject to the GM's whims.)
To summarize, Magic on Aiming a Spell says, "Some spells have a target or targets. You cast these spells on creatures or objects, as defined by the spell itself." However, on Spell Failure says, "If you ever try to cast a spell in conditions where the characteristics of the spell cannot be made to conform, the casting fails and the spell is wasted." Casting a spell on an invalid target is one of these cases that wastes an otherwise perfectly good spell.
Further, according to Saving Throws on Succeeding on a Saving Throw, a caster knows when "a creature’s saving throw succeeds against a targeted spell[, and, because of the target's successful saving throw, the caster knows] that the spell has failed." But when the target is simply—for whatever reason—flat-out immune to the spell, the caster receives no feedback from the spell.
So when a caster that casts a spell on an invalid target—like, for example, a caster that casts the spell cause fear on a 6 Hit Dice foe—, the caster won't know that the spell failed nor will the caster know why the spell doesn't appear to be having the effect it's supposed to have.
In general, you shouldn't know a target's HD in any case. The GM may handwave this as 'intuition', however. In general though, no you don't. The spell is wasted on higher HD creatures.
