When Samurai do awesome things like going out and beyond what they are expected to do, hunting down traitors, and fortifying the heavenly order of the empire, they earn Glory, which is kind of a measure of fame. In fact, that's exactly what it is.
Glory is a measure of fame, not prestige, and those who have lower Glory than a samurai are in no way obligated to obey him. 1
The very portion that follows after that snippet is titled Gaining Glory 1 and explains how glory is gained and for what actions. However, there is no mention of losing glory in that section, or in fact anywhere in the main text of the book. How does Glory fade over time to prevent Samurai that do anything to all have Glory 10.9 at some point?
1 - L5R Core rule book, p.93