In this paper we propose a hierarchical decoder to augment power saving of the sentry-table filter based low power branch target buffer (BTB) of branch predictor in modern processors. The sentry table scheme filtrates unnecessary accesses of branch target buffer to reduce dynamic power consumption, yet the power saving was found hysterically bound by the power dissipation of decoder, especially when the BTB size grows. The proposed hierarchical decoder (H-DEC) can significantly offset the effect of decoder power dissipation. We use CACTI tool, SimpleScalar and Watch simulators and SPEC2000 benchmarks to conduct our experiments. From our empirical studies, power savings for BTB can be further improved from 19-38% to 68-91%; and those for branch predictor from 17-21% to 37-81%.