Spectrum selection is crucial in improving the performance of cognitive radio networks. Most of previous work on spectrum selection consider spectrum utilization ratio as system design criteria and ignore the energy issue. In this paper, we take a cross-layer design approach to select the optimal spectrum considering the time varying and stochastic nature of wireless channels. The proposed scheme selects one spectrum among different alternatives and sets the modulation and coding scheme and frame size in each time slot according to the channel state, which is modeled as finite-state Markov channel, with the objective of minimizing energy consumption. The procedure of spectrum access is formulated as a restless bandit problem, which can be solved using linear programming (LP) and primal-dual index heuristic algorithm. Simulation results are presented to show the performance improvement of the proposed scheme.