Bird strikes present a huge risk for air vehicles, especially since traditional airport bird surveillance is mainly dependent on inefficient human observation. For improving the effectiveness and efficiency of bird monitoring, computer vision techniques have been proposed to detect birds, determine bird flying trajectories, and predict aircraft takeoff delays. Flying bird with a huge deformation causes a great challenge to current tracking algorithms. We propose a segmentation based approach to enable tracking can adapt to the varying shape of bird. The approach works by segmenting object at a region of interest, where is determined by the object localization method and heuristic edge information. The segmentation is performed by Markov random field, which is trained by foreground and background mixture Gaussian models. Experiments demonstrate that the proposed approach provides the ability to handle large deformations and outperforms the most state-of-the-art tracker in the infrared flying bird tracking problem.