Target detection and tracking in the sea clutter environment is a challenging issue for radar system. This paper proposes a robust detection and tracking method based on neural network. By using the different characteristics of motion and scattering between targets and other detections (other targets and false alarms by clutter), a classification model is established to distinguish the categories of the targets. The output of the network is used for both the target detection and target association, with the goal of reducing false alarm and track correlation error more efficiently compared with traditional methods. The feasibility of our method is verified by the measured data.