Abstract: The accurate identification of fish feeding behavior is of great significance to reduce feed waste, reduce water pollution and improve aquaculture benefits. Accurate classification of fish feeding status can provide necessary basic support for accurate identification and quantification of fish feeding behavior. Aiming at the technical needs of accurate classification of continuous feeding behavior of fish, we proposed a feeding state classification method of grass carp based on inter frame optical flow characteristics and improved RNN (recurrent neural network). Firstly, in order to reduce the impact of outdoor natural light conditions on imaging quality, an outdoor pond sampling system was built by using polarization camera to collect water surface images with different polarization angles. And the low reflection angle images were automatically selected based on the image saturation and brightness model, and the image sample database was constructed on this basis; Secondly, the image optical flow was extracted by LK (Lucas Kanade) method, and the two parameters of speed and rotation angle were selected to characterize the image optical flow characteristics. The image was evenly segmented, and the optical flow intensity of each image block was quantified by the optical flow pulsation intensity. On this basis, the image optical flow intensity description vector is constructed to effectively describe the global optical flow characteristics of the image based on local features; Then, based on the switching time length of the feeder, the video samples were segmented, and combined with the extracted inter frame optical flow features, the time series feature sample set is constructed. Finally, the sample set was used to train the improved RNN classification network, and the accuracy, recall and average F1 score were used to evaluate the effectiveness of the network. The method was verified by the experimental data of Ruibo aquaculture professional cooperative in Chongming District, Shanghai. The results show that the average accuracy of this method is 91%, the average recall rate is 92.2%, and the average F1 score is 91.6, which are better than the traditional fish feeding behavior recognition methods. The results can provide a reference for the research of fish precision feeding technology.