Manual observation and classification of animal behaviors is laborious, time-consuming, and of limited ability to process large amount of data. A computer vision-based system was developed that automatically recognizes sow behaviors (lying, sitting, standing, kneeling, feeding, drinking, and shifting) in farrowing crate. The system consisted of a low-cost 3D camera that simultaneously acquires digital and depth images and a software program that detects and identifies the sow’s behaviors. This paper describes the computational algorithm for the analysis of depth images and presents its performance in recognizing the sow’s behaviors as compared to manual recognition. The images were acquired at 6 s intervals on three days of a 21-day lactation period. Based on analysis of the 6 s interval images, the algorithm had the following accuracy of behavioral classification: 99.9% in lying, 96.4% in sitting, 99.2% in standing, 78.1% in kneeling, 97.4% in feeding, 92.7% in drinking, and 63.9% in transitioning between behaviors. The lower classification accuracy for the transitioning category presumably stemmed from insufficient frequency of the image acquisition which can be readily improved. Hence the reported system provides an effective way to automatically process and classify the sow’s behavioral images. This tool is conducive to investigating behavioral responses and time budget of lactating sows and their litters to farrowing crate designs and management practices.