In order to resolve crashed accidents caused by fatigue or drowsiness various fatigue detecting algorithms are investigated. Aiming at the needs of large data, high transmission speed and complex operation, a real-time video-based processing system using SAA7.115 as its capture AD, TMS320DM642 as its core, SAA7105 as its display DA ,is designed, and FPGA is used to control the display to achieve advanced function. The system can meet the needs of capture, processing and display of multi-channel video. Furthermore, it can be used as a video and image processing platform.