Colonoscopy is an important screening tool for colorectal cancer. During a colonoscopic procedure, a tiny video camera at the tip of the endoscope generates a video signal of the internal mucosa of the colon. The video data are displayed on a monitor for real-time analysis by the endoscopist. We call videos captured from colonoscopic procedures "colonoscopy videos". To the best of our knowledge, they are not captured for post procedural review or analysis in the current practice. Because of the unique characteristics of colonoscopy videos, new types of semantic units and new image/video analyzing techniques are required. In this dissertation, we aim to develop new image/video analysis techniques for these videos to extract important semantic units, such as colonoscopic scenes, operation shots, and appendix images. Our contributions include two parts: (a) new definitions of semantic units (colonoscopic scene, operation shot, and appendix image); and (b) novel image/video analysis algorithms, including novel scene segmentation algorithms using audio and visual information to recognize scene boundaries, new computer-aided detection approaches for operation shot detection, and new image analysis methods for appendix image classification. The new image processing and content-based video analysis algorithms can be extended to videos from other endoscopic procedures, such as upper gastrointestinal endoscopy, EGD, enteroscopy, bronchoscopy, cystoscopy, and laparoscopy. Our research is very useful for the following platforms and resources: (a) platforms for new methods to discover unknown patterns of diseases and cancers; (b) platforms for improving and assessing endoscopists procedural skills; and (c) education resources for endoscopic research.