In real time video sequence, object segmentation and tracking method are actively applied in various application tasks, such as surveillance system, mobile robots, augmented reality. This paper propose a robust object tracking method. The background models are constructed by learning the initial part of each video sequences. After that, the moving objects are detected via object segmentation by using background subtraction method. The region of detected objects are continuously tracked by using the HSV color histogram with particle filter. The proposed segmentation method is superior to average background model in term of moving object detection. In addition, the proposed tracking method provide a continuous tracking result even in the case that multiple objects are existed with similar color, and severe occlusion are occurred with multiple objects. The experiment results provided with 85.9 % of average object overlapping rate and 96.3% of average object tracking rate using two video sequences.키워드 : 배경제거, 배경모델, 객체 분할, 객체 추적, 파티클 필터