In recent years, human activity recognition in intelligent visual surveillance has drawn much attention in the field of video analysis technology due to the growing demand from many applications, such as security and surveillance, sports and gaming, healthcare, person identification at distance, crowd behavior analysis, suspicious event detection and alarming in public/private places, traffic management, etc. Generally speaking, the human activity recognition in visual surveillance divided into following stages: object (human or vehicle) segmentation, feature extraction, object classification, object tracking, and activity recognition. Development of robust object segmentation method is the prime objective for any visual surveillance system. Object segmentation is used to detect the regions corresponding to static or moving human or vehicle. In this paper, we provide a comprehensive survey of the recent development of object segmentation (especially on human) algorithms in the context of human activity recognition in visualsurveillance. We will also discuss the strength and weakness of algorithms, complexities in activity understanding and identify the possible future research challenges.