In tracking, there are two fundamental ways to solve the correspondence problem, either as a low-level feature matching or through high-level object matching. Most of the 2D pose tracking methods are based on high-level object matching. This makes them highly dependent on the object detectors, which are typically trained in specific views, limiting pose trackers to those view-points only. We propose a systematic approach for 2D pose tracking that combines lowlevel feature matching and high-level object matching approaches in a unified framework. We utilized brightness constancy assumption to find the corresponding pixels in two consecutive frames. We combine this tracking with frontal and profile pose detectors through a decoding and fusion strategy, to enable continuous pose estimation and tracking over wide range of view-points. The added advantage of our approach is, we not only track each limb, we can also track an articulated joint between them without requiring any 3D estimate of the skeleton. In addition to being computationally efficient, this hybrid tracking framework generalizes to unseen pose variations and compares favorably with existing work.