Pig behavior is an integral part of health and welfare management, as pigs usually reflect their inner emotions through behavior change. The livestock environment plays a key role in pigs’ health and wellbeing. A poor farm environment increases the toxic GHGs, which might deteriorate pigs’ health and welfare. In this study a computer-vision-based automatic monitoring and tracking model was proposed to detect pigs’ short-term physical activities in the compromised environment. The ventilators of the livestock barn were closed for an hour, three times in a day (07:00–08:00, 13:00–14:00, and 20:00–21:00) to create a compromised environment, which increases the GHGs level significantly. The corresponding pig activities were observed before, during, and after an hour of the treatment. Two widely used object detection models (YOLOv4 and Faster R-CNN) were trained and compared their performances in terms of pig localization and posture detection. The YOLOv4, which outperformed the Faster R-CNN model, was coupled with a Deep-SORT tracking algorithm to detect and track the pig activities. The results revealed that the pigs became more inactive with the increase in GHG concentration, reducing their standing and walking activities. Moreover, the pigs shortened their sternal-lying posture, increasing the lateral lying posture duration at higher GHG concentration. The high detection accuracy (mAP: 98.67%) and tracking accuracy (MOTA: 93.86% and MOTP: 82.41%) signify the models’ efficacy in the monitoring and tracking of pigs’ physical activities non-invasively.