Through the usage of digital stethoscopes in combination with telehealth, chest sounds can be easily collected and transmitted for remote monitoring and diagnosis. Chest sounds contain important information about a newborn's cardiorespiratory health. However, low-quality recordings complicate the remote monitoring and diagnosis. In this study, a new method is proposed to objectively and automatically assess heart and lung signal quality on a 5-level scale in real-time, and to assess the effect of signal quality on vital sign estimation. For the evaluation, a total of 207 10 s long chest sounds were taken from 119 preterm and full-term babies. Thirty of the recordings from ten subjects were obtained with synchronous vital signs from the Neonatal Intensive Care Unit (NICU) based on electrocardiogram recordings. As reference, seven annotators independently assessed the signal quality. For automatic quality classification, 400 features were extracted from the chest sounds. After feature selection using minimum redundancy and maximum relevancy algorithm, class balancing, and hyper-parameter optimization, a variety of multi-class and ordinal classification and regression algorithms were trained. Then, heart rate and breathing rate were automatically estimated from the chest sounds using adapted preexisting methods. The results of subject-wise leave-one-out crossvalidation show that the best-performing models had a mean squared error (MSE) of 0.487 and 0.612, and balanced accuracy of 56.8% and 51.2% for heart and lung qualities, respectively. The best-performing models for real-time analysis (<200 ms) had MSE of 0.459 and 0.673, and balanced accuracy of 56.7% and 46.3%, respectively. Our experimental results underscore that increasing the signal quality leads to a reduction in vital sign error, with only high-quality recordings having mean absolute error of less than 5 beats per minute, as required for clinical usage.