This study aimed to test the mediating effects of nature restorativeness, stress, and nature connectedness in the association between nature exposure and quality of life (QoL). Urban and rural Lithuanian inhabitants (n = 924; 73.6% were women), mean age of 40.0 ± 12.4 years (age range of 18–79) participated in the study. In total, 31% of the respondents lived in rural areas. Study participants completed an online survey form with measures on sociodemographic factors, nature proximity, nature exposure, nature connectedness, and nature restorativeness, stress, and QoL assessed by the abbreviated version of the World Health Organization’s Quality of Life Questionnaire’s (WHOQOL-BREF). Path analysis was conducted to test the mediating effects of nature restorativeness, stress, and nature connectedness in the model of nature exposure and QoL. Nature exposure was directly associated with a greater QoL (β = 0.14; B = 2.60; SE = 0.57; p < 0.001) and mediated the association between nature proximity and QoL. Nature restorativeness and lower stress levels were mediators between nature exposure and QoL. Nature connectedness was a mediator between nature exposure and QoL. A path model was invariant across genders and the urban and rural place of residence groups: patterns of loadings of the pathways were found to be similar. Nature restorativeness (β = 0.10–0.12; p < 0.01) had a positive effect on the psychological, physical, social, and environmental domains of QoL. Connectedness to nature positively predicted psychological (β = 0.079; p < 0.05) and environmental (β = 0.082; p < 0.05) domains of QoL. Enhancing nature exposure and nature connectedness might help strengthen QoL in urban and rural inhabitants.