Osteoporosis is a common disease, and the prevalence is increasing in patients with chronic respiratory diseases, with important implications with regard to fractures, hospitalization, and death. Due to inconsistent data and a lack of large cohort follow-up studies on the association between lung function and osteoporosis, the aim of this study was to investigate this issue. We enrolled and followed for a median of 4 years a total of 9059 participants with no history of smoking, bronchitis, emphysema, or asthma from the Taiwan Biobank. Spirometry data, including forced expiratory volume in 1 second (FEV1) and forced vital capacity (FVC), were used to assess lung function. Changes in the calcaneus ultrasound T-score (ΔT-score) were calculated as follow-up T-score—baseline T-score. A ΔT-score ≤ −3 (median value of ΔT-score) meant a fast decline in T-score. Multivariable analysis showed that lower values of FEV1 (β, 0.127, p < 0.001), FVC (β, 0.203, p < 0.001), and FEV1/FVC (β, 0.002, p = 0.013) were significantly associated with a low baseline T-score. In addition, after follow-up, higher values of FEV1 (odds ratio (OR), 1.146, p = 0.001), FVC (OR, 1.110, p = 0.042), and FEV1/FVC (OR, 1.004, p = 0.002) were significantly associated with ΔT-score ≤ −3. FEV1/FVC < 70% (OR, 0.838, p < 0.001) was significantly associated with ΔT-score ≤ −3. In conclusion, lower FEV1, FVC, and FEV1/FVC were associated with a low baseline T-score, and higher FEV1, FVC, and FEV1/FVC were associated with a rapid decline in T-score in follow-up. This suggests that lung disease may be associated with bone mineral density in the Taiwanese population with no history of smoking, bronchitis, emphysema, or asthma. Further research is needed to establish causality.