Background: Periprosthetic joint infection (PJI) represents a serious complication following total hip (THA) and knee arthroplasty (TKA). When preoperative synovial fluid cultures remain inconclusive, open incisional joint biopsy (OIB) can support causative microorganism identification. Objective: This study investigates the potential benefit of OIB in THA and TKA patients with suspected PJI and ambigious diagnostic results following synovial fluid aspiration. Methods: We retrospectively assessed all patients treated from 2016 to 2020 with suspected PJI. Comparing the microbiology of OIB and the following revision surgery, we calculated sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the number needed to treat (NNT). Results: We examined the diagnostic validity of OIB in 38 patients (20 female) with a median age of 66.5 years. In THA patients (n = 10), sensitivity was 75%, specificity was 66.67%, PPV was 60%, NPV was 80%, and NNT was 2.5. In TKA patients (n = 28), sensitivity was 62.5%, specificity was 95.24%, PPV was 83.33%, NPV was 86.96%, and NNT was 1.42. Conclusions: Our results indicate that OIB represents an adequate diagnostic tool when previously assessed microbiological results remain inconclusive. Particularly in TKA patients, OIB showed an exceptionally high specificity, PPV, and NPV, whereas the predictive validity of the diagnosis of PJI in THA patients remained low.