There has been a substantial amount of research reporting the neuroanatomical associations of psychotic symptoms in people with schizophrenia. Comparatively little attention has been paid to the neuroimaging correlates of subclinical psychotic symptoms, so-called “psychotic-like experiences” (PLEs), within large healthy populations. PLEs are relatively common in the general population (7–13%), can be distressing and negatively affect health. This study therefore examined gray and white matter associations of four different PLEs (auditory or visual PLEs, and delusional ideas about conspiracies or communications) in subjects of the UK Biobank study with neuroimaging data (N = 21,390, mean age = 63 years). We tested for associations between any PLE (N = 768) and individual PLEs with gray and white matter brain structures, controlling for sex, age, intracranial volume, scanning site, and position in the scanner. Individuals that reported having experienced auditory hallucinations (N = 272) were found to have smaller volumes of the caudate, putamen, and accumbens (β = −0.115–0.134, pcorrected = 0.048–0.036), and reduced temporal lobe volume (β = −0.017, pcorrected = 0.047) compared to those that did not. People who indicated that they had ever believed in unreal conspiracies (N = 111) had a larger volume of the left amygdala (β = 0.023, pcorrected = 0.038). Individuals that reported a history of visual PLEs (N = 435) were found to have reduced white matter microstructure of the forceps major (β = −0.029, pcorrected = 0.009), an effect that was more marked in participants who reported PLEs as distressing. These associations were not accounted for by diagnoses of psychotic or depressive illness, nor the known risk factors for psychotic symptoms of childhood adversity or cannabis use. These findings suggest altered regional gray matter volumes and white matter microstructure in association with PLEs in the general population. They further suggest that these alterations may appear more frequently with the presentation of different psychotic symptoms in the absence of clinically diagnosed psychotic disorders.