BackgroundTo analyse structured and free text reports of shoulder X-ray examinations evaluating the quality of reports and potential contributions to clinical decision-making.MethodsWe acquired both standard free text and structured reports of 31 patients with a painful shoulder without history of previous trauma who received X-ray exams. A template was created for the structured report based on the template ID 0000154 (Shoulder X-ray) from radreport.org using online software with clickable decision trees with concomitant generation of structured semantic reports. All reports were evaluated regarding overall quality and key features: content, information extraction and clinical relevance.ResultsTwo experienced orthopaedic surgeons reviewed and rated structured and free text reports of 31 patients independently. The structured reports achieved significantly higher median ratings in all key features evaluated (P < 0.001), including facilitation of information extraction (P < 0.001) and better contribution to subsequent clinical decision-making (P < 0.001). The overall quality of structured reports was significantly higher than in free text report (P < 0.001).ConclusionsA comprehensive structured template may be a useful tool to assist in clinical decision-making and is, thus, recommended for the reporting of degenerative changes regarding X-ray examinations of the shoulder.Electronic supplementary materialThe online version of this article (10.1186/s12880-018-0262-8) contains supplementary material, which is available to authorized users.