Radiology reports are an important means of communication between radiologists and other physicians. These reports express a radiologist's interpretation of a medical imaging examination and are critical in establishing a diagnosis and formulating a treatment plan. In this paper, we propose a Bidirectional convolutional neural network (Bi-CNN) model for the interpretation and classification of mammograms based on breast density and chest radiographic radiology reports based on the basis of chest pathology. The proposed approach is a part of an auditing system that evaluates radiology reports to decrease incorrect diagnoses. Our study revealed that the proposed Bi-CNN outperforms convolutional neural network, random forest and support vector machine methods.