Seismic inversion is an approach to obtain the physical properties of the earth layers from the seismic data which aids in reservoir characterization. In seismic inversion, spatially variable physical parameters such as impedance (Z), wave velocities (Vp, Vs) and density can be determined from the seismic data. Among these, impedance is an important parameter used for lithology interpretation. However, the inversion problem is nonlinear and ill-posed due to unknown seismic wavelet, observed data band limitation, and noise. This requires complex wave equation analysis, prior assumptions, human expert effort, and time to analyze the seismic data. To address these issues, deep learning methods were deployed to solve the seismic inversion problem. In this paper, we develop a deep learning framework with an attention module for seismic impedance inversion. The relevant features from the seismic data are emphasized with the integration of the attention module into the network. First, we train the attention-based deep convolutional neural network (ADCNN) by supervised learning with pre-defined acoustic impedance (AI) labels. Next, we train the ADCNN in an unsupervised way with the physics of the forward problem. In the proposed method, the predicted AI is used to calculate the seismic data (calculated seismic) and error is minimized between the input seismic data and calculated seismic data. Unsupervised learning has an advantage when the labeled data is inadequate. The proposed network is trained with Marmousi 2 dataset, and the predicted experimental results show that the proposed method outperforms in comparison to the existing state-of-the-art method.