Nowadays, it is still challenging for the blind image quality assessment (BIQA) to accurately predict quality scores of distorted images, since distorted images have rich content information and complex distortions. To solve these problems, a content perception and distortion inference network for BIQA is proposed, which divides IQA task into content perception and distortion inference processes. Since humans try to understand image content before perceiving quality scores, a content feature extractor is designed to explore content information in an image to deal with the content variation problem. To handle the distortion diversity problem, a distortion feature extractor is proposed to capture distortion features in images. Because extracted content features and distortion ones have different characteristics, attention-based fusion blocks to fuse multi-scale content features and distortion ones as guidance to selectively enhance important features based on calculated weight scores are proposed. With fused features, a quality prediction module is designed to regress multi-scale features to quality scores. Experiments are performed on six public IQA datasets, including LIVE, CSIQ, TID2013, LIVEC, KonIQ-10k, and SPAQ. Experimental results show that the proposed method can effectively predict quality scores for both synthetically and authentically distorted images than its peers, including the state-of-the-art methods.