Common object detection models consist of classification and regression branches, due to different task drivers, these two branches have different sensibility to the features from the same scale levels and the same spatial locations. The pointbased prediction method, which is based on the assumption that the point with high classification confidence has the high regression quality, leads to the misalignment problem. According to our analyses, the problem is further composed of scale misalignment and spatial misalignment specifically. We aim to resolve the phenomenon at minimal cost-a minor adjustment of the head network and a new label assignment method replacing the rigid one. Our experiments show that, compared to the baseline FCOS, a one-stage and anchor-free object detection model, our model consistently get around 3 AP improvement with different backbones, demonstrating both simplicity and efficiency of our method. Our code will be available at https://github.com/HaoGood/MMMC.