BackgroundPreoperative differentiation between malignant and benign soft‐tissue masses is important for treatment decisions.Purpose/HypothesisTo construct/validate a radiomics‐based machine method for differentiation between malignant and benign soft‐tissue masses.Study TypeRetrospective.PopulationIn all, 206 cases.Field Strength/SequenceThe T1 sequence was acquired with the following range of parameters: relaxation time / echo time (TR/TE), 352–550/2.75–19 msec. The T2 sequence was acquired with the following parameters: TR/TE, 700–6370/40–120 msec. The data were divided into a 3.0T training cohort, a 1.5T MR validation cohort, and a 3.0T external validationcohort.AssessmentTwelve machine‐learning methods were trained to establish classification models to predict the likelihood of malignancy of each lesion. The data of 206 cases were separated into a training set (n = 69) and two validation sets (n = 64, 73, respectively).Statistical Tests1) Demographic characteristics: a one‐way analysis of variance (ANOVA) test was performed for continuous variables as appropriate. The χ2 test or Fisher's exact test was performed for comparing categorical variables as appropriate. 2) The performance of four feature selection methods (least absolute shrinkage and selection operator [LASSO], Boruta, Recursive feature elimination [RFE, and minimum redundancy maximum relevance [mRMR]) and three classifiers (support vector machine [SVM], generalized linear models [GLM], and random forest [RF]) were compared for selecting the likelihood of malignancy of each lesion. The performance of the radiomics model was assessed using area under the receiver‐operating characteristic curve (AUC) and accuracy (ACC) values.ResultsThe LASSO feature method + RF classifier achieved the highest AUC of 0.86 and 0.82 in the two validation cohorts. The nomogram achieved AUCs of 0.96 and 0.88, respectively, in the two validation sets, which was higher than that of the radiomic algorithm in the two validation sets and clinical model of the validation 1 set (0.92, 0.88 respectively). The accuracy, sensitivity, and specificity of the radiomics nomogram were 90.5%, 100%, and 80.6%, respectively, for validation set 1; and 80.8%, 75.8%, and 85.0% for validation set 2.Data ConclusionA machine‐learning nomogram based on radiomics was accurate for distinguishing between malignant and benign soft‐tissue masses.Evidence Level3Technical EfficacyStage 2 J. Magn. Reson. Imaging 2020;52:873–882.