South Korea’s agriculture is characterized by a mixture of various cultivated crops. In such an agricultural environment, convergence technology for ICT (information, communications, and technology) and AI (artificial intelligence) as well as agriculture is required to classify objects and predict yields. In general, the classification of paddy fields and field boundaries takes a lot of time and effort. The Farm Map was developed to clearly demarcate and classify the boundaries of paddy fields and fields in Korea. Therefore, this study tried to minimize the time and effort required to divide paddy fields and fields through the application of the Farm Map. To improve the fact that UAV image processing for a wide area requires a lot of time and effort to classify objects, we suggest a method for optimizing cultivated crop recognition. This study aimed to evaluate the applicability and effectiveness of machine learning classification techniques using a Farm Map in object-based mapping of agricultural land using unmanned aerial vehicles (UAVs). In this study, the advanced function selection method for object classification is to improve classification accuracy by using two types of classifiers, support vector machine (SVM) and random forest (RF). As a result of classification by applying a Farm Map-based SVM algorithm to wide-area UAV images, producer’s accuracy (PA) was 81.68%, user’s accuracy (UA) was 75.09%, the Kappa coefficient was 0.77, and the F-measure was 0.78. The results of classification by the Farm Map-based RF algorithm were as follows: PA of 96.58%, UA of 92.27%, a Kappa coefficient of 0.94, and the F-measure of 0.94. In the cultivation environment in which various crops were mixed, the corn cultivation area was estimated to be 96.54 ha by SVM, showing an accuracy of 90.27%. RF provided an estimate of 98.77 ha and showed an accuracy of 92.36%, which was higher than that of SVM. As a result of using the Farm Map for the object-based classification method, the agricultural land classification showed a higher efficiency in terms of time than the existing object classification method. Most importantly, it was confirmed that the efficiency of data processing can be increased by minimizing the possibility of misclassification in the obtained results. The obtained results confirmed that rapid and reliable analysis is possible when the cultivated area of crops is identified using UAV images, a Farm Map, and machine learning.