Ground Penetrating Radar (GPR) is a useful tool in detecting subsurface object or hidden structure defects However, the time-consuming problems and high requirement of professional manpower is required to analyse the GPR data. Machine learning is a tool that endowed with the ability to learn, and it can reduce time taken for the GPR data analysing. To simplify the identification process, a framework is proposed to classify the size of underground metallic pipe by using Histogram of Oriented Gradient (HOG) as a feature extraction algorithm. Two machine learning algorithms namely Support Vector Machines (SVM) and Backpropagation Neural Network were proposed to classify the size of the underground metallic pipe. As a result, the accuracy from the identification is more than 98% for both classifier algorithm.