The conditional random field is a suitable framework for contextual classification of two-dimensional (images) and three-dimensional (point clouds) data. This framework is based on probabilistic graphical models, an alternative representation of a conditional probability distribution over random variables. In general, a graphical model encodes probabilistic relationships between random variables by their edges. However, graph structure (set of edges) is not always known in advance. In particular, if we consider each point of a point cloud as a graph node, we might not have information about nodes interaction (i.e. graph structure is unknown). Given that there is no agreement about what structure to use for point cloud contextual classification, we focused on determining a suitable graph structure by comparing the performance of four different graph structures and twenty feature sets. All experiments were performed in urban environments. The quantitative errors (type I, type II and total error) and classification accuracy were the metrics we used to evaluate the performance of Conditional Random Fields. Results suggest that optimal neighbor and 3D Delaunay structures achieved best classification performance. Theses structures combined with point-slope, curvature, and segment height node based features performed a classification accuracy error of 93.5% and the best type I error, 1.56%.INDEX TERMS Conditional random field, graph structure, ground filtering, machine learning.