Application of Multiple Unsupervised Models to Validate Clusters Robustness in Characterizing Smallholder Dairy Farmers

Abstract

The heterogeneity of smallholder dairy production systems complicates service provision, information sharing, and dissemination of new technologies, especially those needed to maximize productivity and profitability. In order to obtain homogenous groups within which interventions can be made, it is necessary to define clusters of farmers who undertake similar management activities. This paper explores robustness of production cluster definition using various unsupervised learning algorithms to assess the best approach to define clusters. Data were collected from 8179 smallholder dairy farms in Ethiopia and Tanzania. From a total of 500 variables, selection of the 35 variables used in defining production clusters and household membership to these clusters was determined by Principal Component Analysis and domain expert knowledge. Three clustering algorithms, K-means, fuzzy, and Self-Organizing Maps (SOM), were compared in terms of their grouping consistency and prediction accuracy. The model with the least household reallocation between clusters for training and testing data was deemed the most robust. Prediction accuracy was obtained by fitting a model with fixed effects model including production clusters on milk yield, sales, and choice of breeding method. Results indicated that, for the Ethiopian dataset, clusters derived from the fuzzy algorithm had the highest predictive power (77% for milk yield and 48% for milk sales), while for the Tanzania data, clusters derived from Self-Organizing Maps were the best performing. The average cluster membership reallocation was 15%, 12%, and 34% for K-means, SOM, and fuzzy, respectively, for households in Ethiopia. Based on the divergent performance of the various algorithms evaluated, it is evident that, despite similar information being available for the study populations, the uniqueness of the data from each country provided an over-riding influence on cluster robustness and prediction accuracy. The results obtained in this study demonstrate the difficulty of generalizing model application and use across countries and production systems, despite seemingly similar information being collected.

A cluster analysis of variables essential for climate change adaptation of smallholder dairy farmers of Nandi County, Kenya

“…Unsupervised learning algorithm was used for cluster analysis. This algorithms was K-means (Chibanda et al, 2009;Nyambo et al, 2019). In the analysis, the number of groups (K) represented how many farm typologies (clusters) could be defined for each dataset.…”

“…In the analysis, the number of groups (K) represented how many farm typologies (clusters) could be defined for each dataset. The number of clusters that best represented the data was determined using the Elbow method (where a bend or elbow in a graph showing decline of within cluster sum of squares differences as the number of clusters increases provides the best solution) (Nyambo et al, 2019) The elbow method examines the percentage of variance explained by the clustering as a function of the number of clusters k (Kingrani et al, 2017;Syakur et al, 2018). The Kmeans algorithm has been widely used in non-hierarchical clustering and characterizing smallholder dairy farms (Kingrani et al, 2017;Nyambo et al, 2019;Tittonell et al, 2010).…”

“…These features and their amounts were Boolean, Discrete and continuous and derived from household survey of 336 smallholder dairy farmers in Nandi County. These variables would be used to identify ‗natural groupings' of these 28 features to derive the number and type of clusters (Chibanda et al, 2009;Nyambo et al, 2019;Syakur et al, 2018). This was done by minimising the squared Euclidean distance within a decreasing number of clusters containing an increasing number of positively related variables and using Base R Package (RStudio V 1.1.442) within which dendrogram and plot showing optimal number of clusters using k means was generated (Chibanda et al, 2009).…”

“…Smallholder dairy farmers' adaptation to climate change would need the characterization of these farmers with more variables other than farm size as the only limiting factor (Auburger et al, 2015;Herrero et al, 2014;van Averbeke and Mohamed, 2006;Vrieling et al, 2011). There is need for the characterisation of smallholder dairy farmers with additional variables such as land use sizes of the farms, labour, type of livestock housing and other key variables that define smallholders' enterprise (Nyambo et al, 2019;Staal et al, 2002;Waithaka et al, 2007;Zake et al, 2010). These characterisation should use robust unsupervised models and integrated systems research to show natural groupings of smallholder dairy farmers to allow the policy makers to promote climate change adaptation practices (Nyambo et al, 2019;Oborn et al, 2017;Thompson, 2016).…”

“…There is need for the characterisation of smallholder dairy farmers with additional variables such as land use sizes of the farms, labour, type of livestock housing and other key variables that define smallholders' enterprise (Nyambo et al, 2019;Staal et al, 2002;Waithaka et al, 2007;Zake et al, 2010). These characterisation should use robust unsupervised models and integrated systems research to show natural groupings of smallholder dairy farmers to allow the policy makers to promote climate change adaptation practices (Nyambo et al, 2019;Oborn et al, 2017;Thompson, 2016). The lack of a characterisation of smallholder farmer with robust inclusion of key variables based on land survey limits options for policy makers as farm size reduces over time (Bebe et al, 2003).…”