Clusterings should not be compared by visual inspection: response to Gagné & Proulx

In Heikinheimo et al. (Journal of Biogeography, 2007, 34, 1053-1064) we used clustering to analyse European land mammal fauna. Gagné & Proulx criticized our choice of the Euclidean distance measure in the analysis, and advocated the use of the Hellinger distance measure, claiming that this leads to very different clustering results. The criticism fails to take into account the probabilistic nature of the methods used and the fact that in this case the similarity measures correlate strongly. Gagné & Proulx used subjective inspection as the criterion of similarity between clusterings. We show that this is insufficient and misleading. Namely, owing to the local minimum problem, two clustering runs rarely give identical results. In the case of our study, the measured similarity (using the kappa statistic) between the Euclidean- and Hellinger-based clusterings is roughly equal to the similarity between two clusterings that both use the Hellinger distance but different random initialization points.