In this paper, we propose a novel ensemble-based classification system for improving the classification accuracy of hyperspectral images. To generate the ensemble, we run the mean-shift (MS) algorithm several times on different bands randomly selected from the hyperspectral cube and with distinct kernel width parameters. The resulting set of MS maps are then successively labeled via a pair wise labeling procedure with respect to a spectral-based classification map generated by the support vector machine (SVM) classifier.