Ensemble methods are widely applied in classification problems. Ensemble methods combine results from multiple classifiers to overcome the possible deficiency of any single classifier. One important question is how to construct an ensemble system so that it can utilize all individuals most efficiently to improve classification results. An ensemble system thus needs some level of diversity in terms of error among individuals to avoid group mistakes. Novelty Search is a recently published approach in evolutionary computation in which individuals evolve based on a novelty metric, which evaluates how different their behavior is in addition to an objective metric that shows how correct their behavior is. This paper will apply the novelty search to generating classifiers for use in ensemble systems and compare this approach with other published results.