What is the importance of diversity?
In nature, diversity allows biological organisms to occupy all niches, to master every environment, and exploit every resource. There is presently no single organism which can do this alone. Phototrophs are phototrophs. Lithotrophs are lithotrophs. Herbivores are strictly herbovores. Carnivores are mostly carnivores. Extremophiles generally occupy their own extremes, not those of others.
People, it has been argued – with the aid of the tools we are capable of developing (various fuel burning engines, pressurized submersibles, thermal insulation, etc) – come closest to being a “perfect” organism. In reality our efforts are tenuous and clunky at best: for sure, no other organism that we know of has ventured from the Earth to the Moon, but it’s not as if we were able to fit right in when we go there.
And so it is with algorithms.
We are not yet at the stage where we have a single, all-dancing, no-free-lunch-defying Artificial General Learning Machine that can solve all of our problems. But what we do have at our disposal is a diversity of approaches to machine learning.
Let us focus for a second on that. Let us acknowledge, and accept, that algorithm A has poor regularization characteristics when compared to algorithm B, that algorithm B makes naïve assumptions when compared to algorithm C, that algorithm D is not scalable beyond a few hundred dimensions, and that algorithm D defies the principles of Bayesian inference.
Rather than denigrating every algorithm which has been demonstrated in some way inferior to another (such demonstrations abound in the academic literature), as if there is a tacit agreement that we are all searching for a single, ultimate, algorithm… let us instead observe that what we have on our hands are a wealth of approaches, which, collectively, can deal with a multitude of problems. If it helps to take the heat off of our solutions, let us instead brand the occasional problem as being “uncharacteristic”, “easy” or “ill-formed” (no-free lunch theorists ought to like that).
Ensemble theory tells us it is in fact this diversity that puts us in the strongest position.
After all, although no single biological organism has conquered every niche, evolution – through the diversity of the organisms it has brought to bare – would in fact seem to have conquered almost every niche. Even we humans – the apparent best single solution – are in fact a complex ecology, a co-ordinated walking symbiosis of various microbial species and mitochondrial add-ons. Maybe one day nature will come up with a single “perfect” organism, but that hasn’t been the shape of its solutions to date (it may not surprise you that the concept no-free-lunch was previously coined by an Ecologist).
Likewise then, it seems reasonable that research into artificial solutions may have a long journey ahead of it; but that – for the time-being at least – we should not be hung up on the pursuit of a single ultimate neatly-packaged logic, and instead embrace the messy plurality.
It may actually be our greatest strength.