AI is Stuck in 400 BC

The fourth century BC witnessed the height of Greece: Plato was writing the bane of intro to philosophy. And while many may be familiar with the Allegory of the Cave, the core assumptions behind modern artificial intelligence were also being produced.

Plato’s Theory of Forms had an enormous impact on western philosophy. Today it gives us insight into the intuition behind algorithms that make modern AI’s tick—machine learning—and many of the grand challenges facing machine learning researchers today have analogues in the insufficiencies of the Theory of Forms.

Simplified, the Theory of Forms goes something like this:

I have this collection of things, say elephants. Some of them are small, or have a broken tusk. Some are from Asia, some from Africa, but they’re all elephants. We say that each elephant is an instantiation of a form. The form being that of an elephant, with all of the properties of elephant-ness idealized to one canonical elephant: big, gray, mammal, with tusks and a trunk.

And although the canonical elephant does not exist in reality, it’s an intuitively pleasing concept, so much so that we implicitly teach children using some variation on Plato’s forms. Here’s a bunch of pictures of elephants, and here’s a bunch of pictures of zebras. Children can quickly recognize the pattern of properties that make an elephant an elephant, which is what we wanted to convey in the first place. It would be rather difficult to instead exhaustively list off the properties that make an elephant an elephant as opposed to say, a hippo.

This method of learning is the basis for modern image recognition algorithms. I tell the algorithm, “here’s a bunch of a pictures of elephants (e.g. Fig. 1); now, sort through them and find the pattern that’s common to all of them.” I test the algorithm, giving it pictures it hasn’t seen before, and when I’m satisfied with the results, I declare victory and ship an elephant-identifying AI.

What ends up happening under the algorithmic hood, however, is like a statistical approximation of Plato’s Forms. At a high level the algorithm turns the image into a patchwork of numbers and automates the process of statistical evaluation on various patterns in the picture: the relative frequencies of certain colors and shapes for example.

bowtiepre

Fig. 2: Vectorized samples of ImageNet data over their first three principal components.

In the case of image classification using a convolutional neural network, the data, vectorized images (literally lists of thousands of pixel intensities) are initially a scrambled mess (Fig. 2). If we take the first three principal components of the input data, say images of mortarboards, soccer balls, and bow ties (all very different items), the vectorized versions of these images don’t readily follow any discernible pattern.  But after we apply a learned composition of non-linear functions to our data, i.e. the neural network, we find that in the first principle components, our data clearly follows a readily distinguishable pattern (Fig. 3).

bowtiesvm

Fig. 3: Linearly separable image data from ImageNet in the final layer of a convolutional neural network tasked with classifying images by the primary object in it.

These regions of the transformed image space can be thought of as a non-unique approximation of where instantiations of object forms exist. The centroids of these clusters would be approximations of the ideal or canonical form itself, subject to the choice of representation (vectorized digital images in this case). In a sense, modern image recognition algorithms execute a statistical adaptation of Plato’s Forms. If I had every possible sample of a true soccer-ball I could characterize the entire region of soccer-ball space in vector representation, but thanks to statistics and the Law of Large Numbers for example, I only need a finite set of samples to get a decent idea of what the region looks like, e.g. Fig. 3.

But herein lies the problem. Plato’s Forms presented a number of problems (famously, the Third Man Argument, given by Plato himself), and if AI algorithms are ever to transcend, it needs to shed the limitations of its philosophy. We can poke holes in Forms by asking things like “how much do I need to take away from an elephant before its not an elephant anymore?”. Practically speaking, this is precisely what’s going on when an image recognition algorithm mistakes humans for animals, and not without serious social and economic consequences.

Regardless, the AI hype machine waxes prophetic about interpretability and representation learning in truly unsupervised settings given the current philosophy driving the statistics of AI. “Cars will drive themselves!” While there is indeed plenty of valuable ground still to be broken by current iterations of AI machinery, computer scientists should perhaps be thinking ahead in a more philosophical sense. Many scientists are calling for more ontological information or prior systematic knowledge being baked into machine learning algorithms, much like the rule or decision-based systems of early AI. While some computer scientists tend to roll their eyes at such suggestions (NLP, anyone?), perhaps this is a symptom of a critical metaphysical limitation of AI as it applies to representing and creating knowledge.

Modern machine learning algorithms are a fantastic vindication of Plato’s work and philosophy in general, but in order to advance the field of AI we should take a lesson from Plato himself and challenge the underlying philosophy.

P.S. If any students of philosophy find any egregious errors please feel free to slap me with your keyboard in the comments section.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s