Neural Networks Are Not Evil

I recently read an interview with Gary Marcus, a professor of psychology and neural science at NYU. In the interview, he questions whether the big data approach to AI–that of training neural networks on terabyte-scale datasets–is successfully moving humanity towards the intelligent, sentient computers that we all dream of.

Roughly $1.\overline{3}$ years have passed since that interview, and Marcus has now resolved his own question with a piece in the New York Times: “trendy” neural networks are not moving us in the right direction. He says this is because they lack basic concepts of the world that humans seem to learn by osmosis, like the natural laws of physics or linguistics.

I’d argue Marcus is jumping ahead a bit–nobody is saying that neural networks are the only key to AI’s future. I imagine an AI system in the future will encompass multiple systems, much like the human brain: a neural network for pattern recognition, a memory store for recalling things it has learned in the past, a physics simulator, a sentiment analyzer for understanding emotion. Most of these things already exist, and perform pretty well independently. The exercise now becomes piecing all of them together.

One other point I think Marcus fails to address is that of quantum computing. While it’s correct to say that our current AI methods are held back by computational resources, it’s incorrect to say that there’s no solution to that on the horizon. Ask anyone who’s working on quantum right now and they’ll say that the technology will transform the way we live within our lifetime. I think the computational limitations Marcus alludes to will be lifted by quantum computers, and conversations I’ve had with quantum researchers suggest the same thing.

Large scale collaborations are all well and good. But I think preparing AI research for the advent of practical quantum computers is the most important next step for the discipline.