Why Deep Learning Isn’t Always the Best Option
And what to use instead.
Deep learning — a subset of machine learning where big data is used to train neural networks — can do incredible things.
Even amidst all the mayhem of 2020, deep learning brought astonishing breakthroughs in a variety of industries, including natural language (OpenAI’s GPT-3), self-driving (Tesla’s FSD beta), and neuroscience (Neuralink’s neural decoding).
However, deep learning is limited in several ways.
Deep Learning Lacks Explainability
In March 2018, Walter Huang was driving his Tesla on Autopilot in Mountain View, when it suddenly crashed into a safety barrier at 70mph, taking his life.
Many AI systems today make life-or-death decisions, not just self-driving cars. We trust AI to classify cancers, track the spread of COVID-19, and even detect weapons in surveillance camera systems.
When these systems fail, the cost is devastating and final. We can’t bring back a human life. Unfortunately, AI systems fail all the time. It’s called “error.” When they fail, we want explanations. We want to understand the why.
However, deep neural networks and ensembles can’t easily give us the answers we need. They’re called “black box” models, because we can’t look through them.
Transparency isn’t just critical in life-or-death systems, but in everyday financial models, credit risk models, and so on. If a middle-aged person saving for retirement suddenly loses their financial safety net, there better be explainability.
Deep Learning Has a Propensity to Overfit
Overfitting is when a model learns the training data well, but fails to generalize to new data. For instance, if you were to build a trading model to predict financial prices using a neural network, you’ll inevitably come up with an overly-complex model that has high accuracy on the training data, but fails in the real world.
In general, neural networks — particularly deep learning — are more susceptible to overfitting than simple models like logistic regression.
“In logistic regression, the model complexity is already low, especially when no or few interaction terms and variable transformations are used. Overfitting is less of an issue in this case... Compared to logistic regression, neural network models are more flexible, and thus more susceptible to overfitting.”
Deep Learning is More Expensive
Building deep learning models can be expensive, as AI talent income easily runs into the six figures. It doesn’t stop there. Deploying deep learning models is expensive as well, as these large, heavy networks consume a lot of computing resources.
For instance, as of writing, OpenAI’s GPT-3 Davinci, a natural language engine, costs $0.06 per 1,000 tokens. This may seem very cheap, but these costs quickly add up when you’re dealing with thousands or even millions of users.
Let’s compare with traditional machine learning models.
Making a prediction with a 2-layer neural network on a CPU costs around 0.0063 Joules, or 0.00000000175 kWh. For all intents and purposes, the cost of a single prediction is negligible.
The Solution — Explainable, Simple, Affordable Models
Fortunately, it’s easier than ever to create explainable, simple, and affordable machine learning models, using a technique called AutoML, or automated machine learning, which automatically creates a variety of machine learning models given a dataset, and selects the most accurate model.
AutoML isn’t a new phenonmenon, but it has become especially easy in recent years due to the rise of no-code, enabling effortless machine learning tools like Obviously.AI.
In 2010, MIT discussed a “common computer science technique called automated machine learning,” but back then, you’d still need developers to use AutoML tools.
Today, anyone can build and deploy explainable, simple, and affordable AI without any coding or technical skills.
