AI at the Edge, beyond the hype!
Artificial Intelligence at the Edge utilizes a compact architecture, yet it offers an incredible computing approach that attempts to drive local data-informed decision-making which once was primarily Cloud-based.
From the first network ATMs in the 1970’s, to the World Wide Web in the 90’s, and on up to smart meters in the early 2000’s, we’ve come a long way. Cloud computing has gotten a lot of the news in recent years, but the edge is also growing in importance as the IoT (Internet of Things) pool keeps expanding. According to, Per Intel®, IoT growth has gone from 2 billion devices in 2006 to a projected 200 billion by 2020.
And then like any other field, IoT is also not left out for being one of the use-cases of Artificial Intelligence (AI) as the worldwide number of AI edge devices is forecast to jump from 161.4 million units in 2018 to 2.6 billion units by 2025, according to Tratica.
Believe you have got some interest in knowing more about this topic after reading these gigantic numbers! :D
You’ll get answers to the following questions in this post:
— What is AI at the Edge?
— Why is AI at the Edge important?
— How is it different from Cloud Computing?
— What are all the applications of AI at the Edge already in use?
— When to opt for an Edge-based application over Cloud-based?
— Who are all the players working on the hardware and software development of AI at the Edge?
— Will Edge Computing replace Cloud Computing?
But what it actually means by AI at the Edge and why is it even needed?
Many applications with the cloud get data locally, send the data to the cloud, process it, and send it back. The edge means there’s no need to send to the cloud; it can often be more secure (depending on edge device security) and have less impact as well as dependency on a network. Thus, “Edge” means local (or near local) processing. This can be an actual local device like a smart refrigerator, or servers located as close as possible to the source (i.e. servers located in a nearby area instead of on the other side of the world :P).
AI at the edge is needed because of the following reasons,
- Network impacts — network communication can be expensive (bandwidth, power consumption, etc.) and sometimes impossible (think remote locations or during natural disasters)
- Latency consideration — Real-time processing is necessary for applications, which can’t handle latency in making important decisions
- Security concerns — Edge applications could be using personal data (like health data) that could be sensitive if sent to the cloud
- Optimizations for local interface — Optimization software, specially made for specific hardware, can help achieve great efficiency with edge AI models
The impact of model compression techniques like Google’s Learn2Compress that enables squeezing large AI models into small hardware form factors is also contributing to the rise of AI edge processing.
Federated learning and blockchain-based decentralized AI architectures are also part of the shift of AI processing to the edge with part of the training also likely to move to the edge.
So, When to opt for an edge application?
- When there is a need for real-time decision-making, AND
- Where low latency (minimum delay) is necessary, AND
- Where the network itself may not always be available
Let’s take the example of self-driving cars or any autonomous vehicle in general to understand edge applications in a bit detail. Such vehicles need sensory input devices like cameras, radar, and lasers to allow the vehicle to perceive the world around it, taking complex decisions based on the analysis of gathered data, on the fly. What do you think, whether self-driving cars most likely run using the cloud or at the edge?
Give a pat on your back if your answer is, on the edge! Why? Think that, do self-driving cars have the ability to wait on sending data and receiving analysis reports from the cloud as they drive? What if there is a network latency for even a couple of milliseconds?
There are numerous other examples where we are un-knowingly using Edge AI. From Google maps alarming you about bad traffic to your smart refrigerator reminding you to buy some missing dairy stuff to your smartphone recognizing your face to unlock itself to virtual assistants like — Alexa, Siri, Cortana, Google Assistant, Edge-AI is everywhere around us.
Depending on the AI application and device category, there are several hardware options for performing AI edge processing. The options include CPUs, GPUs, TPUs, Application-Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGA) and System-on-a-Chip (SoC) accelerators.
Note that, Edge AI algorithms can still be trained in the cloud, but get run at the edge.
Players currently working on the hardware and software development of AI at the Edge:
The companies, large and small, that focus on IoT edge hardware include BrainChip (Akida Neuromorphic System-on-Chip), CEVA (the NeuPro family), Google (Edge TPU), GreenWave (AI processor GAP8), Huawei (Ascend Chips), Intel (Xeon), NVIDIA (Jetson TX2), Qualcomm (Vision Intelligence Platforms), and STMicroelectronics (STM32 microcontrollers).
Smaller companies tend to focus on IoT edge software. Some focus on learning like Ekkono, FogHorn, and Swim (a cloud-based POS), while others target inference such as Renesas (e-AI). Many companies also develop software with both capabilities, such as Amazon (AWS Greengrass ML Inference model), BrainChip (Studio Software), Google (Cloud IoT Edge), Huawei (Atlas platform), and IBM (Watson IoT Platform).
Will Edge Computing replace Cloud Computing?
Even though edge analytics is an exciting area, it should not be viewed as a potential replacement for cloud/central data analytics. Both can and will supplement each other in delivering data insights and both models have their place in organizations. One compromise of edge analytics is that only a subset of data can be processed and analyzed at the edge and only the results may be transmitted over the network back to central offices. This will result in the ‘loss’ of raw data that might never be stored or processed. So edge analytics is good if this ‘data loss’ is acceptable. On the other hand, if the latency of decisions (& analytics) is not acceptable as in-flight operations or critical remote manufacturing/energy, edge analytics should be preferred.
There are nearly endless possibilities with the edge. But not every single app needs it — you can likely wait for a second while your voice app goes to ask the server a question, or such as when NASA engineers are processing the latest black hole data!
The Final Say
Earlier, ground-breaking AI applications required a massive and expensive datacenter to function, but an AI-enabled edge computing device can reside practically anywhere. This reality doesn’t imply that edge computing will be a replacement for cloud computing. With the digital world becoming more interconnected, in any case, real-time intelligence will shift to the edge. It is undeniable that AI at the edge offers endless opportunities that can help ventures to all the more effectively drive tasks and increment efficiency.
Stay tuned for the upcoming blog post on Leveraging Pre-Trained Models to build powerful Edge applications, without the need to train your own model.
Happy Learning! :)
