The AI Edge Engineer: Extending the power of CI/CD to Edge devices using containers
At the Artificial Intelligence – Cloud and Edge implementations course – I have been exploring the idea of extending CI/CD to Edge devices using containers. In this post, I present these ideas under the framework of the ‘AI Edge Engineer’. Note that the views presented here are personal. I seek your comments if you are exploring similar ideas – especially if you are in academia / research. We are happy to share insights/code as we develop it. You can connect with me on linkedin HERE
Most AI Edge applications today are based on Computer Vision and Voice.
What is in the image or video?
Give me directions to the nearest local branch.;
Convert spoken audio to text;
Natural Language Processing etc
are all cognitive applications, often running on the edge, that do not need you to be connected to the Cloud as long as the trained model is deployed to the edge devices.
But, we are seeing other examples for AI Edge (Edge AI is going beyond voice and vision).
As applications for AI and Edge computing mature, we need more mature development models which unifies both the Cloud and the Edge devices.
In this post, I discuss models of development for AI Edge Engineering based on deploying containers to Edge devices which unifies the Cloud and the Edge.
I call this role ‘AI Edge Engineer’.
The primary focus of the AI Engineer role is to unify development between the Cloud and the Edge using containers so as to deploy and maintain code to Edge devices. Note that in this case, the device lifecycle is still the same: i.e. plan, provision, monitor and retirement. The device also undergoes other states such as reboot, factory reset, configuration, firmware update and status reporting.
The AI Edge Engineer analyses, designs and implements cognitive AI solutions that span the Cloud and Edge devices. From a developer standpoint, AI Edge deployment implies the ability to move cloud and custom workloads to the edge, securely and seamlessly. It involves the ability to run advanced analytics on Edge devices where the model is trained on the cloud and deployed on edge devices. The AI Edge deployment is configured, updated and monitored from the cloud - with a secure solution deployed end-to-end from chipset to cloud
Providing offline support is a key part of AI Edge Engineering. In practise this means - Once you sync with the IoT Hub(cloud) – the Edge device can run independently and indefinitely with messages being queued for deferred cloud delivery
Implementing the AI Edge ecosystem is an engineering led problem i.e. it extends beyond the data science aspect (writing algorithms) because it includes an understanding of devops. We briefly describe the architecture of such a solution based on Azure. (source Azure documentation):
Currently, most IoT applications use a simple architecture where they connect to the Cloud directly i.e. do not undertake edge computing. Increasingly, almost all future IoT applications will involve the ability to process data on edge devices. The requirement to run functionality on edge devices will also require us to run and maintain code on edge devices. Modules will be run on devices using the container mechanism. Hence, the AI Edge Engineer functionality will involve running modules on Edge devices through containers. We describe this mechanism below for a simple telemetry application which is pre-developed.
The steps are
You could expand this idea to
It is now possible to deploy a CI/CD ready IoT Edge project. The availability of a CI/CD pipeline could help developers deliver value faster with rapid updates i.e. extend the power of CI/CD to Edge devices.
This helps developers to create use cases such as:
While the deployment of containers on Edge devices may seem a nascent application, I believe that soon, as intelligence starts to be embedded into devices, many large scale IoT / Edge applications would use this architecture (for example autonomous vehicles. Drones etc). Today, a few applications deploy AI to complex edge devices ex Ocado – but the future may be different. This is broadly true of Telecoms in a post-5G world as can be seen by the Azure AT&T partnership and the Azure – Reliance Jio partnership.
I seek your comments if you are exploring similar ideas – especially if you are in academia / research. We are happy to share insights/code as we develop it. You can connect with me on linkedin HERE