Work stream 5: Edge Gateway

Overview

The Edge Gateway functions as the "nervous system" for the InfiniEdge AI project. It is designed to seamlessly connect AI models with your IoT devices at the edge.

Stakeholders:

• @Tom Qin 

• @Kris He 

• @saiyan68 

Objectives

• This work stream aims to optimize the device discovery experience at the edge, enabling edge gateways to automatically discover IoT devices using various protocols (S7, OPC UA, HTTP, etc.) and deploy the necessary resources so AI can interact with them.

  • Support more protocols like ROS/ROS2 to better serve embodied intelligence

  • Support LwM2M protocol

Approach

Shifu’s approach to adding device discovery capabilities is to integrate Project Akri (https://github.com/project-akri/akri). Akri will serve as Shifu's device management plane, while Shifu will handle the control and data planes.

You can view the approach and proposal here:

• https://docs.google.com/presentation/d/1IB2qr1dYOxNVGVOeMB1zwoO2g9HTnvlpx2aaAsijUXg/edit?usp=sharing

Shifu’s LwM2M support proposal

• https://github.com/Edgenesis/shifu/pull/878

• https://github.com/Edgenesis/shifu/pull/879

Progress

• First video demo  https://youtu.be/tRk5pqiVJVA?si=syFt8Hl3Izhul3WT

• 10/22

  • LwM2M integration done

    • https://github.com/Edgenesis/shifu/pull/1002

    • https://github.com/Edgenesis/shifu/pull/1001

  • Start working on tests and documentation

  • Begin working with ROS2

• 10/29

  • https://github.com/pytorch/executorch/blob/main/examples/models/llama/README.md running LLM on device guide.

  • Working with Meta team Running quantized llama 3.2 model Introducing quantized Llama models with increased speed and a reduced memory footprint on S23 cell phone and try Edge AI function calling inference on device. The AI power are mostly on CPU now.

  • Continue waiting on another quantized model which running AI inference on NPU.