Required Information 

Responses (Please list N/A if not applicable) 

Name of Project 

InfiniEdge AI

Project Description (what it does, why it is valuable, origin and history) 

InfiniEdge AI is a project that brings AI to the edge, enabling real-time AI inference. This technology extends the advantages of AI and Machine Learning to edge devices, thus aligning with the LF Edge Mission Statement. This technology can enhance applications in various sectors including manufacturing, telecommunications, healthcare, automotive (autonomous driving and smart cockpit), retail and etc.

InfiniEdge AI originated as part of the AI Edge Blueprint Family (https://wiki.akraino.org/display/AK/The+AI+Edge+Blueprint+Family) and is based on the Shifu Framework (https://github.com/Edgenesis/shifu) and YoMo (https://github.com/yomorun/yomo)

• Objective of the InfiniEdge AI Project
  • Compared running on centralized mega data centers, InfiniEdge AI Project aims to bring AI inference closer to end users, achieving low-latency response, cost-optimized inference and enhanced privacy protection.
• Importance of integrating AI at the edge
  • Real-time decision making: By Integrating AI and IoT at the edge, we can analyze data right at the edge and make immediate decisions. This eliminates the delay caused by transmitting data back to the cloud or a centralized server for processing. In cases like autonomous driving and industrial 4.0 this can be critical.
  • Data/Traffic management: Edge computing units generate large amount of data. Instead of sending them to the centralized cloud, edge AI can process data locally, deciding what to send for further analysis and what to discard, leading to more efficient data management. Processing data at the edge reduces the amount of data that needs to be sent over the network, saving on bandwidth costs and reducing network traffic. 
  • Security: With AI at edge. Data can be processed without ever leaving the edge. Reducing the risk of privacy violation and data breaches.
• Problem Statement: 
  • Data can not be transferred to AI model in time (e.g. real-time speech recognition by OpenAI whisper, demo: https://edge-ai.yomo.run/).
  • The cost of building and maintaining production-grade AI inference services is high (the infra of running AI inference is different from AI training).
  • More sensitive data will be involved when using AI (e.g. biological data when using speech recognition).

  • Our project aims to enable efficient inference on resource-constrained edge devices by utilizing large models to train and generate smaller models. Edge devices often face limitations in computing power, memory, and energy resources, while modern deep learning models tend to be large and computationally intensive. Directly deploying complex models on edge devices leads to poor performance and high energy consumption, negatively impacting user experience and device practicality.

    To address this challenge, we employ knowledge distillation to assist in training and generating compact small models. Knowledge distillation transfers the knowledge of the large model to the smaller one, enabling the small model to perform similarly to the large model while using fewer computational resources. This approach not only facilitates efficient inference on edge devices but also allows complex tasks to be offloaded to cloud-based large models, fully leveraging the advantages of both edge and cloud computing.

• Goal of the project: 
  • To create a unifying platform for running AI inference on the edge. 
• Expected benefits:  
  • Low-latency processing capabilities.
  • Cost Efficiency.
    • Pre-process / AI Predict / Post-process running on different systems.
    • Optimize AI Model for heterogeneous systems.
  • Improved privacy protections.

Statement on alignment with Foundation Mission Statement 

InfiniEdge AI aligns with the LF Edge Mission Statement by creating an open, scalable, and interoperable framework for edge computing. This project embodies LF Edge's vision for edge applications by extending AI and Machine Learning benefits to edge devices.

High level assessment of project synergy with existing projects under LF Edge, including how the project compliments/overlaps with existing projects, and potential ways to harmonize over time. Responses may be included both here and/or in accompanying documentation.  

InfiniEdge AI enhances the overall LF Edge ecosystem by providing an AI/ML interface for edge devices. It does not overlap significantly with existing projects but brings unique capabilities to the table. Harmonization potential exists with IoT and edge computing-focused projects.

Link to current Code of Conduct 

2 TAC Sponsors, if identified (Sponsors help mentor projects) - See full definition on Project Stages: Definitions and Expectations 

Project license 

Apache 2.0

Source control (GitHub by default) 

GitHub

Issue tracker (GitHub by default) 

External dependencies (including licenses) 

Tom Qin 

Release methodology and mechanics 

Names of initial committers, if different from those submitting proposal 

Current number of code contributors to proposed project 

6

Current number of organizations contributing to proposed project 

4 (Baidu, Allegro, Edgenesis, TikTok)

Briefly describe the project's leadership team and decision-making process 

List of project's official communication channels (slack, irc, mailing lists) 

N/A

Link to project's website 

N/A

Links to social media accounts 

N/A

Existing financial sponsorship 

N/A

Infrastructure needs or requests (to include GitHub/Gerrit, CI/CD, Jenkins, Nexus, JIRA, other ...) 

GitHub 

Currently Supported Architecture 

x86-64, AArch64 

Planned Architecture Support 

N/A 

Project logo in svg format (see https://github.com/lf-edge/lfedge-landscape#logos for guidelines) 

N/A

Trademark status 

N/A 

Does the project have a Core Infrastructure Initiative security best practices badge? (See: https://bestpractices.coreinfrastructure.org) 

No 

Any additional information the TAC and Board should take into consideration when reviewing your proposal? 

N/A