Required Information 

Responses (Please list N/A if not applicable) 

Name of Project 

InfiniEdge AI

Project Description (what it does, why it is valuable, origin and history) 

InfiniEdge AI is a project that brings AI to the edge, enabling real-time AI inference. This technology extends the advantages of AI and Machine Learning to edge devices, thus aligning with the LF Edge Mission Statement. This technology can enhance applications in various sectors including manufacturing, telecommunications, healthcare, automotive (autonomous driving and smart cockpit), retail and etc.

InfiniEdge AI originated as part of the AI Edge Blueprint Family (https://wiki.akraino.org/display/AK/The+AI+Edge+Blueprint+Family) and is based on the Shifu Framework (https://github.com/Edgenesis/shifu) and YoMo (https://github.com/yomorun/yomo)

• Objective of the InfiniEdge AI Project

  • Compared running on centralized mega data centers, InfiniEdge AI Project aims to bring AI inference closer to end users, achieving low-latency response, cost-optimized inference and enhanced privacy protection.

• Importance of integrating AI at the edge

  • Real-time decision making: By Integrating AI and IoT at the edge, we can analyze data right at the edge and make immediate decisions. This eliminates the delay caused by transmitting data back to the cloud or a centralized server for processing. In cases like autonomous driving and industrial 4.0 this can be critical.

  • Data/Traffic management: Edge computing units generate large amount of data. Instead of sending them to the centralized cloud, edge AI can process data locally, deciding what to send for further analysis and what to discard, leading to more efficient data management. Processing data at the edge reduces the amount of data that needs to be sent over the network, saving on bandwidth costs and reducing network traffic. 

  • Security: With AI at edge. Data can be processed without ever leaving the edge. Reducing the risk of privacy violation and data breaches.

• Problem Statement: 

  • Data can not be transferred to AI model in time (e.g. real-time speech recognition by OpenAI whisper, demo: https://edge-ai.yomo.run/).

  • The cost of building and maintaining production-grade AI inference services is high (the infra of running AI inference is different from AI training).

  • More sensitive data will be involved when using AI (e.g. biological data when using speech recognition).

  • Our project aims to enable efficient inference on resource-constrained edge devices by utilizing large models to train and generate smaller models. Edge devices often face limitations in computing power, memory, and energy resources, while modern deep learning models tend to be large and computationally intensive. Directly deploying complex models on edge devices leads to poor performance and high energy consumption, negatively impacting user experience and device practicality.

    To address this challenge, we employ knowledge distillation to assist in training and generating compact small models. Knowledge distillation transfers the knowledge of the large model to the smaller one, enabling the small model to perform similarly to the large model while using fewer computational resources. This approach not only facilitates efficient inference on edge devices but also allows complex tasks to be offloaded to cloud-based large models, fully leveraging the advantages of both edge and cloud computing.

• Goal of the project: 

  • To create a unifying platform for running AI inference on the edge. 

• Expected benefits:  

  • Low-latency processing capabilities.

  • Cost Efficiency.

    • Pre-process / AI Predict / Post-process running on different systems.

    • Optimize AI Model for heterogeneous systems.

  • Improved privacy protections.

Statement on alignment with Foundation Mission Statement 

InfiniEdge AI aligns with the LF Edge Mission Statement by creating an open, scalable, and interoperable framework for edge computing. This project embodies LF Edge's vision for edge applications by extending AI and Machine Learning benefits to edge devices.

High level assessment of project synergy with existing projects under LF Edge, including how the project compliments/overlaps with existing projects, and potential ways to harmonize over time. Responses may be included both here and/or in accompanying documentation.  

InfiniEdge AI enhances the overall LF Edge ecosystem by providing an AI/ML interface for edge devices. It does not overlap significantly with existing projects but brings unique capabilities to the table. Harmonization potential exists with IoT and edge computing-focused projects.

Link to current Code of Conduct 

N/A

2 TAC Sponsors, if identified (Sponsors help mentor projects) - See full definition on Project Stages: Definitions and Expectations 

@Toshimichi Fukuda , Fujitsu; @Tina Tsou (Deactivated) , Arm

Project license 

Apache 2.0

Source control (GitHub by default) 

GitHub

Issue tracker (GitHub by default) 

GitHub

External dependencies (including licenses) 

@Tom Qin 

Release methodology and mechanics 

Names of initial committers, if different from those submitting proposal 

Liya Yu, Baidu @Yu, Liya 

C.C., Allegro @Weixiao Fan 

Jun Chen, Baidu @Jun Chen 

Tom Qin, Edgenesis @Tom Qin 

Yongli Chen, Edgenesis

Kevin Zheng, Edgenesis

Wenhui Zhang, ByteDance/TikTok @Wenhui Zhang 

Joe Speed, Ampere

Ray Chi, Advantech

Roger Chen, SuperMicro

Rick Cao, Meta

Reo Inoue, Fujitsu @inoue 

Ashok Bhat, Arm

Milos Puzovic, Arm

Tina Tsou, InfiniEdge AI, @Tina Tsou 

Qi Wang, Google

FeiMin Yuan, F5

Caleb Jiang, Applied Concept Inc. @Caleb 

Vijay Chintha, Comcast @vijay chintha 

Wilson Wang, ByteDance @Wilson Wang 

Sujata Tibrewala @Sujata Tibrewala 

Current number of code contributors to proposed project 

6

Current number of organizations contributing to proposed project 

4 (Baidu, Allegro, Edgenesis, TikTok)

Briefly describe the project's leadership team and decision-making process 

Ye Wang / Architect, Baidu

C.C. / CEO, Allegro @Weixiao Fan 

Yongli Chen / CEO, Edgenesis  

Advisors

@Ranny Haiby @Tina Tsou (Deactivated) 

List of project's official communication channels (slack, irc, mailing lists) 

N/A

Link to project's website 

N/A

Links to social media accounts 

N/A

Existing financial sponsorship 

N/A

Infrastructure needs or requests (to include GitHub/Gerrit, CI/CD, Jenkins, Nexus, JIRA, other ...) 

GitHub 

Currently Supported Architecture 

x86-64, AArch64 

Planned Architecture Support 

N/A 

Project logo in svg format (see https://github.com/lf-edge/lfedge-landscape#logos for guidelines) 

N/A

Trademark status 

N/A 

Does the project have a Core Infrastructure Initiative security best practices badge? (See: https://bestpractices.coreinfrastructure.org) 

No 

Any additional information the TAC and Board should take into consideration when reviewing your proposal? 

N/A

2 TAC Sponsors, if identified (Sponsors help mentor projects) - See full definition on Project Stages: Definitions and Expectations 

N/A

A presentation at an upcoming meeting of the TAC, in accordance with the project proposal requirements 

N/A

The typical IP Policy for Projects under the LF Edge Foundation is Apache 2.0 for Code Contributions, Developer Certificate of Origin (DCO) for new inbound contributions, and Creative Commons Attribution 4.0 International License for Documentation. Projects under outside licenses may still submit for consideration, subject to review/approval of the TAC and Board. 

Yes 

Upon acceptance, At Large projects must list their status prominently on website/readme 

Yes 

APIs 

Provide 

Cloud Connectivity 

Provide 

Container Runtime & Orchestration 

Consume 

Data Governance 

Provide, Consume 

Data Models 

Provide 

Device Connectivity 

Consume 

Filters/Pre-processing 

N/A 

Logging 

Consume 

Management UI 

Consume 

Messaging & Events 

N/A 

Notifications & Alerts 

N/A 

Security 

N/A 

Storage 

Provide, Consume, Facilitate

Customer Devices (Edge Nodes) 

N/A 

Customer Premises (DC and Edge Gateways) 

Support 

Telco Network Edge (MEC and Far-MEC) 

Support 

Telco CO & Regional 

Possible 

Cloud Edge & CDNs 

Cloud Edge – Support; CDNs: Possible 

Public Cloud 

Support 

Private Cloud 

Support 

Automotive / Connected Car 

✔

Chemicals 

✔

Facilities / Building automation 

✔  

Consumer 

✔ 

Manufacturing 

✔ 

Metal & Mining 

X 

Oil & Gas 

✔ 

Pharma 

X 

Health Care 

✔ 

Power & Utilities 

✔ 

Pulp & Paper 

X 

Telco Operators 

✔ 

Telco/Communications Service Provider (Network Equipment Provider) 

✔ 

Transportation (asset tracking) 

✔  

Supply Chain 

✔ 

Preventative Maintenance 

✔ 

Water Utilities 

X 

Security / Surveillance 

✔ 

Retail / Commerce (physical point of sale with customers) 

✔ 

Other - Please add if not listed above (please notify TAC-subgroup@lists.lfedge.org when you add one) 

No

Gateways (to Cloud, to other placements) 

✔ 

NFV Infrastructure 

X 

Stationary during their entire usable life / Fixed placement edge constellations / Assume you always have connectivity and you don't need to store & forward. 

 
✔ 

Stationary during active periods, but nomadic between activations (e.g., fixed access) / Not always assumed to have connectivity. Don't expect to store & forward. 

 
✔ 

Mobile within a constrained and well-defined space (e.g., in a factory) / Expect to have intermittent connectivity and store & forward. 

X