Work stream 2: Edge Database & Algorithms

Owned by Tina Tsou
Last updated: Aug 20, 2024 by Qi Tang
1 min read

  • Leader: @Rick Cao @Qi Tang

  • Objective: The Edge Database work stream is dedicated to advancing database solutions tailored for edge computing environments. It targets improvements in data handling and storage capabilities on edge devices, enhancing local data processing and decision-making.

  • Approach: Efforts will include the development of lightweight, scalable database systems that support real-time data processing and analytics, pivotal for Edge AI Virtual Agents.

Background

Our initial proposal covers some use cases (Slide #4) of AI virtual agents on the edge. 

We are open for more real-life use cases and edge business service ideas. 

Current direction is to implement AI agent LLM service (e.g. customer service on the edge).

Objectives

Two long-term goals of this work stream: 

1) (Interoperability) Define clean, standardized and lightweight data and data processing interface for AI LLM services on the edge

2) (non-expert) Deliver non-expert, on-premise and low-cost AI virtual agent solution for local and small businesses (e.g. FAQs / customer LLM services in a local store)

Features

Data content / context

  • Real-time information

  • Location-specific information

  • Event information

  • Function calling (_functions, python scripts)

Input Data types

  • Unstructured: PDF, HTML, Audio, Image, Video, etc.

  • Structured: SQL, vector stores, knowledge graphs

  • Files: Json, CSV

  • APIs

Off-the-shelf vector database and embedding models

  1. Vector database

  2. LangChain

  3. LlamaIndex

  4. Sentence transformers 

Data Access Authentication

  1. Public access

  2. Private data (Enterprise use cases) 

  3. Protocols

Algorithms

  1. semantic search

  2. data chunking

  3. ranking

  4. recommendation ML models

  5. Lora Adaptor 

  6. data routing

Evaluation and analytics

  1. Click through rate (from the reference links)

  2. User feedback

  3. Experiment

Timeline

Phase 1: (Prototyping) 1) adopt off-the-shelf solutions; 2) provide benchmark results; 3) develop the evaluation matrix for edge database/algorithm solution

Phase 2: (Standardization) Based on the lessons learnt, consider the scalability and interoperability

Version history

(08/20/2024) initial proposal, outlines of work scope, objective, and approaches