Work stream 2: Edge Database & Algorithms
Leader: @Rick Cao @Qi Tang
Objective: The Edge Database work stream is dedicated to advancing database solutions tailored for edge computing environments. It targets improvements in data handling and storage capabilities on edge devices, enhancing local data processing and decision-making.
Approach: Efforts will include the development of lightweight, scalable database systems that support real-time data processing and analytics, pivotal for Edge AI Virtual Agents.
Background
Our initial proposal covers some use cases (Slide #4) of AI virtual agents on the edge.
We are open for more real-life use cases and edge business service ideas.
Current direction is to implement AI agent LLM service (e.g. customer service on the edge).
Objectives
Two long-term goals of this work stream:
1) (Interoperability) Define clean, standardized and lightweight data and data processing interface for AI LLM services on the edge
2) (non-expert) Deliver non-expert, on-premise and low-cost AI virtual agent solution for local and small businesses (e.g. FAQs / customer LLM services in a local store)
Features
Data content / context
Real-time information
Location-specific information
Event information
Function calling (_functions, python scripts)
Input Data types
Unstructured: PDF, HTML, Audio, Image, Video, etc.
Structured: SQL, vector stores, knowledge graphs
Files: Json, CSV
APIs
Off-the-shelf vector database and embedding models
Vector database
LangChain
LlamaIndex
Sentence transformers
Data Access Authentication
Public access
Private data (Enterprise use cases)
Protocols
Algorithms
semantic search
data chunking
ranking
recommendation ML models
Lora Adaptor
data routing
Evaluation and analytics
Click through rate (from the reference links)
User feedback
Experiment
Timeline
Phase 1: (Prototyping) 1) adopt off-the-shelf solutions; 2) provide benchmark results; 3) develop the evaluation matrix for edge database/algorithm solution
Phase 2: (Standardization) Based on the lessons learnt, consider the scalability and interoperability
Version history
(08/20/2024) initial proposal, outlines of work scope, objective, and approaches