- Leader: Rick Cao Qi Tang
- Objective: The Edge Database work stream is dedicated to advancing database solutions tailored for edge computing environments. It targets improvements in data handling and storage capabilities on edge devices, enhancing local data processing and decision-making.
- Approach: Efforts will include the development of lightweight, scalable database systems that support real-time data processing and analytics, pivotal for Edge AI Virtual Agents.
Background
Our initial proposal covers some use cases (Slide #4) of AI virtual agents on the edge.
We might need to think about the
For an open-source edge project, we believe it could be benefit
Objectives
Two primary long-term goals of this work stream:
1) (Interoperability) Define clean, standardized and lightweight data and data processing interface for AI LLM services on the edge
2) (non-expert) Deliver non-expert, on-premise and low-cost AI virtual agent solution for local and small businesses (e.g. FAQs / customer LLM services in a local store)
Features
Data content / context
- Real-time information
- Location-specific information
- Event information
Data types
- Unstructured: PDF, HTML, Audio, Image, Video, etc.
- Structured: SQL, vector stores, knowledge graphs
- Files: Json, CSV
- APIs
Off-the-shelf vector database and embedding models
- Vector database
- LangChain
- LlamaIndex
- Sentence transformers
Data Access Authentication
- Public access
- Private data (Enterprise use cases)
- Protocols
Algorithms
- semantic search
- data chunking
- ranking
- recommendation ML models
- Lora Adaptor
- data routing
Evaluation and analytics
- Click through rate (from the reference links)
- User feedback
- Experiment
Timeline
Phase 1: (Prototyping) 1) adopt off-the-shelf solutions; 2) provide benchmark results; 3) develop the evaluation matrix for edge database/algorithm solution
Phase 2: (Standardization) Based on the lessons learnt, consider the scalability and interoperability
Version history
(08/20/2024) initial proposal, outlines of work scope, objective, and approaches