- Leaders: Yona Cao C.C. Fan
- Objective: This work stream focuses on optimizing the performance and scalability of cloud infrastructure across multiple geographical locations. It aims to address challenges related to latency, data sovereignty, and efficiency in distributed computing environments.
- Approach: The team will develop strategies for seamless data synchronization and application performance across dispersed networks, ensuring robust, secure, and compliant operations globally.
Introduction to vivgrid (private beta)
vivgrid.com is a geo-distributed public edge cloud for AI agents, Elevate AI applications with
Global AI Inference. Vivgrid helps developers deploy their AI apps closer to users for lightning-fast performance. With vivgrid, developers will enjoy higher efficiency at lower costs with global infrastructure solutions. Give users the seamless experience they deserve.
Features
- Globally located AI inference infrastructure
- Developers can deploy their AI applications with a single click, and Vivgrid will deploy the application across multiple nodes worldwide, automatically selecting the compute node closest to the user based on their location.
- AI API bridge
- For commonly used AI API services such as Open AI API, Vivgrid will provide locally accessible API entry points, ensuring the fastest network route to the API for users, thereby improving the response speed of end-user requests.
- Function calling
With the general function calling service, developers only need to configure function calling once on the Vivgrid platform to correctly invoke the function calling capabilities of different large language models
- Easily prompt management
- Enable non-developers to easily update system prompts directly from the dashboard, applying changes instantly without coding. Say goodbye to constant modifications for developers and streamline your workflow with our intuitive prompt management feature.
Timeline
Phase 1: Provide OpenAI API acceleration gateway, allowing developers to easily start using the globally accelerated OpenAI API on Vivgrid with simple configuration. Additionally, offer easy-to-use best practices for streaming requests and responses with the OpenAI API.
Phase 2: Provide serverless cloud that seamless integrated with function calling. Developers can deploy various serverless functions on the Vivgrid platform to enhance the capabilities of AI agents. For example, if real-time weather information is needed, a serverless function that calls a third-party weather API can be deployed on Vivgrid. With serverless function calling, AI agents can access APIs like weather API, access important private data, and integrate with existing systems.
