Agenda

1. LF AI & Data mini Summit recap, 15 mins, @Moshe Shadmon @Tina Tsou

2. SPEAR Release 2.0 Realtime ASR Demo. 15 mins, @Wilson Wang

Recording

https://bytedance.us.larkoffice.com/minutes/obusff654t4ut576imrbp34r?from=from_copylink

Participants

@Wilson Wang @Chun Liu @Yike Gong @Mengyu Hang @Ning Jiang @Jianwen Pi @Jiayi Song @Lance Lim @Qingkun Li @Mo Li @Tina Tsou @Wenhui Zhang @Hao Xie @Xun Chen @Ye Xu @Ying Wang @Yizheng Jiao @Zixiong Zhang @Akram Sheriff @Alex Liu @anfernee.guan @Ashwanth Gnanavelu @Benjamin Barber @Yona @Yongli Chen @Hashem Jaber @Jenny Yang @李博睿 @刘秉伟 @LU CHENG @MichelleMa @Ori Shadmon @Rachel Roumeliotis @Ruoyu Ying @Sam Han @Shunli @Suhaas Teja Vijjagiri @Sumeet Solanki @TangQi @Tina Tsou @TinaTsou @Tom Qin @Victor Lu @万里 @Wilson Wang @Yin Li @Yu Wang - AIG @周琥恒

Summary

The meeting focused on the recap of FA and data activities, a demo of Spear Release 2.0's real - time ASR feature, introductions of new members, discussions on upcoming work streams, and an overview of an upcoming event. Additionally, there were in - depth discussions about AI ops and related projects. Content follows:

• FA and Data Recap

  • Agenda and Presentations

    • The agenda included a recap of the FA and data activities, with Marsha and Tina involved. Ori Shadmon mentioned the potential and synergy from the previous week's AI and data summit.

    • Tina presented the OPR and Infinite AI integration. She shared slides about what OPR is, an enterprise AI framework, and how it integrates with Edge AI and Infinite AI. The presentation was made with the TCC chair from Infinite AI and the OPR TSC member.

    • There was also a panel on "From Zero to AI" in the foundations AI and data community.

  • OPR and Integration Details

    • The OPR is a framework that addresses enterprise AI challenges. The integration focused on the AI agent layer and the dev and user layers.

    • Tina showed a demo of how different GPUs (AMD and Intel) can work together using OPR. Useful links about OPR architecture were also shared.

    • The data serialization and LM integration (Deepseek, later referred to as just LM) were discussed.

• Spear Release 2.0 Real - Time ASR Demo

  • Demo Overview

    • Wilson demonstrated the real - time ASR capability in Spear 2.0. As long as the backend supports OpenAI's real - time API, it can integrate and provide real - time ASR to the agent.

    • In the demo, an agent received user input, converted it to OpenAI real - time messages, and then made API calls. However, there was an issue where the output was in Chinese instead of English as expected.

  • Language and Model Issues

    • The default configuration was set for English, but the client code had a language part specified as Chinese. Wilson was unsure why this parameter affected the output.

    • The backend used the whisper model, and the next step is to send the transcription to the Xiao Yi model for further actions.

• New Member Introductions

  • Simon's Introduction: Tom introduced Hu Heng, also known as Simon, a new TSC member from Genesis. Genesis is a company that does open - source IoT gateways and works on edge and ESIS.

  • Jay's Introduction: Jay Paul Rush introduced himself as part of a CTR organization based in Madison, Wisconsin, US. He works with Andreas.

• Upcoming Work Streams

  • Work Stream 3: Bore and Wilson will present work stream 3 for Spear next week to go through the requirements.

  • Work Stream 4: Ori will present work stream 4 for the release 2.0 on July 8th. He will consult with his GMV first.

  • Work Stream 5: Tom and Simon will handle work stream 5, which is a minor version prompt and release of the edge gateway, next week.

  • Work Stream 6: Jeff Broward will present work stream 6 on July 8th. He is currently defining a release 9 of a Crano and exploring new use cases.

• Upcoming Event

  • Event Details

    • The Link Foundation, FA and data, and FH co - hosted an event for agent software engineering at the TikTok Singapore campus.

    • Tom's proposal was accepted, and he will be one of the speakers. In total, 15 proposals were accepted.

    • The event host includes the foundation and Data, TikTok, Epic Connectors, Tina, and others.

  • Registration: The registration for one link is closed, but there is still an option to register. So far, 69 people are going through the first link, and 22 through the second. The event can host around 90 - 100 people, and about 200 have already registered.

• AI Ops and Related Projects

  • AI Ops Concept

    • There is an interest in creating an AI ops group, aiming for a different approach in the distributed environment compared to existing mlops projects.

    • Jay and Andreas have been working on this topic and need to refine it and integrate it with other aspects.

  • Benjamin's Projects

    • Benjamin presented several projects, including a virtual file system with multiple cache tiers (the hottest being Apache Arrow in memory), a model server that can serve hugging face models on six hardware targets, and a 3D Avatar project.

    • He is trying to port these projects from Python to typescript for web delivery. He also mentioned a dispute with Anthropic and plans to share some projects next week.

  • Naming and Scope Discussion

    • There was a discussion about naming the project, with suggestions like "Swiss knife multi - agent environment" and "federated computing."

    • The scope of AI ops may include MSS, Emma Ops, and related startup company projects. There is a need to define AI ops and understand how it combines with other concepts like EMEA ops.

• Future Plans

  • Work Stream Presentations: Bore and Wilson will present work stream 3 for Spear next week. Ori will present work stream 4 on July 8th. Tom and Simon will handle work stream 5 next week, and Jeff will present work stream 6 on July 8th.

  • AI Ops Group: Jay and Andreas will refine the AI ops concept and integrate it with other aspects. They will also work on finding the right scope and naming for the project.

  • Event Participation: The event at the TikTok Singapore campus will be held, and those interested can register. Jason will send emails to inform proponents whose proposals were accepted.

  • Project Development: Benjamin will continue to work on his projects, including porting them to typescript and resolving the dispute with Anthropic. He also plans to enter the White House competition for AI education.

This content is only supported in a Feishu Docs

Chapters

00:10 Meeting Agenda Review and AI Integration Discussion

This section is from a meeting where Tina Tsou welcomes Benjamin Barber back from Europe. She shares today's agenda, including an FA and data mini semi - recap and an SPU release 2.0 real - time demo. Ori Shadmon gives a recap of the previous week's AI and data summit. A new TSC member named Simon from the Genesis agency is introduced. There are discussions about the integration of OPR and Infinite AI, and the success of a panel about joining the AI and data community. Tina Tsou is about to share related slides.

05:22 OPR Presentation, Demos and Integration with Community Outlook

This section features Tina Tsou discussing the presentation she and Ezeko gave, covering topics like OPR, a framework addressing enterprise AI challenges and integrating Edge AI, Infinite AI, and Opus. She mentions a scalable AI agents platform, AI orchestration stacks, and focuses on specific layers. They demonstrated OPR's compatibility with different GPUs. Tina also presents a neural divergence companion demo and shows how SPU and OPR integrate. Wilson will talk more about community contributions, and Tina thanks all contributors.

11:05 Spear 2.0 Real-time ASR Demo and Release Plan Discussion

This section covers multiple topics from a meeting. Tina Tsou and Ori first showed a demo. Wilson Wang then presented the real-time ASR feature in spear 2.0 but faced an issue where English input was converted to Chinese. There were discussions about the language model used. The meeting also planned future presentations: Wilson and Borray for work stream 3 next week, others for work streams 4, 5, and 6 on July 8th. Jay introduced himself, and Michelle was asked to present and self - introduce.

25:16 3D Avatar RL Project, Mojo Language, and AGPL Release Plan

This section features Benjamin Barber discussing his work on a Javascript-based web project with a 3D Avatar hooked to reinforcement learning. He collaborated with Chris Latner of Modular, whose system is faster than PyTorch. Benjamin plans to train a new skeleton and facial model using video and image outputs. He's moving repositories to a new organization, Genesis. Tina Tsou offers feedback and mentions relevant frameworks. They also discuss the Mojo language, its SDK, and Benjamin's plan to release his work as an AGPL project and enter a White House AI education competition, while Tina shares upcoming events.

37:13 Event Introduction, AI Ops Discussion and Project Updates

This section of the meeting discusses an event co - hosted by the Link Foundation, FA, data, and FH for software engineering. 15 proposals are accepted, and the event will be held at TikTok Singapore campus. There's also talk about AI ops, with suggestions to create a problem - statement document. Benjamin shares various projects like a virtual file system and a model server. Ideas for naming the project include “Swiss knife multi - agent environment” and “federated computing”. Jason will inform proponents of proposal acceptance, and the meeting concludes.