2024-12-03

Agenda

1. Robotics Data, @Yonghua Lin , 20 - 25 minutes

2. Release 1 readiness: 30 minutes

   Work stream 1: @C.C. Fan @Yona Cao

   Work stream 2: @Qi Tang

   Work stream 3: @Borui Li (李博睿) @Wilson Wang

   Work stream 4: @Moshe Shadmon

   Work stream 5: @Tom Qin @Keguang He

   Work stream 6: @Haruhisa Fukano

Recording

https://bytedance.us.larkoffice.com/minutes/obusbgx675u712jzr9q7zt98

Summary

The meeting discussed various aspects of AI, including project updates, data issues, open source, and integration. The main contents include:

• Introduction and Preparation for a Meeting：Discussed the introduction of participants, presentation arrangements, and meeting preparations related to the topic of AI.

• Infinite Edge AI Project Updates and Discussions：Discussed the progress and plans of various work streams in the Infinite Edge AI project, including the presentation of operational data for robotics and the readiness of each work stream for Release 1.

• Sharing and Discussing Meeting Materials：Discussed the sharing and viewing of meeting files and presentation materials, as well as the progress and plans related to the open data for the robot project.

• The Importance of Datasets in AI and Challenges in Building Them：Discussed the significance of datasets in the development of large models in AI and the challenges faced in building high-quality datasets for embodied AI.

• Open Data Initiative for Robot：Discussed the current situation and future plans of building datasets for embodied AI in robotics, and proposed the Open Data Initiative for collecting more data.

• Data Sharing in Initiative Activity for Embodied AI：Discussed the possibilities and methods of data sharing in an initiative activity related to embodied AI, including dataset, usage policy, platform and tools, and model.

• Initiative for Open Source Data and Model in Robotics：Discussed the initiative of using open source data to train models in the field of robotics, and the plans and requirements for data collection and contribution.

• Open Source Project Initiative and Schedule：Discussed the details of an open source project initiative, including its concept, example project, and rough schedule.

• Robotics and Open Source in AI

  • Open Source Technology：All the technology built will be open source, and close to 100 datasets have been opened, with a large number of downloads globally.

  • Model Size Range：The current model size is between 2 to 20 billion parameters, and efforts are on to reduce the size while maintaining the model's capability.

  • Autonomy Design：The design of robot autonomy is still underway, with some scenarios relying on the model within the robot and others using larger models on the cloud.

• IoT Gateway and AI Model Integration

  • IoT Gateway Function：The gateway is small enough to work on devices without relying on the cloud and is mainly for inference on the device side.

  • Business Model Consideration：Next AI requires payment for creating demos based on specific examples provided.

  • Collaboration Possibility：Yong Li can pitch the idea to customers for potential collaboration as they have high-quality data sources.

  • Work Stream Integration：The work related to this can be integrated into Workstream 5 and 6.

• Managing the Development and Presentation of Preview 1

  • Document Preparation：Wilson and Borui Li are working on the document for preview 1 and will prepare it for the presentation on 17th.

  • Sub Page Creation：Tina Tsou suggested creating a sub page on work stream 3 for release 1 deliverables and mapping items one by one.

  • Using the Table：Everyone is advised to use the same table provided by Kenneth for work stream 1, 2, and 3.

• Work Stream Updates and Deployment Issues

  • CD Lark Deployment：The deployment of CD Lark is automatic and can be pushed to Nexus 3 server. Need to set up the environment or submit an IT ticket. Can ask for help in the Lark group.

  • Work Stream Decisions：Work stream 3 needs to request review time. Work stream 5 has clear table filling and some documents written. Work stream 6 knows what to do and needs to talk to Fukai San.

  • Environment Choices：Need to decide whether to use the community or own company's CD environment for deployment.

• Release 1 Meeting Discussions

  • Meeting Date Decision : December 17th is suggested for the Release 1 review meeting.

  • Work Stream Follow-up : Wen Xin to follow up with work streams 1, 2, and 4 for their participation in Release 1.

  • Edge Applications Consideration : Consider delivering AI agent and marketplace along with work streams.

  • Wiki Page Update : Wen Xin to update the wiki page including the AI Agent Marketplace.

  • App Confirmation : Confirm with Cici Yuna about the edge apps.

  • First Agent Family Preparation : Wilson and Anthony to get the First Agent family for next week's pre-discussion.

Chapters

00:38 Infinite Edge AI Meeting: Introductions and Agenda Discussion

This section is about a meeting. People greet each other at the start. There are discussions regarding sending out the meeting agenda. Then, there is a round - of - table introduction where participants from different organizations introduce themselves. After that, they start to discuss the meeting agenda which includes a presentation on op data for robotics and the readiness of work streams for Release 1. There are also some technical issues with sharing files for the presentation.

11:32 The Need for High - Quality Datasets in Robot - Related Fields and Associated Challenges

This section begins with some technical adjustments. Then Lin Yonghua starts to introduce an initiative for building open data for the robot in collaboration with the global team. She also mentions three important datasets in the past. Next, she points out that there are 2.4 million datasets on Hacking Face but the embody AI domain lacks high - quality datasets and elaborates on related challenges.

17:21 Open Data Initiative for Robot: Building Datasets and an Ecosystem for Robotic AI Training

This section mainly discusses the lack of data standard in robotic data collection which hinders data reuse. It then presents the idea of an Open Data Initiative for the robot, including what kind of data can be shared (real world machine data, human action data, and simulate data), how data can be shared (open source, within the alliance), and related tools and models. Also, it mentions a sample project, a rough timeline, and the background of the Bai institute which has experience in dataset building.

35:44 Discussion on Robot Small Models, Autonomy and Potential Collaborations

This section begins with a question about small models in robotics, specifically the smallest models being worked on. Lin Yonghua mentions the current model size range in terms of parameters and her confidence in reducing it tenfold next year while maintaining capabilities. There's also discussion about on - device AI, Nexa AI, and potential business collaboration, followed by a talk about work streams and tasks allocation for Release 1.

42:44 Discussion on Release 1 Deliverables and Review Dates in Work Streams

This section mainly focuses on work stream 3 and the tasks related to Release 1 requirements. Tina Tsou suggests creating a sub - page and a mapping table for deliverables. There are also discussions about CD Lark and setting up the environment. Regarding work streams 5 and 6, there are talks about filling in a table and presenting on either December 10th or 17th. Tina Tsou also asks Wen Xin to follow up with other work streams.

52:35 Meeting discussions on bot development, AI Agent Marketplace, and future plans

This section is about the meeting arrangements. Noe Otero will make changes before the next meeting to be more impactful for developers. Tina Tsou asks Wen Xin to update the wiki page and they need to confirm with Cici Yuna about edge apps. For next week, there will be a pre - discussion. Tina Tsou also asks Wilson and Anthony to set the foundation for the First Agent family in gaming.