Work stream 4: Federated Learning (with EdgeLake)
- Tina Tsou
- Moshe Shadmon
Overview
Federated Learning at the Edge is with significant advantages:
Rather than centralizing the data or using digital twins, we plan to demo a full lifecycle of AI model creation and Inference on the source data at the edge.
We are using EdgeLake as a data platform at the Edge. Rather than moving the massive amounts of data, EdgeLake will service the data to Edge Nodes to create the model on the distributed source data and as if the edge is a single machine.
See details at https://lfedge.org/projects/edgelake/
The Process
The process with InfiniEdge will leverage Federated Learning:
Nodes at the edge will be deployed with EdgeLake + the code that generates the Sub-Model.
When each participating node generated a sub-model, the sub models are integrated to generate a unified model.
The Unified Model is pushed to all the participating nodes to generate inference data.
The EdgeLake rule Engine acts on the Inferenced Data.
This process is automated and the details are below:
Step | Platform | Function | Comments |
|---|---|---|---|
1 | EdgeLake | Capture data from machines and infrastructure that generate data | EdgeLake is deployed on many Edge Nodes and hosts data from PLCs and sensors connected to each node |
2 | EdgeLake | Service the data to a Model Generation process on each Edge Node | Each Edge Node is deployed with EdgeLake + the code that generates the Sub-Model. |
3 | AI Code | Model Generation on each node creates a Sub-Model | Each edge node is queried using SQL by the Model Generation Process |
4 | EdgeLake | Makes all the Sub-Models available on a single node | Only Sub-Models are moved (not the data) |
5 | AI Code | A unified model is created | by combining all the Sub-Models |
6 | EdgeLake | The Unified Model is pushed to all the Edge Nodes | |
7 | EdgeLake | Service the data to the Inference Process on each Edge Node | |
8 | EdgeLake | The inference data from all edge nodes satisfies queries | Using EdgeLake's Virtual Data Lake - without the need to centralize the data |
9 | EdgeLake | The EdgeLake rule engine on each node can act and alert on the inference data | Machine optimization, setup and maintenance is done based on the inferenced data |
10 | PLCs/Sensors | New data is generated | back to step 1 |
The demo data set
We will use a real Smart City Data, EdgeLake is deployed in multiple nodes a city in Kansas to manage the city's infrastructure including electricity, water and wastewater.
AnyLog (https://anylog.co/) will provide the schema and data sets to train the sub models and execute the inference process on each edge node.
Example Dataset Schema of a city generator (Note: 4 first columns are for internal use and not generated by the infrastructure. The ID represents the generator ID):
row_id : integer
insert_timestamp : timestamp without time zone
tsd_name : char(3)
tsd_id : int
id : int
timestamp : timestamp without time zone
a_current : int
a_n_voltage : int
b_current : int
b_n_voltage : int
c_current : int
c_n_voltage : int
energymultiplier : int
frequency : int
powerfactor : int
reactivepower : int
realpower : intExample 10 Rows of Data:
{"Query":[{"row_id":1,"insert_timestamp":"2024-07-02 01:04:18.050656","tsd_name":"75","tsd_id":6572,"id":1,"timestamp":"2024-06-28 05:09:28.0535560","a_current":30,"a_n_voltage":735,"b_current":32,"b_n_voltage":736,"c_current":30,"c_n_voltage":734,"energymultiplier":1,"frequency":6000,"powerfactor":97,"reactivepower":150,"realpower":660},{"row_id":2,"insert_timestamp":"2024-07-02 01:04:18.050656","tsd_name":"75","tsd_id":6572,"id":2,"timestamp":"2024-06-28 05:09:28.0535560","a_current":24,"a_n_voltage":736,"b_current":24,"b_n_voltage":736,"c_current":27,"c_n_voltage":733,"energymultiplier":1,"frequency":6000,"powerfactor":100,"reactivepower":14,"realpower":550},{"row_id":1,"insert_timestamp":"2024-07-02 01:04:18.050656","tsd_name":"75","tsd_id":6572,"id":1,"timestamp":"2024-06-29 00:00:31.4335830","a_current":41,"a_n_voltage":736,"b_current":52,"b_n_voltage":739,"c_current":44,"c_n_voltage":736,"energymultiplier":1,"frequency":6000,"powerfactor":97,"reactivepower":240,"realpower":976},{"row_id":2,"insert_timestamp":"2024-07-02 01:04:18.050656","tsd_name":"75","tsd_id":6572,"id":2,"timestamp":"2024-06-29 00:00:31.4335830","a_current":38,"a_n_voltage":737,"b_current":32,"b_n_voltage":739,"c_current":37,"c_n_voltage":737,"energymultiplier":1,"frequency":6000,"powerfactor":99,"reactivepower":81,"realpower":782},{"row_id":3,"insert_timestamp":"2024-07-02 02:11:40.767219","tsd_name":"75","tsd_id":5,"id":1,"timestamp":"2024-06-29 00:00:31.4335830","a_current":41,"a_n_voltage":736,"b_current":52,"b_n_voltage":739,"c_current":44,"c_n_voltage":736,"energymultiplier":1,"frequency":6000,"powerfactor":97,"reactivepower":240,"realpower":976},{"row_id":4,"insert_timestamp":"2024-07-02 02:11:40.767219","tsd_name":"75","tsd_id":5,"id":2,"timestamp":"2024-06-29 00:00:31.4335830","a_current":38,"a_n_voltage":737,"b_current":32,"b_n_voltage":739,"c_current":37,"c_n_voltage":737,"energymultiplier":1,"frequency":6000,"powerfactor":99,"reactivepower":81,"realpower":782},{"row_id":5,"insert_timestamp":"2024-07-02 02:11:40.767219","tsd_name":"75","tsd_id":5,"id":3,"timestamp":"2024-06-29 00:00:31.4335830","a_current":64,"a_n_voltage":737,"b_current":46,"b_n_voltage":739,"c_current":54,"c_n_voltage":736,"energymultiplier":1,"frequency":6000,"powerfactor":97,"reactivepower":270,"realpower":1183},{"row_id":6,"insert_timestamp":"2024-07-02 02:11:40.767219","tsd_name":"75","tsd_id":5,"id":4,"timestamp":"2024-06-29 00:00:31.4335830","a_current":25,"a_n_voltage":737,"b_current":26,"b_n_voltage":739,"c_current":25,"c_n_voltage":737,"energymultiplier":1,"frequency":6000,"powerfactor":100,"reactivepower":50,"realpower":556},{"row_id":7,"insert_timestamp":"2024-07-02 02:11:40.767219","tsd_name":"75","tsd_id":5,"id":1,"timestamp":"2024-06-29 00:01:02.9634121","a_current":42,"a_n_voltage":736,"b_current":52,"b_n_voltage":739,"c_current":44,"c_n_voltage":736,"energymultiplier":1,"frequency":6001,"powerfactor":97,"reactivepower":242,"realpower":984},{"row_id":3,"insert_timestamp":"2024-07-02 01:04:18.050656","tsd_name":"75","tsd_id":6572,"id":3,"timestamp":"2024-06-28 05:09:28.0535560","a_current":35,"a_n_voltage":735,"b_current":21,"b_n_voltage":735,"c_current":28,"c_n_voltage":732,"energymultiplier":1,"frequency":6000,"powerfactor":99,"reactivepower":94,"realpower":608}],
"Statistics":[{"Count": 10,"Time":"00:00:00","Nodes": 1}]}