Work stream 4: Federated Learning (with EdgeLake)

Overview

Federated Learning at the Edge is with significant advantages:

Rather than centralizing the data or using digital twins, we plan to demo a full lifecycle of AI model creation and Inference on the source data at the edge.

We are using EdgeLake as a data platform at the Edge. Rather than moving the massive amounts of data, EdgeLake will service the data to Edge Nodes to create the model on the distributed source data and as if the edge is a single machine.

See details at https://lfedge.org/projects/edgelake/

The Process

The process with InfiniEdge will leverage Federated Learning:

Nodes at the edge will be deployed with EdgeLake + the code that generates the Sub-Model.
When each participating node generated a sub-model, the sub models are integrated to generate a unified model.
The Unified Model is pushed to all the participating nodes to generate inference data.
The EdgeLake rule Engine acts on the Inferenced Data.

This process is automated and the details are below:

Step	Platform	Function	Comments

Step	Platform	Function	Comments
1	EdgeLake	Capture data from machines and infrastructure that generate data	EdgeLake is deployed on many Edge Nodes and hosts data from PLCs and sensors connected to each node
2	EdgeLake	Service the data to a Model Generation process on each Edge Node	Each Edge Node is deployed with EdgeLake + the code that generates the Sub-Model.
3	AI Code	Model Generation on each node creates a Sub-Model	Each edge node is queried using SQL by the Model Generation Process
4	EdgeLake	Makes all the Sub-Models available on a single node	Only Sub-Models are moved (not the data)
5	AI Code	A unified model is created	by combining all the Sub-Models
6	EdgeLake	The Unified Model is pushed to all the Edge Nodes
7	EdgeLake	Service the data to the Inference Process on each Edge Node
8	EdgeLake	The inference data from all edge nodes satisfies queries	Using EdgeLake's Virtual Data Lake - without the need to centralize the data
9	EdgeLake	The EdgeLake rule engine on each node can act and alert on the inference data	Machine optimization, setup and maintenance is done based on the inferenced data
10	PLCs/Sensors	New data is generated	back to step 1

The demo data set

We will use a real Smart City Data, EdgeLake is deployed in multiple nodes a city in Kansas to manage the city's infrastructure including electricity, water and wastewater.

AnyLog (https://anylog.co/) will provide the schema and data sets to train the sub models and execute the inference process on each edge node.

Example Dataset Schema of a city generator (Note: 4 first columns are for internal use and not generated by the infrastructure. The ID represents the generator ID):

 row_id   :   integer
 insert_timestamp   :   timestamp without time zone
 tsd_name   :   char(3)
 tsd_id   :   int
 id   :   int
 timestamp   :   timestamp without time zone
 a_current   :   int
 a_n_voltage   :   int
 b_current   :   int
 b_n_voltage   :   int
 c_current   :   int
 c_n_voltage   :   int
 energymultiplier   :   int
 frequency   :   int
 powerfactor   :   int
 reactivepower   :   int
 realpower   :   int

Example 10 Rows of Data:

{"Query":[{"row_id":1,"insert_timestamp":"2024-07-02 01:04:18.050656","tsd_name":"75","tsd_id":6572,"id":1,"timestamp":"2024-06-28 05:09:28.0535560","a_current":30,"a_n_voltage":735,"b_current":32,"b_n_voltage":736,"c_current":30,"c_n_voltage":734,"energymultiplier":1,"frequency":6000,"powerfactor":97,"reactivepower":150,"realpower":660},{"row_id":2,"insert_timestamp":"2024-07-02 01:04:18.050656","tsd_name":"75","tsd_id":6572,"id":2,"timestamp":"2024-06-28 05:09:28.0535560","a_current":24,"a_n_voltage":736,"b_current":24,"b_n_voltage":736,"c_current":27,"c_n_voltage":733,"energymultiplier":1,"frequency":6000,"powerfactor":100,"reactivepower":14,"realpower":550},{"row_id":1,"insert_timestamp":"2024-07-02 01:04:18.050656","tsd_name":"75","tsd_id":6572,"id":1,"timestamp":"2024-06-29 00:00:31.4335830","a_current":41,"a_n_voltage":736,"b_current":52,"b_n_voltage":739,"c_current":44,"c_n_voltage":736,"energymultiplier":1,"frequency":6000,"powerfactor":97,"reactivepower":240,"realpower":976},{"row_id":2,"insert_timestamp":"2024-07-02 01:04:18.050656","tsd_name":"75","tsd_id":6572,"id":2,"timestamp":"2024-06-29 00:00:31.4335830","a_current":38,"a_n_voltage":737,"b_current":32,"b_n_voltage":739,"c_current":37,"c_n_voltage":737,"energymultiplier":1,"frequency":6000,"powerfactor":99,"reactivepower":81,"realpower":782},{"row_id":3,"insert_timestamp":"2024-07-02 02:11:40.767219","tsd_name":"75","tsd_id":5,"id":1,"timestamp":"2024-06-29 00:00:31.4335830","a_current":41,"a_n_voltage":736,"b_current":52,"b_n_voltage":739,"c_current":44,"c_n_voltage":736,"energymultiplier":1,"frequency":6000,"powerfactor":97,"reactivepower":240,"realpower":976},{"row_id":4,"insert_timestamp":"2024-07-02 02:11:40.767219","tsd_name":"75","tsd_id":5,"id":2,"timestamp":"2024-06-29 00:00:31.4335830","a_current":38,"a_n_voltage":737,"b_current":32,"b_n_voltage":739,"c_current":37,"c_n_voltage":737,"energymultiplier":1,"frequency":6000,"powerfactor":99,"reactivepower":81,"realpower":782},{"row_id":5,"insert_timestamp":"2024-07-02 02:11:40.767219","tsd_name":"75","tsd_id":5,"id":3,"timestamp":"2024-06-29 00:00:31.4335830","a_current":64,"a_n_voltage":737,"b_current":46,"b_n_voltage":739,"c_current":54,"c_n_voltage":736,"energymultiplier":1,"frequency":6000,"powerfactor":97,"reactivepower":270,"realpower":1183},{"row_id":6,"insert_timestamp":"2024-07-02 02:11:40.767219","tsd_name":"75","tsd_id":5,"id":4,"timestamp":"2024-06-29 00:00:31.4335830","a_current":25,"a_n_voltage":737,"b_current":26,"b_n_voltage":739,"c_current":25,"c_n_voltage":737,"energymultiplier":1,"frequency":6000,"powerfactor":100,"reactivepower":50,"realpower":556},{"row_id":7,"insert_timestamp":"2024-07-02 02:11:40.767219","tsd_name":"75","tsd_id":5,"id":1,"timestamp":"2024-06-29 00:01:02.9634121","a_current":42,"a_n_voltage":736,"b_current":52,"b_n_voltage":739,"c_current":44,"c_n_voltage":736,"energymultiplier":1,"frequency":6001,"powerfactor":97,"reactivepower":242,"realpower":984},{"row_id":3,"insert_timestamp":"2024-07-02 01:04:18.050656","tsd_name":"75","tsd_id":6572,"id":3,"timestamp":"2024-06-28 05:09:28.0535560","a_current":35,"a_n_voltage":735,"b_current":21,"b_n_voltage":735,"c_current":28,"c_n_voltage":732,"energymultiplier":1,"frequency":6000,"powerfactor":99,"reactivepower":94,"realpower":608}],
"Statistics":[{"Count": 10,"Time":"00:00:00","Nodes": 1}]}