PubSub redesign: introducing nkv 🎨

Owned by Pavel Abramov
Oct 02, 2024
2 min read

Currently, the design document is in Google Docs, since it’s being discussed, once it’s settled, content from that document will migrate here.

Link to the document can be found here, feel free to comment on it! TL;DR of that document is following:

 

  • Design a simpler replacement for PubSub in terms of API

  • Replace golden source of schema from go structs to openapi3 spec (or any other like json-rpc2.0, etc.)

  • Decide on a PubSub replacement candidate

 

Why: Tackling those problems should simplify development and testing, plus it would make it possible to work with multiple programming languages in EVE. In theory, you'll be able to trace your topics in real-time, get the topology in real time and separate sensitive data shared between services. Which would make integration of eve-ui (and probably, installer) way easier.Also, I prepared a simpler replacement for PubSub called NotifyKeyValue or nkv here is the repo. In a nutshell, instead of thinking about publication and subscriptions I think we need a key-value storage with notification mechanism. So basically it boils down to 5 commands:

  • I want to get latest value for the given key

  • I want to put (store or update if exists) a value for a given key

  • I want to delete a value for a given key

  • I want to subscribe to any changes of the value for the given key or it's children

  • I want to unsubscribe from any changes of the value for the given key or it's children

 

Note that keys support keyspaces, so I can have value hierarchy, i.e. domainmgr.appStatus.UUID-{1-999...} and I can get AppStatus for all UUIDs or just for the specific oneI'll schedule a meeting to talk about it around end of the next week, would appreciate if you can check the doc and repo out and give your opinion on this. Repo should contain docs which should be enough to understand what it is.