I’m still thinking about ways to eliminate stateful semantics from large-scale reactive applications for RDP, i.e. pursuing stability instead. Stateless semantics make an application more robust against resets, disruption or partitioning, etc.. To achieve stability without state requires accepting some indeterminism in the semantics (though not the implementation), but this is often a worthy tradeoff. Stable indeterminism is at least predictable.
A lot of recent algorithms work in machine learning seems to be a treasure trove for this purpose – in particular the class of dynamic anytime algorithms, which must deal with time-varying data in a stable and incremental manner.
I spoke in the prior article of creating structure based on constraint-logic rules, but another useful approach is to find ad-hoc structure in data. We can get `stability` from indexes and clustering, i.e. reporting the current sort for time-varying data, especially if there is some overlap of groups such that data doesn’t move to another group until necessary. I think the work on “dynamic decentralized anytime hierarchical clustering“, based on ant swarm clustering might be a very useful tool for organizing large-scale continuous reactive databases and sorting mechanisms.
This could be used for a range of purposes – object detection, large scale open distributed scene graphs, content-centric data streams, etc.
RDP is internally stateless, but is well designed to orchestrate external models. It is easy to hook new state models into RDP without worrying about how they would be implemented in pure RDP, so long as they meet certain constraints. Providing services for even a few specialized dynamic anytime algorithms could offer some very powerful tools for a broader range of use cases than might initially be envisioned. And clustering would be mostly orthogonal to other tools such as constraint-logic modeling.