Dynamic Anytime Algorithms

I’m still thinking about ways to eliminate stateful semantics from large-scale reactive applications for RDP, i.e. pursuing stability instead. Stateless semantics make an application more robust against resets, disruption or partitioning, etc.. To achieve stability without state requires accepting some indeterminism in the semantics (though not the implementation), but this is often a worthy tradeoff. Stable indeterminism is at least predictable.

A lot of recent algorithms work in machine learning seems to be a treasure trove for this purpose – in particular the class of dynamic anytime algorithms, which must deal with time-varying data in a stable and incremental manner.

I spoke in the prior article of creating structure based on constraint-logic rules, but another useful approach is to find ad-hoc structure in data. We can get `stability` from indexes and clustering, i.e. reporting the current sort for time-varying data, especially if there is some overlap of groups such that data doesn’t move to another group until necessary. I think the work on “dynamic decentralized anytime hierarchical clustering“, based on ant swarm clustering might be a very useful tool for organizing large-scale continuous reactive databases and sorting mechanisms.

This could be used for a range of purposes – object detection, large scale open distributed scene graphs, content-centric data streams, etc.

RDP is internally stateless, but is well designed to orchestrate external models. It is easy to hook new state models into RDP without worrying about how they would be implemented in pure RDP, so long as they meet certain constraints. Providing services for even a few specialized dynamic anytime algorithms could offer some very powerful tools for a broader range of use cases than might initially be envisioned. And clustering would be mostly orthogonal to other tools such as constraint-logic modeling.

This entry was posted in Language Design and tagged , , , , . Bookmark the permalink.

One Response to Dynamic Anytime Algorithms

  1. dmbarbour says:

    I wonder if anytime hierarchical clustering results in higher quality clusters. It seems feasible, since it will implicitly utilize information about how data changes over time, rather than just information about where data is sitting when clustering starts. Bits of a feather will flock together.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s