Type-driven Threading and Link Variables

I’ve been thinking recently about how to make Sirea RDP easy to use and more accessible to existing applications. And I’ve had several ideas that appeal to me:

  1. Use types both to model partitions and automatically create threads. My earlier implementation, RDPIO, provided a vat model, but setting up the vats proved to be a painful hassle (even in pseudocode). I encountered this idea of using types to guide creation of threads in the relatively new FRP.Sodium, and it contributed to me abandoning RDPIO and creating Sirea.
  2. Don’t take control of the initiating thread! It would be easy to run RDP behaviors as a permanent loop that simply waits for pending updates and processes them (until killed). And that was my initial idea for Sirea – part of keeping the implementation simple. But many applications already have a main loop, and trying to `own` the main loop is something that has annoyed me in other libraries (it doesn’t compose well). Instead, provide a step-procedure that will process pending updates, e.g. something that can be run every frame in a GLUT loop.
  3. Don’t take control of other threads, either. It would be convenient to do some heavy labor in those threads Sirea is creating for us, such as creating an OpenGL thread on demand to raise a window and present something to the user. Instead, I am thinking to use a typeclass for creation of threads, and to pass the step-procedures to these threads just like is returned to the main function.
  4. Make it easy to transfer data between Sirea and the task loops, for example to get some control data from Sirea or provide some query results. Sirea does provide a generic mechanism for extension (`bUnsafeLnk`) but it is inconvenient to use. I want something that is easy and more accessible.

For the fourth point, I’ve had the idea of Link Variables (LnkVars).

In Sirea, a link variable might look like a pair of behaviors:

bpushLnkVar :: (Typeable a, Typeable p) => B (S p a) (S p ())
bpullLnkVar :: (Typeable a, Typeable p) => B (S p ()) (S p a :|: S p ())

The LnkVar is uniquely identified by its type a and partition p. There are easy idioms using `newtype` and phantom types to create as many unique IDs as needed. I could also add a string, which would make partitions a bit more extensible, but that would also add a lot of complexity and risk of error.

Every LnkVar is written by at most one location in the whole Sirea behavior. This can be enforced easily at ‘compile time’ for the Sirea behavior. The LnkVar doesn’t always have a value; if it does not, this is reflected upon reading it.

If a LnkVar is not written by the Sirea behavior, it may be written by the partition’s thread. The capability would be provided alongside the step function. Something similar to:

data SireaProc = SireaProc {
  sp_on_pending :: IO () -> IO () -- set one-time callback for updates pending
  sp_step :: IO () -- process pending updates
  sp_pullLnkVar :: (Typeable a) => IO (DT -> IO (Maybe a))
  sp_pushLnkVar :: (Typeable a) => IO ([(DT,Maybe a)] -> IO ())

This allows the thread to access these Link Variables and thus interact with the Sirea behavior. The `DT` delta time value allows peeking ahead or slightly into the past, or set future values, and is measured relative to the last `sp_step`. The two-stage IO op is so we only need to perform lookups once (if we’re willing to store the intermediate result).

LnkVar is not intended for state manipulation. Developers still have access to IORef if they need it. LnkVar is instead intended to share time-varying data between Sirea and the surrounding application. Consequently, variables only update on sp_step; you won’t see your own pushes until you step again.

There is a potential conflict here: if we have both sp_pushLnkVar and bpushLnkVar in the system, it is unclear who should `win`. One potential solution is to say that bpushLnkVar always wins while active. Or I could provide a priority option to sp_pushLnkVar so that a variable can be written in up to three `layers` (high priority sp_pushLnkVar, bpushLnkVar, low priority sp_pushLnkVar) with the top layer winning. Any of these choices is compatible with RDP semantics; i.e. I can logically model bpullLnkVar as selecting between available resources in an environment.

One thing I don’t like about LnkVars is that they’re extending Sirea’s interface in some deep ways. They’d be difficult to disentangle from Sirea if I later decide they are a mistake. I could potentially achieve LnkVars without entangling them by using some global state, but that causes its own problems.

One thing I do like about LnkVars is that they’re potentially a great way to achieve implicit data plumbing that would be painful with arrows, and model shared environments. The unique writer requirement avoids semantic and performance overheads relative to using demand monitors or other multi-contributor models.

Speaking of demand monitors, I should add some extra interfaces so I can observe and manipulate `demand` for a LnkVar. If I know that nobody is listening, I might save some energy by shutting down irrelevant computations.

Anyhow… I need to deliberate on this idea further. But it’s a promising one. And I think the implementation would be relatively easy.

Aside: I’ve found a bdisjoin model I’m satisfied with (see addendum to linked article). But it would still be painful to use. I think I would very often prefer to use link variables – just create a fresh variable name (type), use bpushLnkVar at one location and bpullLnkVar at another, voila.

Addendum 1: After having deliberated on this some, I’ve found at least one hole in my reasoning: dynamic behaviors cannot be checked statically for the unique-writer property. I might need to more formally model acquisition of the variable, to ensure the write-capability is unique even in dynamic behaviors, but I’m not sure how to do that without making it less convenient. I’ll table the idea for now, I think, and get back to it if I have any aha moments. (Meanwhile, I can provide state and demand-monitor models by a similar type-based naming mechanism.)

Addendum 2: I think my complaint about tying Sirea too much to one interface can be solved. I can build a generic context object above Data.Typeable, and use it to build LnkVars or other partition-specific resources. Similar to thread-local storage, except I can allow some initial setup when compiling an RDP behavior.

data SireaProc = SireaProc {
  sp_on_pending :: IO () -> IO () -- set one-time callback for updates pending
  sp_step :: IO () -- process pending updates
  sp_data :: (Typeable a) => IO (IORef (Maybe a)) -- partition-local data

This entry was posted in Language Design, Modularity, Open Systems Programming, Reactive Demand Programming. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s