[IdPv3] State Management and Clustering

[Russell Beall]

Thanks Chad for the detailed write up.  I like the direction this is going and I think the solution to provide a means to show a custom error page for a cluster node failure is a perfect solution, much better than trying to replicate a large quantity of session state synchronously.

Do I read this correctly in that the Infinispan optional solution will then only replicate a small packet of data sufficient to enable back-channel support?  The smaller that packet is, the more efficient the clustering will be, and this will be good for clusters over wide networks (for instance the LIGO project).  Will this now be feasible for wide networks?

I had planned to suggest that only a very small packet be synchronously clustered, and then larger amounts of data, such as the attribute consent data, be clustered asynchronously.  Is that part of the idea?  It looks like this isn't necessary if the packet is already small (only a few KB).

I had heard that the attribute consent engine would be packaged in with the new clustering solution.  Is that still happening, or will it still require database support?

Thanks,
Russ.

On Oct 7, 2011, at 6:06 AM, Chad La Joie wrote:

> At our meeting this week, the developers got together and discussed
> our approach to state management and "cluster" support within IdPv3.
> Here's were we ended up.
> 
> We identified two groupings of information within the system: that
> information which lives for the duration of a conversation (i.e.,
> state gathered up to answer a particular protocol request but thrown
> away afterwards) and information which persists across conversations
> (e.g., sessions, artifact mappings) and potentially restarts (e.g.,
> attribute consent).
> 
> Taking the second case first, we observe a couple of things.  Much of
> the information that falls in to this category is state information
> necessary for the operation of back-channel protocols.  The use of
> such protocols has diminished significantly as time has gone on,
> mostly due to the additional complexity they require.  The remainder
> of the information is quite small (e.g., large sessions are only a
> couple KB in size).
> 
> Given these observations, it is our intent to have the *default*
> configuration of IdPv3 use cookies (or possibly HTML 5 local storage
> when available) for that information which is not related to
> back-channel operations.  This includes: session data, attribute
> consent, and terms-of-use acceptance. Use of back-channel operations
> will require configuring an IdP cluster.  If a site is already doing
> that, it *may* also choose to use the cluster to synchronize, between
> nodes, that information which is carried in the cookie.  This would
> have some performance benefits.
> 
> Now, for first case mentioned above (conversational state), such state
> must be replicated in order to support mid-transaction failure
> recovery.  That is to say, a user goes to node 1, gets half way
> through a transaction and node 1 goes down.  User gets redirected to
> node 2 and is able to resume the transaction exactly where they left
> off.
> 
> We have observed that in order to support many of extensions and
> additional functionality within the profile handlers that people have
> requested, the amount of data we need to track and have available to
> any given profile handler step is quite large.  That in turn means
> that amount of data that would be need to be replicated is quite
> large.  Some of this data has no "natural" serialized form (a
> requirement for inter-node replication), and all clustering
> technologies require a priori knowledge of what objects will be
> replicated (e.g., Terracotta makes you list all the classes that will
> be clustered).  In addition, due to the large amount of data, there
> would be quite a bit of network traffic and additional latency within
> the synchronization of inter-node state.
> 
> We have reviewed a number of clustering technologies, and in
> particular focused on their serialization capabilities, in order to
> asses whether we can reasonably address the "Terracotta config
> problem".  What we have found is that there are ways to do this, but
> they require a hard coupling to the clustering technology.  In
> addition, implementing such a thing would add a significant amount of
> development time (we estimate between 4-6 months) and a highly complex
> (and thus likely to contain bugs) set of code to the IdP.  All
> extensions developers would also have to do the same work that we do
> if they are dealing with conversational state.
> 
> We believe that such mid-transaction failure recovery is not worth the
> cost of being locked in to a particular technology, the delay in the
> release of the IdP, nor the additional demands on the network.
> Instead, what we will implement is the detection of such a failure and
> allow sites to put up an error page in the event that such a situation
> occurs.
> 
> The result of this set of decisions is that many sites will be able to
> run their IdP in such a way that all relevant state is kept on the
> client side and thus not require the additional complexity of
> clustering software.  For those needing back-channel support, a
> clustering solution based on Infinispan will be provided but others
> (e.g., memcache, terracotta) could be plugged in.
> 
> -- 
> Chad La Joie
> www.itumi.biz
> trusted identities, delivered
> --
> To unsubscribe from this list send an email to dev-unsubscribe at shibboleth.net