looking for thoughts on IDP deploy architectures

[caleb racey]

Questions  replied to inline below

In terms of size we the  IdP handles an average of 19,000 fresh logins per day,   if you include total authentication requests then it's  36,000 per day (so people on average visit 2 shibbed sites per session).  We use the SPNEGO feature of the Kerberos login handler to provide "true single sign on".  We have 75 SPs registered (many of which are dev boxes)  and our largest SP is a blackboard cluster.

>high throughput -- traditionally, sites have run clustered IDPs, using
>various approaches to clustering (terracotta, jboss, stateless IDPs,
>etc). A newer approach is virtualization, allowing a site to dynamically
>expand and contract the "size" of the machine running an IDP). Is anyone
>doing that ?

We used to use terracotta,  it was the number one causes of service outages and made service recovery harder when there was another cause of outage.   We now just use vanilla shibboleth with a live instance and a warm spare, uptime has gone up significantly and maintenance has decreased.   Performance wise the cpu load is not an issue in that it uses one virtual cpu out of the 4 available,   it is memory heavy with 8GB statically  assigned in VMware.   We have a large attribute payload which may explain the memory use. 

>high availability, failover -- traditionally, sites have run multiple
>IDPs behind a load balancer. If an IDP encounters problems, or is
>undergoing maintenance, it is removed from the pool. Are sites using
>other approaches to this requirement ?

We don't use a load balancer, we have a warm spare with manual failover to it,  the servers are on top of a vmware private cloud which gives us site and hardware failure resilience

>High availability could also extend to services that the IDP may rely
>on. Authentication (perhaps kerberos) and attribute stores (perhaps
>ldap) are obvious examples, and are easy to also run with the "multiple
>server" approach.

Authentication is against a Microsoft active directory forest  and uses Kerberos so has failover between domain servers baked into the protocol (with appropriate config) Attributes are stored in the active directory so we have resilience from the forest there.   Group attributes and entitlement are managed via grouper and pushed into the active directory for shib to consume via ldap.

>less obvious is if the IDP requires a database (eg for storing
>persistent ID values, or perhaps consent decisions). What approaches are
>sites using to address high availability and failover requirements for
>the DB ?

Doesn't apply to us. 

Regards

Cal