Shibboleth IdP Capacity Planning

[Cameron Kerr]

To give you one (albeit vague) datapoint, we have recently deployed Shibboleth IdP which successfully authenticates perhaps a similar rate, likely more.

It's a single, small VM, and it spends most of its time largely idle; it is neither CPU or memory bound.

Nor does it seem to consume much in the way of system entropy.

Its performance is largely bound on that of the LDAP server that its talks to.

I would suggest that you don't look at average so much; I keep statistics based on 5-minute readings (it gets pumped into a custom RRD database). The graphs from this show intervals throughout the day (maximum resolution, shows variability quite well), weekly (for looking at daily trends), down to yearly (for trending service adoption). Its good to track maximums as well; particularly for capacity planning.

My main statistics are based on the releases, in idp-audit.log, which I break up into the various different major SPs, so I can more easily learn about per-SP activity. After changing the log timestamp format, I also charted the AuthN and AuthZ activity as well.

The variability is very high, so if you average over an hour you'll get a radically different capacity planning outcome.

Another thing I do, since I have Apache fronting Tomcat, is to adjust Tomcat logging so that it includes the service-time (%D). I then chart mimimum, average and maximum service time in another RRD database. In my particular environment, it turns out to be quite useful as a way to keep an eye on user experience.


Since performance in my environment is (so) largely dependent on LDAP activity, if I were load-testing, it would be easiest to do the load-test on the LDAP server instead; just remember to load-test with the same types of queries, as response time is largely dependent on the type of query submitted.

With regard to testing the IdP, I would suggest that load is the wrong (or at least not the best) thing to measure. For example, if you have a clustered IdP deployment, I would be more interested to know how easy/likely it would be that a failure would surface if the cluster were to be in flux (eg. session instability, assymetrical loading, learning what sort of maintenance operations we can/cannot get away without people noticing, such as routine -- possibly automated -- patching and rebooting). I would also suggest that if you're doing this, you should also consider doing the same modelling for things like Logout (or Artifact Resolution -- or anything else that gets complicated in a clustered deployment).

Also, user-experience in our experience is more largely affected by the workflow -- particularly if using something like uApprove, which we found to be the most objectionable/confusing part for users and adoptors. As far as performance goes, the Shibboleth IdP raises no issues of its own (only mirroring those from our LDAP server -- and those it seems to deal with gracefully enough).

Hope it helps,
Cameron

-----Original Message-----
From: users-bounces at shibboleth.net [mailto:users-bounces at shibboleth.net] On Behalf Of Cantor, Scott
Sent: Tuesday, 4 March 2014 4:34 a.m.
To: Shib Users
Subject: Re: Shibboleth IdP Capacity Planning

On 3/3/14, 9:46 AM, "Carlos Milán Figueredo" <cmilanf at hispamsx.org> wrote:

>Hi *,
> 
>I¹m a bit confused about how to do some capacity planning on Shibboleth 
>IdP in order to know how many servers I have to deploy in order to 
>support highly loaded environments.

You load test, and then extrapolate, pretty much the way you would with any server software.

> 
>Is it a good guidance to follow the same specs as their current 
>GNU/Linux based LDAP servers?

I doubt it.

-- Scott


--
To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net