IdP capacity / load testing advice

[Cantor, Scott]

On 7/10/15, 2:14 PM, "users on behalf of Jerry Shipman" <users-bounces at shibboleth.net on behalf of jes59 at cornell.edu> wrote:
>
>In case anyone is interested, I found a couple of things so far: 
>(1) telling the IdP to log at WARN instead of DEBUG increased the throughput by about 60%.

Yes, as I said, drastic.

>(2) IdP was CPU limited ... adding a second (virtual?) CPU to the VM (whatever it means) almost exactly doubled the throughput.

Also normally the case.

>After I did those two things, it looks (just from looking at the machines' "load average") like the SP is the limiting factor. (the SP still just has the one virtual CPU so I think it makes sense.)

That would be very surprising unless that VM is a dog.

>I think that I have got it set up such that the nameid is different for each iteration, even though the user is the same.
>* can someone verify that this is how I would tell: by looking at this in the SP's transaction.log ?

Looks like it.

>I am trying to figure out how to set up the IdP (or how many VMs I need, etc) in order to comfortably handle about 200 logins/second, which is (unless I screwed something up, which is always possible) the peak that I measured from our other (un-federated) login system.

I think that is vastly overestimating the load you'll see, but it's your environment. We don't see anything within an order of magnitude of that (no webmail admittedly). We do about 500,000 logins a day peak, more often closer to 250,000-300,000 and during summer much lower.

Even if you put 400,000 into an 8 hour window and ignore the other 16, the math doesn't come anywhere close to 200 per second. So maybe you have 4,000,000 logins a day through web?

-- Scott