Odd memory leak

[Josh Helmer]

Hey team.

I wanted to pass this along just in case.  

We have been running an shib 2.4 for quite a while on one of the
projects I am part of.  It's been working fine.

We recently upgraded to 3.2.  Now that traffic is starting to pick up
again, we've had issues where a couple of times the IDP has died with
memory problems.   Something changed between 2.4 and 3.2 where we were
leaking memory.   

After spending a couple of days setting up jmeter scripts and digging
through heaps, I finally figured out what is going on.   We are using
the JAAS module and then have custom code in place to authenticate to a
local database.   In 2.4 that worked fine.  In 3.2 it seems that for
every login attempt the IDP calls the initialize() method of the
LoginModule.  Our code was constantly creating new connection pools
ever time you logged in.  Those pools were getting attached to Thread
pools that were long lived.  So, the longer the IDP was up, the more
connection pools we would generate.  Eventually we would have 20,000 or
more of them active and the IDP would run out of memory.   For some
reason that wasn't occurring with 2.4.  

So, I just wanted to pass along this finding in case anyone else runs
into similar problems with the JAAS configuration.  I'm not quite sure
what changed, but something in the way that the LoginModule impl is
called is different and that led to a situation where it exposed a bug
in our code.

Josh