IdP use of LDAP and connection pooling

[Manuel Haim]

Hi Scott,

just my two cents here...

> - are pools necessary to get reasonable performance on highly loaded IdPs?
We have an IdP cluster and define three OpenLDAP servers within the
attribute resolver's DataConnector (for failover reasons, i.e. each
server has the same data). Initially, we did not define a
ConnectionPool. With this setup, we did some load testing and debugging.

When performing just one login at a time, and some seconds later just
another one, the attribute resolver performs a costly LDAP BIND
operation for each of these logins (taking about 50ms), i.e. the LDAP
connection is closed shortly after attribute resolution. This is okay,
as the few users will not recognize this 50ms, and open connections are
reduced to a minimum.
Then, if we fire lots of login requests to the IdP (using "The
Grinder"), the IdP seems to use a connection pool even if it has not
been explicitly defined (there is no LDAP BIND operation each time).
This is just fine, too, as the system manager does not need to care
about connection pools, and each single user still gets good performance.

For comparison, after the LDAP BIND, a single LDAP request just takes
about 3ms here.

A bottleneck, however, appears to be in the LDAP JAAS login module at
login time (see login.config). By default, the SearchDnResolver (which
resolves the user's DN according to the specified userFilter) never does
connection pooling, thus the IdP always performs an LDAP BIND here where
it could keep the connection open. We replaced the SearchDnResolver by a
static one for test purposes, and our IdP cluster now handled about
twice as much logins per second. (The IdP is not "blocked" by the LDAP
BINDs, but maybe the number of threads or network connections is at a
limit here?!)

This issue has been reported at:
http://code.google.com/p/vt-middleware/issues/detail?id=118

Please remember that the LDAP AUTHN request itself (in contrast to the
SearchDnResolver issue) will always remain using an LDAP BIND, as it
must bind in the name of the user who is going to be authenticated.

> - do they handle failed connections reasonably without ever surfacing them
> as actual data connector failures?
This has not been an issue here yet. "The Grinder" reports approximately
20 errors per 10.000 logins, but I did never catch one in the browser
myself.

-Manuel