Ldap - BlockingTimeoutException

[emilio.penna]

Hi, I am analyzing an issue with an IdP that happened last sunday. The 
IdP  (3.1.2, with openldap)  was functioning normally, with low load, 
and from a certain time all login attempts begin to fail with a "Block 
time exceeded" message, until tomcat was restarted.

Reviewing the logs of that day:

- the idp was functioning normally, with low load and no abnormal errors 
in the log
- at 15:11 hs there was one exception: "LDAP response read timed out" 
(copied below)
- immediatly afterwards began the exception 
"org.ldaptive.pool.BlockingTimeoutException: Block time exceeded"
- next, all login attemps fail with the previous exception
- hours later, an administrator restarted tomcat (and not restarted 
openldap) and the exception stopped and the idp started to work normally.

I dont see anything abnormal in other logs.

It would seem like the connection pool was blocked or all the 
connections was blocked and not returned to the pool.

The pool configuration is the default, the only modified property is 
idp.authn.LDAP.connectTimeout = 6000

Any ideas on what may have happened?
Any suggestions to avoid this situation, maybe pool adjustment?

thanks
Emilio

----

2018-07-22 15:11:38,628 - WARN 
[net.shibboleth.idp.authn.impl.ValidateUsernamePasswordAgainstLDAP:212] 
- Profile Action ValidateUsernamePasswordAgainstLDAP: Login by ... 
produced exception
org.ldaptive.LdapException: javax.naming.NamingException: LDAP response 
read timed out, timeout used:6000ms
         at 
org.ldaptive.provider.ProviderUtils.throwOperationException(ProviderUtils.java:77)
Caused by: javax.naming.NamingException: LDAP response read timed out, 
timeout used:6000ms.
         at com.sun.jndi.ldap.Connection.readReply(Connection.java:481)

2018-07-22 15:11:44,613 - WARN 
[net.shibboleth.idp.authn.impl.ValidateUsernamePasswordAgainstLDAP:212] 
- Profile Action ValidateUsernamePasswordAgainstLDAP: Login ... produced 
exception
org.ldaptive.pool.BlockingTimeoutException: Block time exceeded
         at 
org.ldaptive.pool.BlockingConnectionPool.blockAvailableConnection(BlockingConnectionPool.java:243)