DB-backed storage service: testing database pool status?

[Wessel, Keith]

Hi, all,

We're running the database-backed storage service, and we've run into a problem twice in the past two weeks where all of our database connections are handed out and the IDP can't get a connection from the pool. For whatever reason, they all get handed out and never returned. Result is that, after submitting username and password, the IDP hangs. I'm trying to tweak my resource parameters to prevent this situation from happening, but when it does, our usual monitoring via /idp/status isn't detecting it, obviously.

Short of having our monitoring log into the IDP, is there anything I can do to hit a URL or perform some simple action that would verify that my database connection pool is behaving itself? If not, I may have to resort to a log grepping script.

FWIW, the error in the log is "Cannot get a connection, pool error timeout waiting for idle object

Any suggestions on how I can monitor this from our service monitoring and our  SLB health checks?

Keith