Freeze tomcat

Fri Jun 4 10:06:35 UTC 2021

not sure this is IDP specifics 
you may have tomcat and/or JVM tunning needed 
could this be related ? 
https://stackoverflow.com/questions/49978938/1000-threads-blocks-on-tomcat-response-process/49979529 

jehan . 

De: "jeanchristophe martin" <jeanchristophe.martin at orange.com> 
À: "Jehan PROCACCIA" <jehan.procaccia at tem-tsp.eu>, "Shib Dev" <dev at shibboleth.net>, "users" <users at shibboleth.net> 
Envoyé: Vendredi 4 Juin 2021 10:35:01 
Objet: RE: Freeze tomcat 

Hi, 

- shibboleth-idp : 3.4.6 

- java version : 1.8.0_292 

- tomcat version : 8.5.66 : apr 1.7 

- jvm sizing : -Xms1G -Xmx1G -XX:+UseG1GC 

Context : 

· LocalDynamicMetadataProvider : ~100 sp-metadata 

· JPAstorageService : Session Storage in Mariadb 

Problem : tomcat freeze periodically and we need to restart tomcat 

catalina.out -> 

"https-openssl-apr-8443-exec-193" #49381 daemon prio=5 os_prio=0 tid=0x00007f7c3c55b000 nid=0x29a1 waiting for monitor entry [0x00007f7c21928000] 

java.lang.Thread.State: BLOCKED (on object monitor) 

at org.opensaml.storage.ReplayCache.check(ReplayCache.java:132) 

- waiting to lock <0x00000000c8942538> (a org.opensaml.storage.ReplayCache) 

De : jehan Procaccia tem-tsp <jehan.procaccia at tem-tsp.eu> 
Envoyé : vendredi 4 juin 2021 09:48 
À : Shib Dev <dev at shibboleth.net>; MARTIN Jean-Christophe OBS/DD <jeanchristophe.martin at orange.com>; Shib Users <users at shibboleth.net> 
Objet : Re: Freeze tomcat 

Hi 

you would probably better send to [ mailto:users at shibboleth.net | 
users at shibboleth.net ] (which I add in copie) 

could you elaborate more 

- shibboleth-idp version 

- java version 

- tomcat version and httpd (if in front / reverse-proxy for TLS ?) 

- context (it happened after which action: update, restart , change config ...) 

- the log you show us, are tomcat logs, apache, idp-process ? 

thanks 

jehan . 

Le 04/06/2021 à 09:29, [ mailto:jeanchristophe.martin at orange.com | 
jeanchristophe.martin at orange.com ] a écrit : 

Hi 

We’ve recurent freeze on our shibboleth instance. 

In the thread dump, we’ve a lot of java.lang.Thread.State: BLOCKED (on object monitor) 

"https-openssl-apr-8443-exec-193" #49381 daemon prio=5 os_prio=0 tid=0x00007f7c3c55b000 nid=0x29a1 waiting for monitor entry [0x00007f7c21928000] 

java.lang.Thread.State: BLOCKED (on object monitor) 

at org.opensaml.storage.ReplayCache.check(ReplayCache.java:132) 

- waiting to lock <0x00000000c8942538> (a org.opensaml.storage.ReplayCache) 

Could you help me ? 

Thks 

BR 

_________________________________________________________________________________________________________________________ 
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc 
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler 
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, 
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. 
This message and its attachments may contain confidential or privileged information that may be protected by law; 
they should not be distributed, used or copied without authorisation. 
If you have received this email in error, please notify the sender and delete this message and its attachments. 
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. 
Thank you. 

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you. 

De: "jeanchristophe martin" <jeanchristophe.martin at orange.com> 
À: "Jehan PROCACCIA" <jehan.procaccia at tem-tsp.eu>, "Shib Dev" <dev at shibboleth.net>, "users" <users at shibboleth.net> 
Envoyé: Vendredi 4 Juin 2021 10:35:01 
Objet: RE: Freeze tomcat 

Hi, 

- shibboleth-idp : 3.4.6 

- java version : 1.8.0_292 

- tomcat version : 8.5.66 : apr 1.7 

- jvm sizing : -Xms1G -Xmx1G -XX:+UseG1GC 

Context : 

· LocalDynamicMetadataProvider : ~100 sp-metadata 

· JPAstorageService : Session Storage in Mariadb 

Problem : tomcat freeze periodically and we need to restart tomcat 

catalina.out -> 

"https-openssl-apr-8443-exec-193" #49381 daemon prio=5 os_prio=0 tid=0x00007f7c3c55b000 nid=0x29a1 waiting for monitor entry [0x00007f7c21928000] 

java.lang.Thread.State: BLOCKED (on object monitor) 

at org.opensaml.storage.ReplayCache.check(ReplayCache.java:132) 

- waiting to lock <0x00000000c8942538> (a org.opensaml.storage.ReplayCache) 

De : jehan Procaccia tem-tsp <jehan.procaccia at tem-tsp.eu> 
Envoyé : vendredi 4 juin 2021 09:48 
À : Shib Dev <dev at shibboleth.net>; MARTIN Jean-Christophe OBS/DD <jeanchristophe.martin at orange.com>; Shib Users <users at shibboleth.net> 
Objet : Re: Freeze tomcat 

Hi 

you would probably better send to [ mailto:users at shibboleth.net | 
users at shibboleth.net ] (which I add in copie) 

could you elaborate more 

- shibboleth-idp version 

- java version 

- tomcat version and httpd (if in front / reverse-proxy for TLS ?) 

- context (it happened after which action: update, restart , change config ...) 

- the log you show us, are tomcat logs, apache, idp-process ? 

thanks 

jehan . 

Le 04/06/2021 à 09:29, [ mailto:jeanchristophe.martin at orange.com | 
jeanchristophe.martin at orange.com ] a écrit : 

BQ_BEGIN

Hi 

We’ve recurent freeze on our shibboleth instance. 

In the thread dump, we’ve a lot of java.lang.Thread.State: BLOCKED (on object monitor) 

"https-openssl-apr-8443-exec-193" #49381 daemon prio=5 os_prio=0 tid=0x00007f7c3c55b000 nid=0x29a1 waiting for monitor entry [0x00007f7c21928000] 

java.lang.Thread.State: BLOCKED (on object monitor) 

at org.opensaml.storage.ReplayCache.check(ReplayCache.java:132) 

- waiting to lock <0x00000000c8942538> (a org.opensaml.storage.ReplayCache) 

Could you help me ? 

Thks 

BR 

_________________________________________________________________________________________________________________________ 
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc 
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler 
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, 
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. 
This message and its attachments may contain confidential or privileged information that may be protected by law; 
they should not be distributed, used or copied without authorisation. 
If you have received this email in error, please notify the sender and delete this message and its attachments. 
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. 
Thank you. 

BQ_END

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you. 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20210604/b88c5227/attachment.htm>