Shibboleth IdP plugin release - JDBC StorageService

[John C. Pfeifer]

I run Postgres w/ Commons DBCP in my production environment. I just tried the new plugin in my local development env and didn’t see any issue with logging…but there are issues. I  see in the logs:

shib-idp;idp-process.log;net.shibboleth.idp.session.SessionException: Session IDs are too large for StorageService, check configuration

In idp.properites:

idp.session.StorageService = shibboleth.JPAStorageService

The storage service configuration (post conversion) in global.xml is:

    <bean id="shibboleth.JPAStorageService"
      parent="shibboleth.JDBCStorageService"
      p:cleanupInterval="%{idp.storage.cleanupInterval:PT10M}"
      p:dataSource-ref="shibboleth.JPAStorageService.DataSource"/>

    <bean id="shibboleth.JPAStorageService.DataSource"
        class="org.apache.commons.dbcp2.BasicDataSource" destroy-method="close" lazy-init="true"
        p:driverClassName="org.postgresql.Driver"
        p:url="%{edu.umd.idp.dbConnection}"
        p:password="%{idp.postgres.password}" />


//
John Pfeifer
Division of Information Technology
University of Maryland, College Park

> On Jul 6, 2022, at 6:15 AM, Martin Hitschel via users <users at shibboleth.net> wrote:
> 
> Dear list,
> 
> by now we've had two customers trying the new plugin. One with MySQL and
> Hikari, that one works well with the new plugin.
> 
> The other has PostGres and Tomcat DBCP2, that one does not work. Upon
> conversion of global.xml to the new plugin and Tomcat restart, the IdP
> does not log anymore to the process log. In the journalctl, we see no
> real error except that it can't find the class for Logging and the
> memoryStorageServive class, which seems strange to us.
> 
> Here's global.xml before:
> 
> (...)
> 
>     <bean id="shibboleth.PostgreSQLDataSource"
>       class="org.apache.tomcat.dbcp.dbcp2.BasicDataSource"
>           p:driverClassName="org.postgresql.Driver"
>           p:url="jdbc:postgresql://localhost/shibboleth"
>           p:username="shibboleth"
>           p:password="%{psql.password}"
>           p:validationQuery="select 1"
>           p:validationQueryTimeout="5" />
> 
>     <bean id="shibboleth.JPAStorageService"
>           class="org.opensaml.storage.impl.JPAStorageService"
>           p:cleanupInterval="%{idp.storage.cleanupInterval:PT10M}"
>          
> c:factory-ref="shibboleth.JPAStorageService.entityManagerFactory" />
> 
>     <bean id="shibboleth.JPAStorageService.entityManagerFactory"
>          
> class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
>         <property name="packagesToScan" value="org.opensaml.storage.impl"/>
>         <property name="dataSource" ref="shibboleth.PostgreSQLDataSource"/>
>         <property name="jpaVendorAdapter"
> ref="shibboleth.JPAStorageService.JPAVendorAdapter"/>
>         <property name="jpaDialect">
>             <bean
> class="org.springframework.orm.jpa.vendor.HibernateJpaDialect" />
>         </property>
>     </bean>
> 
>     <bean id="shibboleth.JPAStorageService.JPAVendorAdapter"
>          
> class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"
>           p:generateDdl="true"
>           p:database="POSTGRESQL"
>           p:databasePlatform="org.hibernate.dialect.PostgreSQL9Dialect" />
> 
> (...)
> 
> After conversion, it looks like this:
> 
> (...)
> 
>     <bean id="shibboleth.PostgreSQLDataSource"
>       class="org.apache.tomcat.dbcp.dbcp2.BasicDataSource"
>           p:driverClassName="org.postgresql.Driver"
>           p:url="jdbc:postgresql://localhost/shibboleth"
>           p:username="shibboleth"
>           p:password="%{psql.password}"
>           p:validationQuery="select 1"
>           p:validationQueryTimeout="5" />
> 
>     <bean id="shibboleth.JPAStorageService"
>       parent="shibboleth.JDBCStorageService"
>       p:cleanupInterval="%{idp.storage.cleanupInterval:PT10M}"
>       p:dataSource-ref="shibboleth.PostgreSQLDataSource"/>
> (...)
> 
> There are also httpclient beans in global.xml, but they do not seem to
> matter. Actually, journalctl outputs that the correct number of beans
> have been loaded from the file.
> 
> We've tried back and forth, with the old config the IdP logs and works,
> with the new it does not log and does not write/read to/from the database.
> 
> We've also tried Hikari, that had the same issues.
> 
> Cheers
> 
> Martin
> 
> 
> Am 14.06.22 um 17:31 schrieb Cantor, Scott via announce:
>> The Shibboleth Project has released a new plugin for the IdP software
>> that replaces the built-in but now deprecated Hibernate/JPA-based
>> storage option with a JDBC-based implementation.
>> 
>> https://shibboleth.atlassian.net/wiki/spaces/IDPPLUGINS/pages/2989096970/JDBCStorageService
>> 
>> This plugin is a drop in replacement for the JPA version, same
>> database schema and essentially similar configuration (docs describe
>> how to switch). Due to reliability challenges and the lack of
>> appropriately signed Java artifacts for Hibernate, we will be removing
>> the original code in the next major IdP release (V5) sometime next year.
>> 
>> We are making the replacement available now so people can move to it
>> ahead of time (and help find any bugs so we can get it into shape). It
>> also features control over the SQL for special cases and includes
>> internal locking at the code level to reduce contention at least
>> within a particular server node in a cluster in the hopes that it will
>> reduce the constant "deadlocks that aren't actually deadlocks" caused
>> by modern database mis-design.
>> 
>> We hope it will be more reliable (better performance is not the goal,
>> stability is), but that locking is optional and not on by default.
>> 
>> Bugs can be reported against the JJDBC project.
>> 
>> Enjoy,
>> -- Scott
>> 
>> --
>> To unsubscribe from this list send an email to
>> announce-unsubscribe at shibboleth.net
> 
> -- 
> Dr. Martin Hitschel, Senior Consultant
> 
> DAASI International
> Europaplatz 3                   D-72072 Tübingen                Germany                    
> phone: +49 7071 407109-0
> fax:   +49 7071 407109-9  email: martin.hitschel at daasi.de
> web:   www.daasi.de
> 
> Sitz der Gesellschaft: Tübingen
> Registergericht: Amtsgericht Stuttgart, HRB 382175
> Geschäftsleitung: Peter Gietz
> 
> -- 
> For Consortium Member technical support, see https://shibboleth.atlassian.net/wiki/x/ZYEpPw
> To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net