Errors after Relying Party Service Reload

Tue Apr 3 09:07:35 BST 2012

Just filed.

SIDP-545

R.
--
Dr Rhys Smith
Identity, Access, and Middleware Specialist
Cardiff University & Janet - the UK's education and research network

email: smith at cardiff.ac.uk / rhys.smith at ja.net
GPG: 0xDE2F024C

On 3 Apr 2012, at 08:49, Rhys Smith wrote:

> FYI, just tried this out on my preprod cluster and I'm seeing exactly the same issue. I have the prod and preprod clusters reloading its config files automagically, but hadn't actually changed the relying-party.xml file since switching to this new setup a few months ago (though have changed attribute config and filtering rules, which works just fine). Simply touched the file, it reloaded it fine with no errors, but then I'm seeing this in the logs:
> 
> 08:44:27.686 - ERROR [edu.internet2.middleware.shibboleth.idp.profile.saml2.AbstractSAML2ProfileHandler:923] - Could not resolve a key encryption credential for peer entity: https://sh2testsp1.iay.org.uk/shibboleth
> 08:44:27.688 - ERROR [edu.internet2.middleware.shibboleth.idp.profile.saml2.AbstractSAML2ProfileHandler:289] - Unable to construct encrypter
> org.opensaml.xml.security.SecurityException: Could not resolve key encryption credential
> 
> IdP v2.3.6 on CentOS 6 x64, tomcat 6.0.24, OpenJDK 1.6.0_22, Terracotta 3.6.1
> 
> Time to file a bug I guess...
> 
> R.
> --
> Dr Rhys Smith
> Identity, Access, and Middleware Specialist
> Cardiff University & Janet - the UK's education and research network
> 
> email: smith at cardiff.ac.uk / rhys.smith at ja.net
> GPG: 0xDE2F024C
> 
> On 2 Apr 2012, at 21:02, Slaughter, Brett wrote:
> 
>> We’ve configured IdP v2.3.6 on RHEL6 x64 to reload the attribute filter engine, attribute resolver, and relying party configuration manager services every 2 hours in service.xml. The IdP starts normally and runs fine, however the IdP starts throwing errors after the relying party configuration manager reloads following a manual change to relying-party.xml even though the affected SPs were accessed successfully prior to the reload. These issues are resolved after restarting Tomcat. The SPs involved in the error’d transactions differ after each Tomcat restart/relying-party update. The specific errors encountered on the IdP are pasted below, and result in a FatalProfileException with message “Unable to encrypt assertion” on the SP.
>>  
>> 12:02:00.516 - ERROR [edu.internet2.middleware.shibboleth.idp.profile.saml2.AbstractSAML2ProfileHandler:923] - [cl.ie.nt.ip] - Could not resolve a key encryption credential for peer entity: https://some-sp.domain.edu/shibboleth
>> 12:02:00.518 - ERROR [edu.internet2.middleware.shibboleth.idp.profile.saml2.AbstractSAML2ProfileHandler:289] - [cl.ie.nt.ip] - Unable to construct encrypter
>> org.opensaml.xml.security.SecurityException: Could not resolve key encryption credential
>>         at edu.internet2.middleware.shibboleth.idp.profile.saml2.AbstractSAML2ProfileHandler.getEncrypter(AbstractSAML2ProfileHandler.java:924) ~[shibboleth-identityprovider-2.3.6.jar:na]
>>         at edu.internet2.middleware.shibboleth.idp.profile.saml2.AbstractSAML2ProfileHandler.buildResponse(AbstractSAML2ProfileHandler.java:286) ~[shibboleth-identityprovider-2.3.6.jar:na]
>>         at edu.internet2.middleware.shibboleth.idp.profile.saml2.SSOProfileHandler.completeAuthenticationRequest(SSOProfileHandler.java:310) [shibboleth-identityprovider-2.3.6.jar:na]
>>         at edu.internet2.middleware.shibboleth.idp.profile.saml2.SSOProfileHandler.processRequest(SSOProfileHandler.java:177) [shibboleth-identityprovider-2.3.6.jar:na]
>>         at edu.internet2.middleware.shibboleth.idp.profile.saml2.SSOProfileHandler.processRequest(SSOProfileHandler.java:88) [shibboleth-identityprovider-2.3.6.jar:na]
>>         at edu.internet2.middleware.shibboleth.common.profile.ProfileRequestDispatcherServlet.service(ProfileRequestDispatcherServlet.java:84) [shibboleth-common-1.3.5.jar:na]
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) [servlet-api.jar:na]
>>         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) [catalina.jar:6.0.35]
>>         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:6.0.35]
>>         at edu.internet2.middleware.shibboleth.idp.util.NoCacheFilter.doFilter(NoCacheFilter.java:50) [shibboleth-identityprovider-2.3.6.jar:na]
>>         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) [catalina.jar:6.0.35]
>>         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:6.0.35]
>>         at edu.internet2.middleware.shibboleth.idp.session.IdPSessionFilter.doFilter(IdPSessionFilter.java:81) [shibboleth-identityprovider-2.3.6.jar:na]
>>         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) [catalina.jar:6.0.35]
>>         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:6.0.35]
>>         at edu.internet2.middleware.shibboleth.common.log.SLF4JMDCCleanupFilter.doFilter(SLF4JMDCCleanupFilter.java:52) [shibboleth-common-1.3.5.jar:na]
>>         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) [catalina.jar:6.0.35]
>>         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:6.0.35]
>>         at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) [catalina.jar:6.0.35]
>>         at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) [catalina.jar:6.0.35]
>>         at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) [catalina.jar:6.0.35]
>>         at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [catalina.jar:6.0.35]
>>         at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [catalina.jar:6.0.35]
>>         at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) [catalina.jar:6.0.35]
>>         at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190) [tomcat-coyote.jar:6.0.35]
>>         at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291) [tomcat-coyote.jar:6.0.35]
>>         at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:776) [tomcat-coyote.jar:6.0.35]
>>         at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:705) [tomcat-coyote.jar:6.0.35]
>>         at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:898) [tomcat-coyote.jar:6.0.35]
>>         at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690) [tomcat-coyote.jar:6.0.35]
>>         at java.lang.Thread.run(Thread.java:722) [na:1.7.0_03]
>>  
>>  
>>  
>> The reload configuration for relying party in service.xml is (polling attributes are the same for attribute filter and resolver):
>> <srv:Service id="shibboleth.RelyingPartyConfigurationManager" xsi:type="relyingParty:SAMLMDRelyingPartyConfigurationManager" depends-on="shibboleth.SAML1AttributeAuthority shibboleth.SAML2AttributeAuthority" configurationResourcePollingFrequency="PT120M" configurationResourcePollingRetryAttempts="12">
>> <srv:ConfigurationResource file="/opt/shibboleth-idp/conf/relying-party.xml" xsi:type="resource:FilesystemResource"/>
>> </srv:Service>
>>  
>> The IdP Common Errors page on the wiki suggests the “Unable to construct encrypter” error is related to the SP metadata not having an encryption certificate. I’ve confirmed that each of the SPs encountering this error all have a certificate, all of which omit the ‘use’ attribute. Somewhere in the reload of the relying party file, certain certificates are not being read from the SP metadata files, though they are read successfully during IdP startup.
>>  
>> This behavior can be reproduced in my DEV and QA environments. The only change made to relying-party.xml between automatic reloads is to `touch` the file to update the modified timestamp. I’ve removed the other reloading configurations, leaving relying party as the only service being reloaded, and the problem persists. There’s nothing in the logs (at DEBUG level) for the reload indicating anything failed or was not successfully reloaded.
>>  
>> We are well aware of the warning on the wiki regarding use of the configuration reloading in production, and we have sufficient configuration testing in place to prevent invalid configurations from being loaded into the production environment. The goal of using the reloading features is to provide shorter turn-around when adding new SPs to the IdP and modifying attributes and release policies without resetting session state for all authenticated users by restarting Tomcat entirely. Given this issue, however, we’ve delayed implementation of this feature and instead scheduled a restart of Tomcat nightly to pick up any changes made the previous day, which means even the smallest change doesn’t take effect until the following day.
>>  
>> What is preventing the IdP from using an SP’s encryption certificate following a reload of the relying party service, when the same SP works fine prior the reload?
>>  
>>  
>> Server: RHEL6 x64
>> IdP: 2.3.6
>> Tomcat: 6.0.35.0
>> JVM: 1.7.0_03
>>  
>>  
>> Brett Slaughter
>> Information Security & Access Management | ISAM
>> Division of IT, University of Missouri
>> slaughter at missouri.edu
>> --
>> To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://shibboleth.net/pipermail/users/attachments/20120403/3b351ece/attachment-0001.html 