IdP 3.3.2 FileBackedHTTPMetadataProvider: RequiredValidUntilFilter & HTTP Conditional GET

[David Arnold]

I was hoping to get some guidance regarding an issue we experienced
using the Shibboleth IdP 3.3.2 FileBackedHTTPMetadataProvider with a
RequiredValidUntilFilter.

We're members of the Canadian Access Federation (CAF) so we have a
metadata provider configuration for their domestic aggregate:

<MetadataProvider id="CAF" xsi:type="FileBackedHTTPMetadataProvider"
  metadataURL="
https://caf-shib2ops.ca/CoreServices/caf_metadata_signed_sha256.xml"
  maxRefreshDelay="PT1H"
  backingFile="%{idp.home}/metadata/caf_metadata_signed_sha256.xml">
  ...
  <MetadataFilter xsi:type="RequiredValidUntil"
    maxValidityInterval="P30D" />
  ...
</MetadataProvider>

Their metadataURL endpoint supports HTTP conditional GET, so our IdP is
using it.  I verified that by turning on DEBUG logging for
org.apache.http:

http-outgoing-11308 >> GET /CoreServices/caf_metadata_signed_sha256.xml
HTTP/1.1
http-outgoing-11308 >> If-None-Match: "e0dbd-57edd7965dee5"
http-outgoing-11308 >> If-Modified-Since: Mon, 07 Jan 2019 12:36:13 GMT
http-outgoing-11308 >> Host: caf-shib2ops.ca
http-outgoing-11308 >> Connection: Keep-Alive
http-outgoing-11308 >> User-Agent: ShibbolethIdp/3.3.2 OpenSAML/3.3.1
http-outgoing-11308 >> Accept-Encoding: gzip,deflate
http-outgoing-11308 >> Connection: close
http-outgoing-11308 << "HTTP/1.1 304 Not Modified[\r][\n]"
http-outgoing-11308 << "Date: Thu, 17 Jan 2019 15:58:30 GMT[\r][\n]"
http-outgoing-11308 << "Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-
fips[\r][\n]"
http-outgoing-11308 << "Connection: close[\r][\n]"
http-outgoing-11308 << "ETag: "e0dbd-57edd7965dee5"[\r][\n]"

One day earlier this month, their server responded with new metadata
that had a validUntil slightly outside our validity window:

2019-01-07 07:56:48,875 - ERROR
[org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataReso
lver:468] - Metadata Resolver FileBackedHTTPMetadataResolver CAF: Error
filtering metadata from 
https://caf-shib2ops.ca/CoreServices/caf_metadata_signed_sha256.xml
org.opensaml.saml.metadata.resolver.filter.FilterException: Metadata's
validity interval P0Y0M30DT0H33M11.126S is larger than is allowed
P0Y0M30DT0H0M0.000S

No immediate problem since we still had the backing file from the
previous version... until the backing file version validUntil value
moved into the past yesterday.  The IdP still hadn't refreshed because
the HTTP conditional GET didn't detect a new version.  The upshot is
that a RequiredValidUntilFilter failure for a given payload is
essentially permanent (or you restart your IdP, clearing memory and any
HTTP cached values).

Is this expected behaviour?

Regards,

D.
-- 
David Arnold, Lead Architect - PeopleSoft
University Technology Services
McMaster University BSB233
1280 Main Street West
Hamilton, ON Canada L8S 4K1
Email: arnoldd at mcmaster.ca
Phone: 905-525-9140 x23843
Web: http://www.mcmaster.ca/uts