Problem reading MetaData from URL

[Peter Schober]

* Adam Portier <aportier at haverford.edu> [2013-11-07 15:59]:
> We are trying to integrate our IdP with a SAML2 service provided by a
> vendor. When I try to set up a FileBackedHTTPMetadataProvider I get an
> marshaling error and that particular entry refuses to load. However, if I
> use curl to pull the metadata down into a file and delete the first line
> (containing the XML namespace declaration and encoding), then point a
> FilesystemMetadataProvider at that file, it loads it fine and we are able
> to use the service. Until the metadata recycles that is (every 24 hours).

Metadata establishes technical trust into entities by containing their
registered identity (entityID), protool endpoints and cryptographic
material (public key).
By regularly pulling SAML metadata from http is completely insecure
and as such not fit for establishing technical trust in the entity and
its keys.

> My question is twofold. Could someone who is better at debugging Metadata
> than I take a look at the link I have provided and tell me why Shibboleth's
> parsing engine is choking on the first line? I want to have something
> specific to ask these vendors to look at.

The wiki has en extensive page on that, called MetadataCorrectness.

That metadata is not schema valid, they can't just stuff several
certificates into a single KeyDescriptor and a single X509Certificate
element. The XML declaration (processing instruction) at the first
line won't do harm and removing it cannot make this invalid metadata
usable.
I have no explanation for the behaviour you describe.

> Second, in the meantime I am writing a script to pull the metadata
> down into a file manually and clean it up. Do I need to do anything
> to Shibboleth to get it to periodically re-read this file so we
> don't experience outages?

That's still fundamentall insecure to establish trust in any of the
information in that metadata.
They can't sensibly put a validUntil date on unsigned metadata hosted
via plain http and expect you to pull regularly. That doesn't make any
sense, security-wise.

> Metadata
> http://myreviewroom.com/saml2/metadata/

Both xmllint and XmlSecTool b0rk on the invalid representation of
several certs. So does the Shib IdP, with "Unable to unmarshall
metadata".
-peter