outage due to not downloading incommon metadata

Thu Jan 4 15:22:28 EST 2018

While this isn't the underlying cause here, another issue in the December time frame was that InCommon failed over http://md.incommon.org/ from the Michigan server (207.75.165.125) to the Los Angeles server (163.253.32.9) between at least 12/13/2017 and 12/27/2017.

We'd sent our campus SP users a number of notices about the (potential) need for an additional firewall rule back in August 2017, but sure enough, we started receiving reports from a few campus SPs who had ignored the memo and were experiencing failed logins in late December.

Michael

________________________________
From: users <users-bounces at shibboleth.net> on behalf of Cantor, Scott <cantor.2 at osu.edu>
Sent: Thursday, January 4, 2018 1:15:44 PM
To: Shib Users
Subject: Re: outage due to not downloading incommon metadata

On 1/4/18, 2:06 PM, "users on behalf of Hyzer, Chris" <users-bounces at shibboleth.net on behalf of mchyzer at isc.upenn.edu> wrote:

> QUESTION 1: Any idea why some servers are downloading and some aren’t?

Some are fully patched and some are running the broken xmltooling pushed out with the original 2.6 patch.

> QUESTION 2: Should I be bouncing shibd weekly?

No.

> QUESTION 3: I didn’t have the /Status handler hooked up, so it took a user report to find out about this.  Would the
> /Status return a 500 and Nagios would tell me or do I need to look at the results for an <OK/>?

Simply monitoring with a login is the usual complete answer but otherwise if the status dump includes the dates associated with the metadata, it's probably worth something to look at it.

 > Bouncing the SP fixed this issue for now.

It won't stay fixed if it's not patched.

> QUESTION 4: This was logged as a WARN, should it be an error?

No...

> https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPLogging
>“WARN: Indicates something that's noteworthy, but not an actual error.
> ERROR: Something's wrong. Users are likely to be affected any time one appears.”

That is not at all consistent with the the levels as they're used, so I'll have to change the text.

-- Scott

--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20180104/89591cbd/attachment.html>