<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">We had a partial outage with our SPs.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The error was “Unknown or Unusable Identity Provider”<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">It is supposed to periodically download the incommon idp only metadata<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> <MetadataProvider type="XML" uri="http://md.incommon.org/InCommon/InCommon-metadata-idp-only.xml"<o:p></o:p></p>
<p class="MsoNormal"> backingFilePath="InCommon-metadata-idp-only.xml" reloadInterval="7200"><o:p></o:p></p>
<p class="MsoNormal"> <MetadataFilter type="RequireValidUntil" maxValidityInterval="2419200"/><o:p></o:p></p>
<p class="MsoNormal"> <MetadataFilter type="Signature" certificate="incommon.pem"/><o:p></o:p></p>
<p class="MsoNormal"> </MetadataProvider><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">But the files on our server s are not consistent, and some are old. The file on the affected server was last downloaded in dec, and the 4 week interval expired and the SP stopped logging people in. You can see in some of our nodes (which
are configured the same), it is working correctly, and some need to be updated.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">ls -altr /var/cache/shibboleth/InCommon-metadata-idp-only.xml<o:p></o:p></p>
<p class="MsoNormal">Server1: -rw-r--r--. 1 shibd shibd 20748191 <span style="background:yellow;mso-highlight:yellow">
Dec 21</span> 00:59 /var/cache/shibboleth/InCommon-metadata-idp-only.xml<o:p></o:p></p>
<p class="MsoNormal">Server2: -rw-r--r--. 1 shibd shibd 20738297 Jan 3 15:18 /var/cache/shibboleth/InCommon-metadata-idp-only.xml<o:p></o:p></p>
<p class="MsoNormal">Server3 -rw-r--r--. 1 shibd shibd 20738297 Jan 3 15:17 /var/cache/shibboleth/InCommon-metadata-idp-only.xml<o:p></o:p></p>
<p class="MsoNormal">Server4: -rw-r--r--. 1 shibd shibd 20738297 Jan 3 15:16 /var/cache/shibboleth/InCommon-metadata-idp-only.xml<o:p></o:p></p>
<p class="MsoNormal">Server5: -rw-r--r--. 1 shibd shibd 20748191 <span style="background:yellow;mso-highlight:yellow">
Dec 21</span> 00:59 /var/cache/shibboleth/InCommon-metadata-idp-only.xml<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">QUESTION 1: Any idea why some servers are downloading and some aren’t?
<o:p></o:p></p>
<p class="MsoNormal">QUESTION 2: Should I be bouncing shibd weekly?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">QUESTION 3: I didn’t have the /Status handler hooked up, so it took a user report to find out about this. Would the /Status return a 500 and Nagios would tell me or do I need to look at the results for an <OK/>?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Bouncing the SP fixed this issue for now.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><Version Shibboleth="2.6.0"/><o:p></o:p></p>
<p class="MsoNormal"><NonWindows sysname="Linux" machine="x86_64"/><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">QUESTION 4: This was logged as a WARN, should it be an error?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">2018-01-04 11:17:50 WARN Shibboleth.SessionInitiator.SAML2 [6]: unable to locate metadata for provider (https://idp.pennkey.upenn.ed<o:p></o:p></p>
<p class="MsoNormal">u/idp/shibboleth)<o:p></o:p></p>
<p class="MsoNormal">2018-01-04 11:17:53 WARN OpenSAML.MetadataProvider [7]: ignored expired metadata instance for (https://idp.pennkey.upenn.edu/idp/shi<o:p></o:p></p>
<p class="MsoNormal">bboleth)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[root@fastprod-small-b-01 shibboleth]# grep -i error *<o:p></o:p></p>
<p class="MsoNormal">[root@fastprod-small-b-01 shibboleth]# grep -i fatal *<o:p></o:p></p>
<p class="MsoNormal">[root@fastprod-small-b-01 shibboleth]# pwd<o:p></o:p></p>
<p class="MsoNormal">/var/log/shibboleth<o:p></o:p></p>
<p class="MsoNormal">[root@fastprod-small-b-01 shibboleth]#<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><a href="https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPLogging">https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPLogging</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">“WARN: Indicates something that's noteworthy, but not an actual error.<o:p></o:p></p>
<p class="MsoNormal">ERROR: Something's wrong. Users are likely to be affected any time one appears.”<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks!<o:p></o:p></p>
<p class="MsoNormal">Chris<o:p></o:p></p>
</div>
</body>
</html>