<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:Calibri;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:Calibri;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Hello shibboleth users,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I’m running into an issue where authentication seems to hang with this message in the idp-process.log:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">2016-12-15 11:33:40,858 - DEBUG [org.ldaptive.auth.PooledSearchDnResolver:244] - resolve user=joe_random<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">2016-12-15 11:33:40,859 - DEBUG [org.ldaptive.auth.PooledSearchDnResolver:310] - searching for DN using userFilter<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Normally this would be followed by a successful bind and then authentication but occasionally it will just hang there and then seemingly all logins are blocked until I restart tomcat.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I have three dc’s in my ldap pool but I can’t get the connectionStrategy to change from ACTIVE_PASSIVE but I don’t know if that has any bearing on this issue. DNS is working fine and an ldapsearch from any
of the servers returns the expected results immediately but I thought I should mention this. I’ve also done a tcpdump and watched as a number of users authenticate and the authentication – when it’s not hung – always goes to the first dc in the list. To
rule out a particular dc, I’ve changed the order and the first one in the list is always used and the problem also always eventually happens so I can’t say that a dc isn’t timing out but all other AD functionality is working fine.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I’m running it on a cluster of three identical virtual machines on Vmware that are load balanced with lvs with persistence at five minutes.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I am also using the memcached storage service which I suspect may be playing a role in this but I can’t put my finger on it.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I’ve tried removing all but one of the idp servers from the cluster for a day or so at a time but the issue always happens eventually so I don’t think it’s related to a particular box.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Any help would be greatly appreciated.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">My configuration is:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Idp version 3.1.1 <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Tomcat info:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Server version: Apache Tomcat/8.5.5<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Server built: Aug 31 2016 19:51:16 UTC<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Server number: 8.5.5.0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">OS Name: Linux<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">OS Version: 3.10.0-327.36.3.el7.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Architecture: amd64<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">JVM Version: 1.8.0_111-b14<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">JVM Vendor: Oracle Corporation<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">The spymemcached jar is spymemcached-2.11.4.jar.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Ldaptive jar is ldaptive-1.0.6.jar.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Ldap.properties:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># LDAP authentication configuration, see authn/ldap-authn-config.xml<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## Authenticator strategy, either anonSearchAuthenticator, bindSearchAuthenticator, directAuthenticator, adAuthenticator<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.authenticator = bindSearchAuthenticator<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## Connection properties ##<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.ldapURL = ldaps://my.redacted.domain.controller:3269 ldaps:// my.redacted.domain.controller:3269 ldaps:// my.redacted.domain.controller:3269 <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.useStartTLS = false<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.useSSL = true<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.authn.LDAP.connectTimeout = 5000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## SSL configuration, either jvmTrust, certificateTrust, or keyStoreTrust<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.authn.LDAP.sslConfig = certificateTrust<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## If using certificateTrust above, set to the trusted certificate's path<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.trustCertificates = %{idp.home}/credentials/redacteddomain.edu.crt<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## If using keyStoreTrust above, set to the truststore path<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.trustStore = %{idp.home}/credentials/ldap-server.truststore<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## Return attributes during authentication<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## NOTE: this is not used during attribute resolution; configure that directly in the<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## attribute-resolver.xml configuration via a DataConnector's <dc:ReturnAttributes> element<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.returnAttributes = mail,givenname,sn,objectguid,samaccountname<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">## DN resolution properties ##<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># Search DN resolution, used by anonSearchAuthenticator, bindSearchAuthenticator<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># for AD: CN=Users,DC=example,DC=org<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.baseDN = dc=redacted,dc=edu<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.subtreeSearch = true<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.userFilter = (| (samaccountname={user}) (userprincipalname={user}) )<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># bind search configuration<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># for AD: idp.authn.LDAP.bindDN=adminuser@domain.com<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.bindDN = cn=REDACTED_AD_USERNAME RO,ou=shibboleth,ou=services,dc=parkernet,dc=edu<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.bindDNCredential = REDACTED_AD_PASSWORD<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># Format DN resolution, used by directAuthenticator, adAuthenticator<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># for AD use idp.authn.LDAP.dnFormat=%s@domain.com<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.authn.LDAP.dnFormat = uid=%s,ou=people,dc=example,dc=org<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># LDAP attribute configuration, see attribute-resolver.xml<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.attribute.resolver.LDAP.ldapURL = %{idp.authn.LDAP.ldapURL}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.attribute.resolver.LDAP.baseDN = %{idp.authn.LDAP.baseDN}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.attribute.resolver.LDAP.bindDN = %{idp.authn.LDAP.bindDN}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.attribute.resolver.LDAP.bindDNCredential = %{idp.authn.LDAP.bindDNCredential}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.attribute.resolver.LDAP.useStartTLS = %{idp.authn.LDAP.useStartTLS:true}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.attribute.resolver.LDAP.trustCertificates = %{idp.authn.LDAP.trustCertificates}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">idp.attribute.resolver.LDAP.searchFilter = (| (samaccountname=$requestContext.principalName) (userprincipalname=$requestContext.principalName) )<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"># LDAP pool configuration, used for both authn and DN resolution<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.pool.LDAP.minSize = 3<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.pool.LDAP.maxSize = 10<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.pool.LDAP.validateOnCheckout = false<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.pool.LDAP.validatePeriodically = true<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.pool.LDAP.validatePeriod = 300<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.pool.LDAP.prunePeriod = 300<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.pool.LDAP.idleTime = 600<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.pool.LDAP.blockWaitTime = 3000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">#idp.pool.LDAP.failFastInitialize = false<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">memcached bean from global.xml:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><bean id="shibboleth.MemcachedStorageService"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> class="org.opensaml.storage.impl.memcached.MemcachedStorageService"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> c:timeout="2"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <constructor-arg name="client"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <bean class="net.spy.memcached.spring.MemcachedClientFactoryBean"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> p:servers="10.13.6.39:11211,10.13.6.42:11211,10.13.6.46:11211"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> p:protocol="BINARY"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> p:locatorType="ARRAY_MOD"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> p:failureMode="Redistribute"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <property name="hashAlg"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <util:constant static-field="net.spy.memcached.DefaultHashAlgorithm.FNV1_64_HASH" /><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </property><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <property name="transcoder"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <!-- DO NOT MODIFY THIS PROPERTY --><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <bean class="org.opensaml.storage.impl.memcached.StorageRecordTranscoder" /><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </property><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </bean><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </constructor-arg><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </bean><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
</body>
</html>