=?Windows-1252?Q?Umlaut_(=DC, =C4, =D6)_in_attribute_resolution_not_UTF-8_e?= ncoded

[Grüter, Philip]

Hi all,

I currently have a problem with Umlauts (Ü, Ä, Ö) with my Shibboleth instance.
The output I get (with aacli.sh and over a normal SAML2 session) is this:
  {
    "name": "sn",
    "values": [
        "Müller"
    ]
  },

“ü” is the corresponding Latin-1 encoding for Ü, but I can't find where it's coming from.

I'm still running 4.3.0, we are in the process of upgrading. The server uses Jetty9 and java-11-amazon-corretto-jdk.

We pull attributes out of a MySQL database during attribute resolution. The database is encoded with UTF-8.
The "table" we access, is a view, which is creted from two different tables. Both tables and the view is also set to UTF-8.

These are my configs and the Log output:

gloabl.xml:
<bean id="shibboleth.MySQLExtraDataSource"
                class="%{mysqlextra.class}"
                p:driverClassName="com.mysql.jdbc.Driver"
                p:jdbcUrl="%{mysqlextra.url}"
                p:username="%{mysqlextra.username}"
                p:password="%{mysqlextra.password}" />


idp.properties DB Connection:
mysqlextra.class = com.zaxxer.hikari.HikariDataSource
mysqlextra.url = jdbc:mysql://REDACTED IP:6033/userdb?useUnicode=true&characterEncoding=UTF-8


attribute-resolver.xml:

    <AttributeDefinition id="sn" xsi:type="Template">
        <InputDataConnector ref="mysqlextra" attributeNames="db_surname" />
        <Template>
          <![CDATA[
               ${db_surname}
          ]]>
        </Template>
    </AttributeDefinition>

…

    <DataConnector xsi:type="RelationalDatabase" id="mysqlextra">
        <BeanManagedConnection>shibboleth.MySQLExtraDataSource</BeanManagedConnection>
            <QueryTemplate>
            <![CDATA[
                    SELECT email,forename,surname from moodle2_auth where username='$resolutionContext.principal'
            ]]>
            </QueryTemplate>
        <Column columnName="email" attributeID="db_mail" />
        <Column columnName="forename" attributeID="db_forename" />
        <Column columnName="surname" attributeID="db_surname" />
    </DataConnector>


Debug Log Output after requesting Attributes with the aacli.sh Utility:
2025-02-13 09:56:15,701 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Initiating attribute resolution with label: admin/resolvertest
2025-02-13 09:56:15,701 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Attempting to resolve the following attribute definitions [uid, givenName, sn, mail]
2025-02-13 09:56:15,701 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Resolving dependencies for 'uid'
2025-02-13 09:56:15,701 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Finished resolving dependencies for 'uid'
2025-02-13 09:56:15,702 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.AbstractAttributeDefinition:?] - Attribute Definition 'uid': produced an attribute with the following values [StringAttributeValue{value=REDACTED USER ID}]
2025-02-13 09:56:15,704 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Attribute definition 'uid' produced an attribute with 1 values
2025-02-13 09:56:15,706 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Resolving dependencies for 'givenName'
2025-02-13 09:56:15,706 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Resolving dependencies for 'mysqlextra'
2025-02-13 09:56:15,706 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Finished resolving dependencies for 'mysqlextra'
2025-02-13 09:56:15,706 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Resolving data connector mysqlextra
2025-02-13 09:56:15,717 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.dc.rdbms.TemplatedExecutableStatementBuilder:?] - Template text SELECT email,forename,surname from moodle2_auth where username='$resolutionContext.principal' yields SELECT email,forename,surname from moodle2_auth where username='REDACTED USER ID'
2025-02-13 09:56:15,758 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.dc.rdbms.StringResultMappingStrategy:?] - Remapping column email to db_mail
2025-02-13 09:56:15,758 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.dc.rdbms.StringResultMappingStrategy:?] - Remapping column forename to db_forename
2025-02-13 09:56:15,759 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.dc.rdbms.StringResultMappingStrategy:?] - Remapping column surname to db_surname
2025-02-13 09:56:15,759 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.AbstractDataConnector:?] - Data Connector 'mysqlextra': produced the following 3 attributes during resolution [db_surname, db_mail, db_forename]
2025-02-13 09:56:15,759 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.AbstractDataConnector:?] - Data Connector 'mysqlextra': Attribute 'db_surname': Values '[StringAttributeValue{value=Müller}]'
2025-02-13 09:56:15,759 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.AbstractDataConnector:?] - Data Connector 'mysqlextra': Attribute 'db_mail': Values '[StringAttributeValue{value=REDACTED USER MAIL}]'
2025-02-13 09:56:15,759 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.AbstractDataConnector:?] - Data Connector 'mysqlextra': Attribute 'db_forename': Values '[StringAttributeValue{value=REDACTED USER NAME}]'
2025-02-13 09:56:15,760 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Data connector 'mysqlextra' resolved the following attributes: [db_surname, db_mail, db_forename]
2025-02-13 09:56:15,762 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Finished resolving dependencies for 'givenName'
2025-02-13 09:56:15,763 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?] - Attribute Definition 'givenName': Determining value 1
2025-02-13 09:56:15,764 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?] - Attribute Definition 'givenName': Adding value 'REDACTED USER NAME' for attribute 'db_forename' to the template context
2025-02-13 09:56:15,764 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?] - Attribute Definition 'givenName': Evaluating template
2025-02-13 09:56:15,766 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?] - Attribute Definition 'givenName': Result of template evaluating was 'REDACTED USER NAME'
2025-02-13 09:56:15,766 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.AbstractAttributeDefinition:?] - Attribute Definition 'givenName': produced an attribute with the following values [StringAttributeValue{value=REDACTED USER NAME}]
2025-02-13 09:56:15,766 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Attribute definition 'givenName' produced an attribute with 1 values
2025-02-13 09:56:15,766 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Resolving dependencies for 'sn'
2025-02-13 09:56:15,766 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Finished resolving dependencies for 'sn'
2025-02-13 09:56:15,767 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?] - Attribute Definition 'sn': Determining value 1
2025-02-13 09:56:15,767 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?] - Attribute Definition 'sn': Adding value 'Müller' for attribute 'db_surname' to the template context
2025-02-13 09:56:15,767 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?] - Attribute Definition 'sn': Evaluating template
2025-02-13 09:56:15,768 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?] - Attribute Definition 'sn': Result of template evaluating was 'Müller'
2025-02-13 09:56:15,768 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.AbstractAttributeDefinition:?] - Attribute Definition 'sn': produced an attribute with the following values [StringAttributeValue{value=Müller}]
2025-02-13 09:56:15,768 - 127.0.0.1 - DEBUG [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] - Attribute Resolver 'ShibbolethAttributeResolver': Attribute definition 'sn' produced an attribute with 1 values


Things I tried:
- Adding "?useUnicode=true&characterEncoding=UTF-8" to the jdbc URL
- Changing locales with ISO-8859-1 to UTF-8 in /etc/jetty9/webdefault.xml

Does anyone have any other ideas on things I can try to fix this?

Greetings,
Philip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20250213/a69e83ec/attachment.htm>