Re: Umlaut (Ü, Ä, Ö) in attribute resolution not UTF-8 encoded
Gianluca Amato
gianluca.amato.74 at gmail.com
Thu Feb 13 15:40:57 UTC 2025
Are you sure it is not a problem related to your terminal ?
>From the behaviour you are describing, it seems that your terminal is
incorrectly set to Latin-1 encoding, so that when the sequence of bytes
0xC3 0xBC is displayed on the screen, it produces ü instead of ü.
--gianluca
On Thu, Feb 13, 2025 at 10:42 AM Grüter, Philip <philip.grueter at secunet.com>
wrote:
> Hi all,
>
>
>
> I currently have a problem with Umlauts (Ü, Ä, Ö) with my Shibboleth
> instance.
>
> The output I get (with aacli.sh and over a normal SAML2 session) is this:
>
> {
>
> "name": "sn",
>
> "values": [
>
> "Müller"
>
> ]
>
> },
>
>
>
> “ü” is the corresponding Latin-1 encoding for Ü, but I can't find where
> it's coming from.
>
>
>
> I'm still running 4.3.0, we are in the process of upgrading. The server
> uses Jetty9 and java-11-amazon-corretto-jdk.
>
>
>
> We pull attributes out of a MySQL database during attribute resolution.
> The database is encoded with UTF-8.
>
> The "table" we access, is a view, which is creted from two different
> tables. Both tables and the view is also set to UTF-8.
>
>
>
> These are my configs and the Log output:
>
>
>
> gloabl.xml:
>
> <bean id="shibboleth.MySQLExtraDataSource"
>
> class="%{mysqlextra.class}"
>
> p:driverClassName="com.mysql.jdbc.Driver"
>
> p:jdbcUrl="%{mysqlextra.url}"
>
> p:username="%{mysqlextra.username}"
>
> p:password="%{mysqlextra.password}" />
>
>
>
>
>
> idp.properties DB Connection:
>
> mysqlextra.class = com.zaxxer.hikari.HikariDataSource
>
> mysqlextra.url = jdbc:mysql://REDACTED
> IP:6033/userdb?useUnicode=true&characterEncoding=UTF-8
>
>
>
>
>
> attribute-resolver.xml:
>
>
>
> <AttributeDefinition id="sn" xsi:type="Template">
>
> <InputDataConnector ref="mysqlextra" attributeNames="db_surname" />
>
> <Template>
>
> <![CDATA[
>
> ${db_surname}
>
> ]]>
>
> </Template>
>
> </AttributeDefinition>
>
>
>
> …
>
>
>
> <DataConnector xsi:type="RelationalDatabase" id="mysqlextra">
>
>
> <BeanManagedConnection>shibboleth.MySQLExtraDataSource</BeanManagedConnection>
>
> <QueryTemplate>
>
> <![CDATA[
>
> SELECT email,forename,surname from moodle2_auth where
> username='$resolutionContext.principal'
>
> ]]>
>
> </QueryTemplate>
>
> <Column columnName="email" attributeID="db_mail" />
>
> <Column columnName="forename" attributeID="db_forename" />
>
> <Column columnName="surname" attributeID="db_surname" />
>
> </DataConnector>
>
>
>
>
>
> Debug Log Output after requesting Attributes with the aacli.sh Utility:
>
> 2025-02-13 09:56:15,701 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Initiating attribute
> resolution with label: admin/resolvertest
>
> 2025-02-13 09:56:15,701 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Attempting to resolve the
> following attribute definitions [uid, givenName, sn, mail]
>
> 2025-02-13 09:56:15,701 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Resolving dependencies
> for 'uid'
>
> 2025-02-13 09:56:15,701 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Finished resolving
> dependencies for 'uid'
>
> 2025-02-13 09:56:15,702 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.AbstractAttributeDefinition:?] -
> Attribute Definition 'uid': produced an attribute with the following values
> [StringAttributeValue{value=REDACTED USER ID}]
>
> 2025-02-13 09:56:15,704 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Attribute definition
> 'uid' produced an attribute with 1 values
>
> 2025-02-13 09:56:15,706 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Resolving dependencies
> for 'givenName'
>
> 2025-02-13 09:56:15,706 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Resolving dependencies
> for 'mysqlextra'
>
> 2025-02-13 09:56:15,706 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Finished resolving
> dependencies for 'mysqlextra'
>
> 2025-02-13 09:56:15,706 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Resolving data connector
> mysqlextra
>
> 2025-02-13 09:56:15,717 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.dc.rdbms.TemplatedExecutableStatementBuilder:?]
> - Template text SELECT email,forename,surname from moodle2_auth where
> username='$resolutionContext.principal' yields SELECT
> email,forename,surname from moodle2_auth where username='REDACTED USER ID'
>
> 2025-02-13 09:56:15,758 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.dc.rdbms.StringResultMappingStrategy:?]
> - Remapping column email to db_mail
>
> 2025-02-13 09:56:15,758 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.dc.rdbms.StringResultMappingStrategy:?]
> - Remapping column forename to db_forename
>
> 2025-02-13 09:56:15,759 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.dc.rdbms.StringResultMappingStrategy:?]
> - Remapping column surname to db_surname
>
> 2025-02-13 09:56:15,759 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.AbstractDataConnector:?] - Data
> Connector 'mysqlextra': produced the following 3 attributes during
> resolution [db_surname, db_mail, db_forename]
>
> 2025-02-13 09:56:15,759 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.AbstractDataConnector:?] - Data
> Connector 'mysqlextra': Attribute 'db_surname': Values
> '[StringAttributeValue{value=Müller}]'
>
> 2025-02-13 09:56:15,759 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.AbstractDataConnector:?] - Data
> Connector 'mysqlextra': Attribute 'db_mail': Values
> '[StringAttributeValue{value=REDACTED USER MAIL}]'
>
> 2025-02-13 09:56:15,759 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.AbstractDataConnector:?] - Data
> Connector 'mysqlextra': Attribute 'db_forename': Values
> '[StringAttributeValue{value=REDACTED USER NAME}]'
>
> 2025-02-13 09:56:15,760 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Data connector
> 'mysqlextra' resolved the following attributes: [db_surname, db_mail,
> db_forename]
>
> 2025-02-13 09:56:15,762 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Finished resolving
> dependencies for 'givenName'
>
> 2025-02-13 09:56:15,763 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?]
> - Attribute Definition 'givenName': Determining value 1
>
> 2025-02-13 09:56:15,764 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?]
> - Attribute Definition 'givenName': Adding value 'REDACTED USER NAME' for
> attribute 'db_forename' to the template context
>
> 2025-02-13 09:56:15,764 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?]
> - Attribute Definition 'givenName': Evaluating template
>
> 2025-02-13 09:56:15,766 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?]
> - Attribute Definition 'givenName': Result of template evaluating was
> 'REDACTED USER NAME'
>
> 2025-02-13 09:56:15,766 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.AbstractAttributeDefinition:?] -
> Attribute Definition 'givenName': produced an attribute with the following
> values [StringAttributeValue{value=REDACTED USER NAME}]
>
> 2025-02-13 09:56:15,766 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Attribute definition
> 'givenName' produced an attribute with 1 values
>
> 2025-02-13 09:56:15,766 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Resolving dependencies
> for 'sn'
>
> 2025-02-13 09:56:15,766 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Finished resolving
> dependencies for 'sn'
>
> 2025-02-13 09:56:15,767 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?]
> - Attribute Definition 'sn': Determining value 1
>
> 2025-02-13 09:56:15,767 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?]
> - Attribute Definition 'sn': Adding value 'Müller' for attribute
> 'db_surname' to the template context
>
> 2025-02-13 09:56:15,767 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?]
> - Attribute Definition 'sn': Evaluating template
>
> 2025-02-13 09:56:15,768 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.ad.impl.TemplateAttributeDefinition:?]
> - Attribute Definition 'sn': Result of template evaluating was 'Müller'
>
> 2025-02-13 09:56:15,768 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.AbstractAttributeDefinition:?] -
> Attribute Definition 'sn': produced an attribute with the following values
> [StringAttributeValue{value=Müller}]
>
> 2025-02-13 09:56:15,768 - 127.0.0.1 - DEBUG
> [net.shibboleth.idp.attribute.resolver.impl.AttributeResolverImpl:?] -
> Attribute Resolver 'ShibbolethAttributeResolver': Attribute definition 'sn'
> produced an attribute with 1 values
>
>
>
>
>
> Things I tried:
>
> - Adding "?useUnicode=true&characterEncoding=UTF-8" to the jdbc URL
>
> - Changing locales with ISO-8859-1 to UTF-8 in /etc/jetty9/webdefault.xml
>
>
>
> Does anyone have any other ideas on things I can try to fix this?
>
>
>
> Greetings,
>
> Philip
> --
> For Consortium Member technical support, see
> https://shibboleth.atlassian.net/wiki/x/ZYEpPw
> To unsubscribe from this list send an email to
> users-unsubscribe at shibboleth.net
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20250213/787972e5/attachment.htm>
More information about the users
mailing list