AW: Umlaut (Ü, Ä, Ö) in attribute resolution not UTF-8 encoded

[Grüter, Philip]

Hi Tim,

my JDBC connection string is already enforcing the UTF-8 character set:
jdbc:mysql://REDACTED IP:6033/userdb?useUnicode=true&characterEncoding=UTF-8
And the database and table is also in UTF-8.
That’s why I’m confused about the wrong encoding.

Greetings
Philip

Von: users <users-bounces at shibboleth.net> Im Auftrag von Timothy Spear via users
Gesendet: Donnerstag, 13. Februar 2025 19:09
An: Shib Users <users at shibboleth.net>
Cc: Timothy Spear <n614cd at gmail.com>
Betreff: Re: Umlaut (Ü, Ä, Ö) in attribute resolution not UTF-8 encoded

Philip,

That suggests the JDBC connection is using the default latin1 character set.
Try forcing the character set assignment on the JDBC URL connection string.

Tim


On Thu, Feb 13, 2025 at 12:58 PM Grüter, Philip <philip.grueter at secunet.com<mailto:philip.grueter at secunet.com>> wrote:
Hi Scott and Gianluca,

I don't really think it has something to do with my terminal or even the output of the aacli.sh tool.
The IDP is connected via SAML2 to a moodle system, which is also getting the wrong umlauts.
Also, the log is also displaying the wrong encoding. When I use the same terminal to connect to the MySQL Database, it displays the Ü correctly.

Also, I got it working, but I don't consider my current solution a good one. I changed the SQL Query in my attribute-resolver config to:
SELECT email,forename,CONVERT(surname USING utf8) AS surname FROM moodle2_auth WHERE username='$resolutionContext.principal'
This outputs the correct result. So in my opinion, the wrong encoding has something to do with the gathering of the data.

- Philip

-----Ursprüngliche Nachricht-----
Von: users <users-bounces at shibboleth.net<mailto:users-bounces at shibboleth.net>> Im Auftrag von Cantor, Scott via users
Gesendet: Donnerstag, 13. Februar 2025 17:20
An: Shib Users <users at shibboleth.net<mailto:users at shibboleth.net>>
Cc: Cantor, Scott <cantor.2 at osu.edu<mailto:cantor.2 at osu.edu>>
Betreff: Re: Umlaut (Ü, Ä, Ö) in attribute resolution not UTF-8 encoded

> Are you sure it is not a problem related to your terminal ?

Without going that far, I will say it's incredibly hard to test and debug this sort of thing because of all the issues with log files, terminal display, etc. It's easy to mix up where the issue is.

But I wouldn't swear the output logic for the view in that flow isn't wrong somewhere. I would suggest as one thing to try the --saml2 switch and see if the XML seems to be right.

I'm in the middle of working out some of the Unicode issues with the SP redesign, and I truly believe aliens invading and mandating a new language would be simpler for the world than handling Unicode in C++.

-- Scott


--
For Consortium Member technical support, see https://shibboleth.atlassian.net/wiki/x/ZYEpPw
To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net<mailto:users-unsubscribe at shibboleth.net>
--
For Consortium Member technical support, see https://shibboleth.atlassian.net/wiki/x/ZYEpPw
To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net<mailto:users-unsubscribe at shibboleth.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20250213/5cd77951/attachment.htm>