<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0">
<base href="https://issues.shibboleth.net/jira">
<title>Message Title</title>
</head>
<body class="jira" style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 1.429">
<table id="background-table" cellpadding="0" cellspacing="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; background-color: #f5f5f5; border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt" bgcolor="#f5f5f5">
<!-- header here -->
<tbody>
<tr>
<td id="header-pattern-container" style="padding: 0px; border-collapse: collapse; padding: 10px 20px">
<table id="header-pattern" cellspacing="0" cellpadding="0" border="0" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tbody>
<tr>
<td id="header-avatar-image-container" valign="top" style="padding: 0px; border-collapse: collapse; vertical-align: top; width: 32px; padding-right: 8px" width="32"> <img id="header-avatar-image" class="image_fix" src="cid:jira-generated-image-avatar-e6f2180a-95f5-45e6-b0bc-f41bb2b24735" height="32" width="32" border="0" style="border-radius: 3px; vertical-align: top"> </td>
<td id="header-text-container" valign="middle" style="padding: 0px; border-collapse: collapse; vertical-align: middle; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 1px"> <a class="user-hover" rel="cantor.2@osu.edu" id="email_cantor.2@osu.edu" href="https://issues.shibboleth.net/jira/secure/ViewProfile.jspa?name=cantor.2%40osu.edu" style="color:#0052cc;; color: #3b73af; text-decoration: none">Scott Cantor</a> <strong>commented</strong> on <a href="https://issues.shibboleth.net/jira/browse/SSPCPP-926" style="color: #3b73af; text-decoration: none"><img src="cid:jira-generated-image-avatar-2760b80d-210d-43cc-bde9-a361a6776c7c" height="16" width="16" border="0" align="absmiddle" alt="Task"> SSPCPP-926</a> </td>
</tr>
</tbody>
</table> </td>
</tr>
<tr>
<td id="email-content-container" style="padding: 0px; border-collapse: collapse; padding: 0 20px">
<table id="email-content-table" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; border-spacing: 0; border-collapse: separate">
<tbody>
<tr>
<!-- there needs to be content in the cell for it to render in some clients -->
<td class="email-content-rounded-top mobile-expand" style="padding: 0px; border-collapse: collapse; color: #ffffff; padding: 0 15px 0 16px; height: 15px; background-color: #ffffff; border-left: 1px solid #cccccc; border-top: 1px solid #cccccc; border-right: 1px solid #cccccc; border-bottom: 0; border-top-right-radius: 5px; border-top-left-radius: 5px; height: 10px; line-height: 10px; padding: 0 15px 0 16px; mso-line-height-rule: exactly" height="10" bgcolor="#ffffff"> </td>
</tr>
<tr>
<td class="email-content-main mobile-expand " style="padding: 0px; border-collapse: collapse; border-left: 1px solid #cccccc; border-right: 1px solid #cccccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #ffffff" bgcolor="#ffffff">
<table class="page-title-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tbody>
<tr>
<td style="vertical-align: top;; padding: 0px; border-collapse: collapse; padding-right: 5px; font-size: 20px; line-height: 30px; mso-line-height-rule: exactly" class="page-title-pattern-header-container"> <span class="page-title-pattern-header" style="font-family: Arial, sans-serif; padding: 0; font-size: 20px; line-height: 30px; mso-text-raise: 2px; mso-line-height-rule: exactly; vertical-align: middle"> <a href="https://issues.shibboleth.net/jira/browse/SSPCPP-926" style="color: #3b73af; text-decoration: none">Re: Replace XML serialization in remoting layer</a> </span> </td>
</tr>
</tbody>
</table> </td>
</tr>
<tr>
<td id="text-paragraph-pattern-top" class="email-content-main mobile-expand comment-top-pattern" style="padding: 0px; border-collapse: collapse; border-left: 1px solid #cccccc; border-right: 1px solid #cccccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #ffffff; border-bottom: none; padding-bottom: 0" bgcolor="#ffffff">
<table class="text-paragraph-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 2px">
<tbody>
<tr>
<td class="text-paragraph-pattern-container mobile-resize-text " style="padding: 0px; border-collapse: collapse; padding: 0 0 10px 0"> <p style="margin: 10px 0 0 0; margin-top: 0">After much (seriously) research, I have the beginnings of a handle on the mess that is the web, and surprise, it's completely broken in theory, just not in practice.</p> <p style="margin: 10px 0 0 0">The URL in the request line is not allowed to have characters above 0x7F in them without percent-encoding. What is NOT specified or even knowable is that actual character encoding of the percent-encoded data. Using %80 and above can't be deterministically turned back into a code point in a Java String, period. What Jetty does at least is defer that step to the handlers that resolve content, and the default behavior is to attempt to transcode the resulting byte sequence into UTF-8 and if that fails, into ISO-8859-1. Any other desired outcome has to be addressed with plugged in code.</p> <p style="margin: 10px 0 0 0">Using IRIs falls into the first branch because actual Unicode inside a URL is defined for HTTP by forcing it into UTF-8 and then percent-encoding, so you get the right answer in at least Jetty.</p> <p style="margin: 10px 0 0 0">For the SP's purposes, these issues don't really matter. The encoded URL is essentially opaque to the software, and the SAML endpoints will only be compared on the basis of the encoded value, never a decoded one (e.g. Recipient or Destination checks). Thus, as long as the URL fetched from Apache/IIS is encoded and passed along as is, nothing breaks. If a web server were to do something terrible like decode the URL first and supply it in that form, you'd be dead meat.</p> <p style="margin: 10px 0 0 0">The HTTP header situation is a different story because it is NOT illegal, merely a bad idea, to embed characters above 7F in the values.</p> <p style="margin: 10px 0 0 0">What Jetty does with header parsing is treat each byte as a discrete character and they're promoted into 16-bit Unicode code points via (char) (0xff & byte). As a result, it is impossible to successfully process a UTF-8 "literal" multi-byte sequence in a header value, because it will be ionized into two or more Unicode characters, which is not the same string at all. If you knew that and accounted for it, you could play application games with the Java String and turn it back into bytes and go over it again, I suppose.</p> <p style="margin: 10px 0 0 0">This is again left to applications to deal with when it comes to cookies, for example. What you get is the 16-bit String and it's your problem.</p> <p style="margin: 10px 0 0 0">The SP's issue is that it can therefore definitely not treat the headers as a whole as "knowable" because Jetty has in effect corrupted them. It can only operate safely on its own data, e.g. cookies it creates itself and handles defensively. Of course, the SP now percent-encodes anything risky already, it's just that it's not really required to by the RFC.</p> <p style="margin: 10px 0 0 0">The upshot is that I think it's doable, but it has to be case by case and done with some care. The remoting I did can work since it safely turns any bytes into ASCII but being that this isn't Jetty, I would need to perform some kind of similar heuristics to get Java Strings back out, and the eventual processing in Java has to be very careful to understand which data elements are "knowably" non-corrupted and which are not. If we hit a case where something is not knowably safe but has to be operated on, it will require great care. I'm not aware of anything like that offhand, but I don't know for certain.</p> </td>
</tr>
</tbody>
</table> </td>
</tr>
<tr>
<td class="email-content-main mobile-expand " style="padding: 0px; border-collapse: collapse; border-left: 1px solid #cccccc; border-right: 1px solid #cccccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #ffffff" bgcolor="#ffffff">
<table id="actions-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 1px">
<tbody>
<tr>
<td id="actions-pattern-container" valign="middle" style="padding: 0px; border-collapse: collapse; padding: 10px 0 10px 24px; vertical-align: middle; padding-left: 0">
<table align="left" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tbody>
<tr>
<td class="actions-pattern-action-icon-container" style="padding: 0px; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 0; vertical-align: middle"> <a href="https://issues.shibboleth.net/jira/browse/SSPCPP-926#add-comment" target="_blank" title="Add Comment" style="color: #3b73af; text-decoration: none"> <img class="actions-pattern-action-icon-image" src="cid:jira-generated-image-static-comment-icon-5f912404-1998-427a-8ee1-b8b8d2c502fa" alt="Add Comment" title="Add Comment" height="16" width="16" border="0" style="vertical-align: middle"> </a> </td>
<td class="actions-pattern-action-text-container" style="padding: 0px; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 4px; padding-left: 5px"> <a href="https://issues.shibboleth.net/jira/browse/SSPCPP-926#add-comment" target="_blank" title="Add Comment" style="color: #3b73af; text-decoration: none">Add Comment</a> </td>
</tr>
</tbody>
</table> </td>
</tr>
</tbody>
</table> </td>
</tr>
<!-- there needs to be content in the cell for it to render in some clients -->
<tr>
<td class="email-content-rounded-bottom mobile-expand" style="padding: 0px; border-collapse: collapse; color: #ffffff; padding: 0 15px 0 16px; height: 5px; line-height: 5px; background-color: #ffffff; border-top: 0; border-left: 1px solid #cccccc; border-bottom: 1px solid #cccccc; border-right: 1px solid #cccccc; border-bottom-right-radius: 5px; border-bottom-left-radius: 5px; mso-line-height-rule: exactly" height="5" bgcolor="#ffffff"> </td>
</tr>
</tbody>
</table> </td>
</tr>
<tr>
<td id="footer-pattern" style="padding: 0px; border-collapse: collapse; padding: 12px 20px">
<table id="footer-pattern-container" cellspacing="0" cellpadding="0" border="0" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tbody>
<tr>
<td id="footer-pattern-text" class="mobile-resize-text" width="100%" style="padding: 0px; border-collapse: collapse; color: #999999; font-size: 12px; line-height: 18px; font-family: Arial, sans-serif; mso-line-height-rule: exactly; mso-text-raise: 2px"> This message was sent by Atlassian Jira <span id="footer-build-information">(v8.5.4#805004-<span title="0444eab799707f9ad7b248d69f858774aadfd250" data-commit-id="0444eab799707f9ad7b248d69f858774aadfd250}">sha1:0444eab</span>)</span> </td>
<td id="footer-pattern-logo-desktop-container" valign="top" style="padding: 0px; border-collapse: collapse; padding-left: 20px; vertical-align: top">
<table style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tbody>
<tr>
<td id="footer-pattern-logo-desktop-padding" style="padding: 0px; border-collapse: collapse; padding-top: 3px"> <img id="footer-pattern-logo-desktop" src="https://issues.shibboleth.net/jira/images/mail/atlassian-email-logo.png" alt="Atlassian logo" title="Atlassian logo" width="191" height="24" class="image_fix"> </td>
</tr>
</tbody>
</table> </td>
</tr>
</tbody>
</table> </td>
</tr>
</tbody>
</table>
</body>
</html>