Troubleshooting the "Unable to decode" (IdP 3.3)

[Brent Putman]

On 8/31/17 3:41 PM, Cantor, Scott wrote:
>> Now that I know where it is coming from and why, are there any constraints I can put in place at the IdP to reject these requests,
>> or should I make those restrictions at the web server?
> Web server.

Probably the simplest thing to do first is see if their crawler
respects a web server's robots.txt.  You can google for plenty of
references of how to set that up, if you don't know already.  You can
block server paths and many other things with that.

Actually looks like they do.  From following the URL in the User-Agent
string:

"Facebot is Facebook's web crawling robot that helps improve
advertising performance. Facebot is designed to be polite. It attempts
to access each web server no more than once every few seconds, in line
with industry standards, and will respect your robots.txt settings."
 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20170831/65426421/attachment-0001.html>