Configuring a file-backed HTTP resource for attribute filters

[Brent Putman]

On 7/21/15 12:51 PM, Cantor, Scott wrote:
>
> Ok, admittedly I interpreted "no backup file yet" as equivalent to the file cache not being populated either, which aren't necessarily equivalent. So that is the first question to ask, and one can always flush that directory.

Looking at the code, the backup file referenced is definitely ours. 
The root issue happens before that, something is going wrong with the
FileBacked- class call to super.getInputStream() (the regular
HttpResource).

What would be very informative is to see:

1) the DEBUG output from
net.shibboleth.ext.spring.resource.HTTPResource.  That will report what
the caching status of the file as reported by HttpClient.

2) the ERROR level output reporting exactly what HTTP status code was
effectively returned from the client call.  My previous testing and
understanding what that if the HttpClient served the response from
cache, then you get back a 200, but perhaps that's not always the
case.  The superclass getInputStream() has the code block:

        if (httpStatusCode != HttpStatus.SC_OK) {
            String errMsg =
                    "Non-ok status code " + httpStatusCode + " returned
from remote resource " + resourceURL;
            log.error(errMsg);
            throw new IOException(errMsg);
        }

I'd like to confirm that that is what's happening to throw the
IOException caught by the FileBacked- subclass, and what the exact code
is.  Perhaps we are misusing the HttpClient there, i.e. maybe codes
other than SC_OK should be treated as success.  Just guessing. Or maybe
there really is a bug in HttpClient somewhere.

Also interesting would be DEBUG output from org.apache.http.  (Warning:
that will be a lot, since org.apache.http.wire does a wire level dump
of all the traffic).

Another thing to look at might be the max cache entry size:  We have
for the FileCachingHttpClient default of:

p:maxCacheEntrySize="%{idp.httpclient.filecaching.maxCacheEntrySize:10485760}

so 10MB.  Could the resource being fetched here be bigger than that? I
have thought that if the resource is bigger, it simply isn't cached,
but perhaps it's more complicated and/or there is a bug around that
(ours or theirs).

Finally, might be informative to see if instead switching to the
shibboleth.MemoryCachingHttpClient resolves the issue.  Might hint
where the problem is.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20150721/a71f521a/attachment.html>