Configuring a file-backed HTTP resource for attribute filters

[Wessel, Keith]

Thanks, Brent, especially for the lead to the file cache dir.

I see that it is empty after shutdown, and after start-up the files are pulled down and stored. So, the 304 message I was seeing in the webserver log from the server hosting the attribute filter files must have been a subsequent fetch, though I didn't see that in the logs. I can look again. At any rate, I'm getting valid files in the cache dir; they're just not getting created in the IDP conf dir.

And yes, when I mentioned manually pulling them down, I meant wgetting the files and dropping them in place where the IDP would store them in the conf dir before starting the IDP. Interestingly when I do this then start the IDP, it does update these files, or at least update their timestamps. I did this before noon and left the IDP running, and the attribute filter files in the conf dir were having their timestamps updated every 15 minutes as appropriate.

So, with all this said, which of the classes do you want me to provide debug-level logging for in Gira?

Keith


From: users [mailto:users-bounces at shibboleth.net] On Behalf Of Brent Putman
Sent: Tuesday, July 21, 2015 2:33 PM
To: users at shibboleth.net
Subject: Re: Configuring a file-backed HTTP resource for attribute filters


On 7/21/15 2:46 PM, Wessel, Keith wrote:



Where shall we start? First, this file cache you referred to. That's an in-memory cache?

Well, based on looking at the Spring XML Rod gave you earlier in the thread, it looked like you were probably using shibboleth.FileCachingHttpClient.  So that's a filesystem-based cache.  The actual cache directory for that is defined in services.properties as 'idp.httpclient.filecaching.cacheDirectory'.  It defaults to:  %{idp.home}/tmp/httpClientCache.  So what you might do is shutdown, clear out that directory, restart and see what's there.



Something that gets cleaned out by a restart? 'Cause this error occurs on jetty/IDP start-up.

As I mentioned later in the thread, I think that an IdP shutdown *should* be clearing this directory out.  If you do the above to get it to a known good state, you should see stuff there.  If you then shut down again, I think you should see it empty again.

So my initial guess was probably wrong.  For that and another reason, I don't think you should effectively see HttpClient-level caching across restarts.

However, what you might be seeing is: First, a call to check whether the resource exists() happen earlier (don't know for sure, we need to check), and then a second call to actually get the resource.  The second would be served from cache, which was populated by the first call.



So, if it's in-memory, I should be starting with a clean cache and without the local backin file existing.

The backing file is something else.  That's *our* (OpenSAML/IdP) backup file for the HTTP resource.  AFAIK, that *should* persist across restarts.



BTW, when I do suck downthe file manually before starting the IDP, the warnings go away and the IDP happily accepts the 304 return code.


Sorry, what do you mean by this exactly?  How are you sucking it down outside of the IdP?  Do you mean you are manually downloading it and storing it in the backing file location?  If so, then I understand and that's what's supposed to happen (although obviously you're not supposed to have to do that...)

Also, exactly where are you seeing this 304 return code?  Can you post either here or in Jira the exact log message and some of surrounding log messages, for context?



Shall I try turning up logging as Brent suggested and see what that tells me? And perhaps, if there's nothing obvious in the output, put that output in Gira?




That will certainly tell us more about what's going on. So please do so if/when you have the time.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20150721/06e79f1f/attachment.html>