Performing conditional gets with the metadata aggregator

[Tom Scavo]

On Thu, Apr 6, 2017 at 1:04 PM, Wessel, Keith <kwessel at illinois.edu> wrote:
> Finally getting back to this, and Tom, the bash library you mentioned looks like it'll do exactly what we need. Thank you!
>
> One question: it looks like there are some hard-coded refrences to /tmp in ilb/md_tools.sh on lines 348, 469, and 584. If you're like us and have a very tiny /tmp and are trying to fetch in the InCommon metadata in full, this quickly runs out of disk space.

No, there aren't any hardcoded references to /tmp in the code but
there are references to a semi-standard environment variable called
TMPDIR. Not all Linux flavors provide that environment variable so the
calling script must check for that and create a temporary directory if
one doesn't already exist.

I'll document this inline and commit an update later today.

Thanks Keith.

Tom

> -----Original Message-----
> From: users [mailto:users-bounces at shibboleth.net] On Behalf Of Tom Scavo
> Sent: Tuesday, March 21, 2017 9:31 AM
> To: Shib Users <users at shibboleth.net>
> Subject: Re: Performing conditional gets with the metadata aggregator
>
> On Tue, Mar 21, 2017 at 9:58 AM, Wessel, Keith <kwessel at illinois.edu> wrote:
>> Sadly, yes, we’re just running the cli every five minutes to produce a
>> hybrid document from the different sources. If there’s a better approach to
>> produce the same end result, but with caching to enable conditional gets,
>> I’m not opposed.
>
> Not sure if this meets your needs but this bash library includes a
> conditional GET client:
> https://github.internet2.edu/InCommon/bash-library
>
> Tom
>
>> From: users [mailto:users-bounces at shibboleth.net] On Behalf Of Ian Young
>> Sent: Tuesday, March 21, 2017 7:03 AM
>> To: Shib Users <users at shibboleth.net>
>> Subject: Re: Performing conditional gets with the metadata aggregator
>>
>>
>>
>>
>>
>> On 20 Mar 2017, at 21:31, Wessel, Keith <kwessel at illinois.edu> wrote:
>>
>>
>>
>> So, with that said, Ian, what would the Spring code look like to perform a
>> conditional get in comparison to what's in the IdP?
>>
>>
>>
>> I took a quick look at this today. One thing to watch out for is that the
>> java-support cacheing client builders only cache documents up to a given
>> size, and that size is in both cases too low for the InCommon metadata
>> specifically. There are properties for these things, though.
>>
>>
>>
>> The other question, though, goes back to your original post:
>>
>>
>>
>> Wondered how one would configure the MDA to do a conditional get of metadata
>> in a bean using the file backed HTTP resource class.
>>
>>
>>
>> How are you using the MDA in your application? Are you using the provided
>> CLI interface, the mdq-server code or some other framework to invoke the MDA
>> stages?
>>
>>
>>
>> I ask because the cacheing provided by the java-support http client builders
>> doesn't seem at first glance to persist the document metadata, just its
>> body. That means that if you're just running the CLI once every five
>> minutes, you're not going to get any cacheing. Just want to make sure I am
>> not chasing down a blind alley here.
>>
>>
>>
>> Cheers,
>>
>>
>>     -- Ian
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>> To unsubscribe from this list send an email to
>> users-unsubscribe at shibboleth.net
> --
> To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net
> --
> To unsubscribe from this list send an email to users-unsubscribe at shibboleth.net