<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p><br>
</p>
<br>
<div class="moz-cite-prefix">On 9/14/16 4:14 PM, Cantor, Scott
wrote:<br>
</div>
<blockquote cite="mid:1D1957CF-8078-4683-9B39-A543A2C3297F@osu.edu"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">1) at "fetch" time, you write the data to the cache. (In the batch ones this is once, at
init time.
</pre>
</blockquote>
<pre wrap="">
In batch, it's not just init, it's every load of a changed batch. Doesn't really change anything, just noting it.</pre>
</blockquote>
Good point, I misspoke. I'm too focused on "initing" right now...<br>
<br>
<blockquote cite="mid:1D1957CF-8078-4683-9B39-A543A2C3297F@osu.edu"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">2) at init time, you load (all) the cached data to make it "live".
</pre>
</blockquote>
<pre wrap="">
I'm not opposed, but a simple Predicate optionally injected to control which entities get loaded seems like it would cover all bases, but defaulting that way doesn't bother me.</pre>
</blockquote>
<br>
That seems pretty simple to add.<br>
<br>
<br>
<blockquote cite="mid:1D1957CF-8078-4683-9B39-A543A2C3297F@osu.edu"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">An initial concern with the preemptive load was the memory usage.
</pre>
</blockquote>
<pre wrap="">
More the init time, I would say. I don't think any amount of memory usage from this would come close to what the batch loads cost.</pre>
</blockquote>
<br>
I agree, the memory isn't close to the batch case. Init time -
yeah, I see your point. I still think it's probably preferable to
pay that cost up front rather than at request time. We could
instead theoretically load from the cache in a background thread at
init time, so it doesn't delay the startup. But there's some
synchronization complexity there, if you get requests in for
metadata before or while it's running for entityIDs that overlap,
and newer ones are resolved than the cache. That complexity is
evil, as it always is, and doesn't seem worth it to me personally.<br>
<br>
<blockquote cite="mid:1D1957CF-8078-4683-9B39-A543A2C3297F@osu.edu"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">So the shared code I have is for an interface for an XMLObject load/save manager. It is
based on the (simple) idea of loading and storing an XMLObject indexed under a String key.
</pre>
</blockquote>
<pre wrap="">
Slight danger there if we re-introduce the problem of writing things out as XMLObjects, and risk the serializer breaking signatures, the bug we just fixed in the SP. I think the IdP currently backs up via streaming the data it gets directly, not with an XML roundtrip.</pre>
</blockquote>
<br>
Yeah, that is a problem, and I already noted it in my personal
notes. Don't have an answer yet. In the abstract dynamic resolver,
the abstract fetch method the subclass must implement is defined
currently to return an XMLObject, not a byte[] like in the batch
resolvers. So we don't have the original data currently. It's not
really related to this new cache work per se, rather the original
dynamic design. The proximate reason was that all this was designed
around the HTTP case, and in the HTTP subclass I'm using an
HttpClient ResponseHandler<XMLObject> to simplify the
processing. So that percolated up to the abstract dynamic one.<br>
<br>
I'm not actually sure though if the roundtrip does cause a problem
in the Java code. It's certainly a strong possibility, but not
certain. <br>
<br>
If the roundtrip is a problem, I suppose we could formally deprecate
that method and have a new method used in preference that returns a
byte[]. A bit tricky to do in a non-breaking way in a minor rev,
though. The new method might have to be non-abstract, with a
default returning null, I think.<br>
<br>
I suppose another really crazy idea, not fully baked, is I could
cache the original byte[] from the HttpClient call on the
XMLObject's objectMetadata, and persist that to cache instead.
Hmmm.... <br>
<pre wrap="">
</pre>
<br>
<blockquote cite="mid:1D1957CF-8078-4683-9B39-A543A2C3297F@osu.edu"
type="cite">
<pre wrap="">There certainly is one for databases. </pre>
</blockquote>
Of course, duh. Brain fart.<br>
<br>
<br>
<blockquote cite="mid:1D1957CF-8078-4683-9B39-A543A2C3297F@osu.edu"
type="cite">
<blockquote type="cite">
<pre wrap=""> However I think it would be possible to have an impl of this load/save manager interface
that just delegates to a supplied StorageService, if that ever seems useful.
</pre>
</blockquote>
<pre wrap="">
I would prefer that we do that than ever build anything other than the file backed one, and yes, having additonal interfaces on top is fine, just like we do now for the replay cache, artifact map, session cache, etc. The complexity should be buried in those pieces, not in either the storage plugins or the code using those interfaces.</pre>
</blockquote>
<br>
I'm not opposed.<br>
<br>
However, I was just looking again at the StorageService API to
confirm viability. I think I missed something earlier and it's not
currently possible. I don't see mechanism on the SS API for
enumeration. At a minimum, a mechanism to list the storage keys
present in a context would be needed - which in turn could be used
to iterate the entries themselves. I assume there was some reason
for that omission? Probably some of the envisioned types weren't
amenable to enumeration? I guess we could potentially address with
a SS sub-interface that added enumeration method(s).<br>
<br>
That's unfortunately key (no pun intended) to what I proposed, since
you have to be able to iterate over all the cache entries in order
to do the load at init time.<br>
<br>
I *think* all the other load/save API methods can be implemented in
terms of SS primitives.<br>
<br>
</body>
</html>