StorageService API

[Marvin S. Addison]

> Can you look at the replay cache implementation and try and explain
> why you think it's awkward there?

ReplayCache is a fairly different use case; there it's natural to deal 
with strings and multiple contexts. In the case of CAS tickets it's most 
natural to deal with a Ticket object that has a creation date, ticket 
identifier string, and expiration date where all tickets are written to 
a single context. It's only awkward by comparison:

storageService.create(ticket);

vs

storageService.createString("http://jasig.org/cas/tickets", 
ticket.getId(), ticket.getId(), ticket.getExpirationDate());

Not a problem, just more work than I'd like.

> My experience with this API is that simple things are simple and hard
> things are possible, which is pretty much the sweet spot for me.

It certainly meets that critieria. Most of what I suggested should be 
considered seasoning, which is always a matter of personal taste.

>> It may be helpful if not necessary to provide a registry for
>> context names.
>
> Certainly doable, but in practice it's not been a major concern
> because anything that's not core code ought to use package-syle or
> URL-style contexts. Have to think about whether it's worthwhile to
> add a registry.

I was unaware that use of URIs was recommended, and as long as that's 
documented clearly it should be sufficient.

> Basically, I would be terrified of using Hibernate, to use one
> obvious example.

I think it's wise to carefully evaluate complex technologies like 
Hibernate. They're not free or magic.

> To do an object-based store is not at all simpler for me to do, it's
> much, much more work. It's easier for the callers of the API, but the
> amount of code being saved there is not large in amount or
> complexity. So it was a conscious trade-off.

Fair enough.

> Yes, it assumes BLOB (or more precisely text) storage in a database.
> Could you elaborate about why  that is a headache? I can't see how
> requiring custom table schema in a database could be seen as a better
> option, which is the only alternative to object storage I'm aware of,
> but I need to understand your experience here.

The problems are very much a function of the database platform, but I've 
had experience with PostgreSQL, Oracle, and MySQL (in descending order). 
On PostgreSQL we had problems with orphaned large objects; one 
transaction would delete the object and for some reason other 
transactions would not get notice and the others would error out. It's a 
recoverable error, but not a very graceful one. Generally the 
maintenance around large objects was a pain on PG.

The second class of issues is deadlocks, and we have only suggestive 
data that those were exacerbated by LOB storage. I would be more 
confident in saying that a mixture of OLTP workload (heavy read/write 
concurrency), foreign keys, and large objects tends to exacerbate 
deadlocks. The severity of the problems varied dramatically with 
platform; the rate of deadlocks on PG was orders of magnitude higher 
than Oracle.

The last issue is a system issue of capacity planning and platform 
limits. LOB data types do not declare storage limits like other database 
data types, so it's much harder to measure and estimate storage. I 
believe most platforms have tools for tracking down LOB data, but in 
many cases it's non trivial, and the application folks troubleshooting 
database issues tended not to be DBAs in my experience.

> I'm not trying to be difficult, really, but right now we have a lot
> of stuff to build, and I'm really trying to avoid true object
> persistence being one of those things, unless we have a clear offer
> to build it.

Totally understand. With tools like Hibernate it's not terribly 
difficult to provide this for RDBMS backend, but it is work. I guess 
it's just a matter of the extent to which you want to support all 
possible backends with a single API. It's perfectly reasonable to say 
database is an edge case and the existing API will support it in some 
manner even if it isn't the most natural.

M