A note about mod_shib2 and mod_http2

[Cantor, Scott]

On 8/18/17, 9:48 PM, "users on behalf of Jacob Lundberg" <users-bounces at shibboleth.net on behalf of jacob at collegenet.com> wrote:

> Yes, very nearly.  I think I misunderstood what you were telling me to
> restart.  You mean we should restart the httpd every time we update the
> shibd's reloadable configuration, is that right?

No, not really. They share the same file, they are not the same configuration. If you changed the file and restarted httpd and left shibd to reload the change itself, you'd be in basically the same end state but with no locking bug. If you didn't restart httpd and kept it from reloading the file, it would depend on what the change actually was and whether it was something that impacted that half of the system.

> However, this doesn't really make much difference because to restart the
> httpd we have to take it out of service, which takes five to ten minutes
> and another five to ten minutes to return it to service.  There are a
> *lot* of virtual servers to instruct our load balancer to unbind and
> rebind.  This is why we use graceful reloads...

I guess at the end of the day the real issue is that you don't want to be changing the config that often to begin with. That suggests something else is not really being approached in a manner that really comports with the design of the software, which I'm guessing is perhaps to do with metadata, and Apache doesn't need to know about that.

> We're probably downloading metadata from more sources than most.

Unless all those sources are hosting signed, frequently-expiring metadata files, what you're doing is very likely problematic for security anyway. The trust model is invalidated in a variety of subtle ways. That's why InCommon exists and why it has value. So I suspect you have some dangerous behavior there. Typically, what you have to do to handle systems not federating safely is to do it manually, which you do by creating a separate aggregate file that may change a lot, but is still just one extra source. And you shouldn't rely on remote metadata when it's being maintained and offered in unsafe ways (e.g. without expiring in days or weeks and being constantly re-signed).

But the important point for this discussion is that mod_shib does not load the metadata, that's shibd only. You could add metadata left and right and never have to restart Apache.
 
> I'm afraid I'm not entirely following you on this suggestion.  Do you
> mean we should make a configuration change to our MetadataProviders?
> What change are you recommending?  I just bumbled around in the wiki a
> bit but so far I haven't found anything that really jumps out.

It's the verifyBackup flag on the signature filter, but if you're not dealing with large metadata files, it doesn't matter (though you could certainly eliminate that InCommon overhead.

Anyway, disabling the reloads to keep mod_shib from noticing the updated file is likely one fix. That requires some obnoxious trickery involving one of the top level mod_shib Apache commands that I would have to dig up. I haven't done it myself but I know others have.

But in general I would say the real problem is one of strategy. You just can't really handle that many metadata sources (or get that many to actually operate with any semblance of safety). You can't add separate static metadata sources for every IdP, that just doesn't scale.

As a possible longer term move, you could possibly look at converting some of this to rely on the Dynamic plugin. If you had a separate set of technical bits pulling in that metadata, however dangerously, and sticking it behind an Apache server as a lookup service by entityID, the Dynamic plugin would turn all those sources into one on-demand source with no startup cost. That is obviously where all this is headed anyway.

It's even possible to be a bit tricky and do it all with files. There's a hacky feature in the 2.6 code that allows it to take an entityID and hash it into a filename to lookup on disk and load that via the Dynamic plugin. So you can actually drop ugly looking filenames into a folder to auto-add an IdP. I haven't seen anybody use it, but this is kind of the scenario it was built for.

-- Scott