Proxying server setup help

[Peter Schober]

* Antony den Dulk <antony at selestiasolutions.com> [2016-01-20 22:45]:
> > In addition to that thousands of vhosts existed (foo.example.org,
> > bar.example.org, etc.) which usually would route all request URIs to a
> > single backend (i.e., everything in a non-www.example.org vhost would
> > be on the same server; though foo.example.org/bar could still have
> > been routed to a different server, account and file system directory
> > than any of the other content or paths).
> 
> I am not sure I understand what you are saying here. Is the IP address
> of foo.example.org the same as www.example.org? If so, then the server
> should know that foo.example.org should be reversed proxied to the
> same server as www.example.org/foo. Is that what you are describing
> here?

(IP addresses don't factor into any of this, and all the HTTP client
sees is the proxy's v4 and v6 address.)

Either way, paths need to remain intact, no matter the amount of
proxying rewriting going on. (I.e., what Scott said.)

My example is a bit convoluted I guess since for legacy reasons (back
in the day, when there was one huge server handling www.example.org)
everything was on paths / request URIs on that one vhost
e.g. www.example.org/department-A/, www.example.org/project-B/ etc.)
That amouns to lots and lots of resources being publishing on that one
vhost. So in the redesign (scaling out) only content from that
specific vhost was spread over several backend web servers.
All other content from all other vhosts usually lives on one (and only
one) backend web server. E.g. foo.example.org/ and foo.example.org/bar
and foo.example.org/baz all are being proxied to the same web server,
which could at one time be server05 and 10 min later server75.

That accounts for part of the complexity in that setup, otherwise you
could do all this without ApplicationOverride, I think.

But all else being equal (or different, for that matter) the backend
web server needs to know the request URI / path. If that's lost,
there's info missing.

If you can restore the info (based on reversible mappings of paths to
vhosts and back) then that's your choice, but as Scott said it may not
play well with the Shib SP.

> > To handle the www.example.org case, where depending on the request URI
> > a different backend web server would process the request -- which also
> > means SAML protocol messages would have to be routed to the right
> > backend, and so the couldn't be all using ^/Shibboleth.sso as
> > handlerURL -- we ended up exposing the backend via distinct publicly
> > accessible handlerURLs, i.e. all backend web server shared the same
> > ApplicationDefaults/@entityID for the www.example.org vhost (and key
> > material for that SP), plus each backend web server additionally had
> > its own ApplicationOverride, with its own entityID and handlerURL and
> > key material.
> > The individual handlerURLs encoded the "id" of each backend, say 42,
> > and the front end proxy statically routed those requests to the
> > specifc backend. E.g. /webserver42/Shibboleth.sso/... would be routed
> > to websrv42.example.org for handling of www.example.org/vogon/poetry
> 
> Do you happen to have an example of this? I can then see if I can
> adapt it to my situation.

Not sure what you're asking. The above is the example.

In the simple case of only proxying content based on the originally
requested vhost (i.e., all requests for foo.example.org go to
webserver42.example.org, paths remaining intact) you can keep the
default handlerURL at ^/Shibboleth.sso for every vhost.

The differing handlerURLs only came about since we wanted to spread
content over different backend web servers where the content was all on
the same vhost (www.example.org).
That means unless you add a SPOF where you handle the SAML
interactions for all physical web servers on one host (proxing
www.example.org/Shibboleth.sso to, say, shib-sp.internal) and share
sessions between all web servers, you will need to be able to expose
the different backend web servers' handlers via the front end proxy.

I.e., we chose to "split" the load by exposing an internal identifier
of the backend web server that is currently responsible for any given
content content (say, www.example.org/vogon/poetry is proxied to
websrv42.internal and www.example.org/vogon/songs to
websrv18.internal) and expose that back end server's shib handler via
publicly visible URLs that could automatically be proxied to the right
server, e.g. www.example.org/server42/Shibboleth.sso/ in case when
someone wanted to log in to content that was hosted on
websrv42.internal, and www.example.org/server18/Shibboleth.sso/ when
the data was on websrv18.internal, etc.

So there is no clustering anywhere, there are only independent web
servers with independent Shib SPs and independent sessions.
If load distribution (across several web servers) based on vhost
*alone* is sufficient you don't need to do anything special wrt
Shib/SAML.
To make that work "distributing" content (request URLs, paths) from a
single vhost across several web servers "sharing nothing" you'll need
to expose the proxied web server's shib handlers to the browser.
(Or protect the reverse proxy, as Scott said. That wasn't a natural
fit in our case as we wanted to allow people to easily protect content
using familiar tools: the .htaccess files in their unpriviledged
accounts they were already accustomed to be using.)

N.B.: As a consequence there's also no automatic recognition of SP
sessions between www.example.org/vogon/poetry and
www.example.org/vogon/songs should those be handled by different back
ends. But since there SSO on the SAML level that's only a click away
("log in").

HTH,
-peter