two apache instances using one shibd process possible?

[Gernot Hassenpflug]

"Cantor, Scott" <cantor.2 at osu.edu> writes:

> On 1/30/13 5:00 AM, "Gernot Hassenpflug" <ha4h-grnt at asahi-net.or.jp> wrote:
> >
> >The reason is to try to overcome the memory limit on 32-bit apache,
> >since we are using mod_perl with preforked apache.
> 
> Prefork and Shibboleth don't get along well, so one spike in traffic will
> take you out unless you lower the thread stack size.

Hello Scott,

Thanks for the feedback. I am still not clear as to how best I should
be managing session information in my case, hopefully the below will
make it clear what the problem is.

Regarding stack size:

I did find a msg from you on 11 Jan 2010 on the shibboleth-dev where
you wrote that decreasing from the default (confirmed with ulimit -a)
of 10M to something like 1M or 64K or less, will be possible (and
necessary) to enable more than about 300 threads to be handled by
shibd.

So we can certainly do that.
 
> >However, if I activate both, then I receive the IdP login screen
> >correctly, but after entering the user credentials the IdP seems to
> >continuously send the redirect messages, without the redirect actually
> >happening in the browser. This seems to indicate some kind of looping,
> >but I am unsure where, and also whether the architecture I am using is
> >technically possible.
> 
> If it's looping, then your logs will make it pretty clear. Live Headers
> and your logs are the best tool for figuring them out.

I understand that
- the session information is stored in shibd; and
- each apache process with the mod_shib module caches session information.

Assuming my server is named testserver.jp, we have two apache
processes, each on a different local address (127.0.0.{1,2}), then
after the IdP login page, the redirection step to the accessed page
will involve several attempts by the IdP (it seems to keep on sending
redirect requests), and eventually the redirection works.

Presumably success happens when the IdP redirect request is
intercepted by an apache instance belonging to the process where the
initial access was made.

I've now tested also the TCPListener, and this too "works" because at
some point in the redirection repeats the matching apache process will
intercept a repear redirect request. Obviously this does not function
under any load.

  <TCPListener address="127.0.0.1" port="1600" acl="127.0.0.1 127.0.0.2"/>

(shibd, and two apache processes are on the same physical box)

I thought from reading 
https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPClustering
that I could run two apache processes as though they were on different
boxes.  I believe this works exactly as it should.

However, nginx in front of these two instances defines a backend
comprising both of them:

upstream backend {
 server 127.0.0.1:81;
 server 127.0.0.2:81;
}

and then uses this backend variable in the proxy_pass command to the
httpd processes.

E.g.:

        location /shibboleth {
                proxy_pass    http://backend;
        }


        location /Shibboleth.sso/ {
                proxy_pass    http://backend/Shibboleth.sso/;
                proxy_redirect http://testserver.jp/ https://testserver.jp/;
                proxy_redirect http://testserver.jp:443/ https://testserver.jp/;

So this presumably causes httpd and shibd to have problems, since
there is no telling which instance of which apache process will handle
any particular request.

Would it be necessary to have session-awareness in the front-end, or
is there some way to avoid the problem as is? Or perhaps I am mistaken
in what my problems are?

Regards,
-- 
Gernot Hassenpflug