load balancing 2 shibboleth IdP servers

[Brian Moon]

We have the pleasure here of having our IdP servers both on-prem and hosted
in GCP (well, as of tonight when our GCP instances go live).  The load
balancing question I believe has mostly been answered here.  All I will add
is that if you are doing active-active, make sure you have sticky
session/client affinity configured so that clients always reach the same
IdP server.  I have also found it very helpful to terminate SSL at the load
balancer, although I still recommend SSL from the load balancer to the IdP
servers (which is very easy to manage if you can use self-signed certs on
the backend).

For keeping things in sync, I wrote a script long ago that will use rsync
to copy specific directories from the primary node to all of the secondary
nodes.  The script would also look at the files that have been
transferred to identify what changes were made and then determine if any
services needed to be reloaded, or if there was a major change that
required a full restart of the web application.  This worked great for just
the on-prem deployment.  Now that we have migrated into GCP as well, things
look a bit different.

With GCP, we have a source repository with all of the server configs (not
just IdP), a startup script that installs all of the required packages on
our template VMs, images made from the template VM, and then that same
startup script is used on the actual instance template to pull the configs
from the source repository, drop them in the proper place (with the help of
another script), and restart services.  Effectively, any change we make to
the configs in the source repository requires either restarting or
replacing the instances in the managed instance group (which is pretty
quick) and ensures a clean and consistent state.  We also store all of our
passwords and certs in Google's secret manager so that we don't have to
store them in the source repository.

Our on-prem script was then modified to pull that source repository
locally, then rsync from the local copy to the shibboleth installation and
restart any shibboleth services as needed.  Then it will do the same rsync
from that primary node to all of the secondary nodes and restart services
as needed.

If anyone is interested in actually seeing these scripts or the setup, let
me know and I'll be happy to share the information.

Cheers!

Brian Moon
Senior System Administrator, Enterprise Systems
Santa Clara University


On Wed, Aug 5, 2020 at 11:21 AM IAM David Bantz <dabantz at alaska.edu> wrote:

> I have 2 production instances behind F5. One is primary, the other a
> secondary/fail-over if the primary is unreachable. I deploy new
> integrations in the secondary, validate by DNS override in client etc/host
> file (by-pass the F5 to directly use the secondary IdP instance), then
> migrate changes to the primary and force re-load. It's been a notably
> robust configuration, but if we deploy additional nodes at remote sites or
> in the cloud, may need modification.
>
> A fairly recent change was to terminate TLS on the F5 to enable
> application layer decision-making on the F5; I insisted on re-encryption
> from the F5 to the IdPs.
>
> David St. Pierre Bantz
> U Alaska
>
> On Wed, Aug 5, 2020 at 10:08 AM Cantor, Scott <cantor.2 at osu.edu> wrote:
>
>> On 8/5/20, 2:05 PM, "users on behalf of Donald Lohr" <
>> users-bounces at shibboleth.net on behalf of lohrda at jmu.edu> wrote:
>>
>> >    I don't believe load is an issue either, folks want to use a
>> commercial
>> >    load balancing product to share the work between more than 1 IdP.
>> More
>> >    over to maintain the SSO feature of Shibboleth across more than one
>> IdP
>> >    being balanced.
>>
>> That's why client-side is the default, it takes that out of the equation.
>>
>> -- Scott
>>
>>
>> --
>> For Consortium Member technical support, see
>> https://wiki.shibboleth.net/confluence/x/coFAAg
>> <https://urldefense.com/v3/__https://wiki.shibboleth.net/confluence/x/coFAAg__;!!MLMg-p0Z!VkM9EeMlxiuOnx_6sQeQ6JoJ-0bws4mhYKXPDTJTb-pkoBqkLeXHJiByr576$>
>> To unsubscribe from this list send an email to
>> users-unsubscribe at shibboleth.net
>>
> --
> For Consortium Member technical support, see
> https://urldefense.com/v3/__https://wiki.shibboleth.net/confluence/x/coFAAg__;!!MLMg-p0Z!VkM9EeMlxiuOnx_6sQeQ6JoJ-0bws4mhYKXPDTJTb-pkoBqkLeXHJiByr576$
> To unsubscribe from this list send an email to
> users-unsubscribe at shibboleth.net
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://shibboleth.net/pipermail/users/attachments/20200805/f10e520b/attachment.htm>