Problem with SOAP call to 2.4.4 IdP / port 8443 / F5 load balancer

[Benji Wakely]

Fixed.  

Drafting an email but-not-sending for a while investigating was ... instructive,
and helped keep the noise off this list while I worked through problems.

Scott, thanks for your earlier response.
Figured I should note what happened here for reference.

Problem:
	- SP making a call to a previously-single-instance IdP (apache-->tomcat) fails when IdP is placed behind a load balancer. (f5-->apache-->tomcat)
		# There were actually two problems:
		- Disparate SP certs for back-and-front-end calls to IdP
		- back-end calls sometimes would route to Node A on load balancer where Node B had the Session created for the user

Actions:
	For the disparate SP certs:
	- Fixed initially (workaround) by splicing togther the differing back-and-front-end certificates
	  (duplicated '<KeyDescriptor>' section in local metadata file at /opt/shibboleth-idp/metadata/serviceFoo.xml (referenced in relying-party.conf)
	  replaced cert with the one revealed by turning Debugging up to high.)

	- Fixed Properly by discussing with Vendor.

	For the back-end/front-end / split sessions:
	- Not possible to Simply load balance this and keep Attribute Query functionality.
	   Can use Terracotta.  Varying reports of Terracotta being a Pain.

	- Better option would be to use memcache as detailed below.
	   References / reading: https://wiki.shibboleth.net/confluence/display/SHIB2/IdPStatelessClustering
				https://wiki.shibboleth.net/confluence/display/SHIB2/Memcached+StorageService

Unresolved / never got to the bottom of:
	- How or even conclusively If apache behaves differently when placed behind a load-balancer that Should be merely routing the traffic verbatim,
	   or if this was some kind of Weird Testing Fail on my behalf.

Would have been nice to know / have learned:
	- You can have more than certificate set in relying party metadata that will validate a SP.
	   If this was a simple google fail, my apologies.

Complicating issues / affecting diagnosis :

	- You can't load balance a back-channel request where the front-channel request hits a different node.
	   This was also occurring initially here and muddied the waters slightly.

Cheers,
--Benji