SP logging to syslog can not set syslogHost

[Jacob Lundberg]

Hi all,

On Wed, 2014-07-30 at 16:33 +0000, Cantor, Scott wrote:
> My suggestion to anybody trying to use syslog would be to just run
> something like minion to tail the log and remote the entries that way.

The log tailing systems we have tried have some drawbacks when you try
to use them at scale ... eventually you find some instances that have
stopped forwarding logs or lost their place and forwarded multiple
copies of the same logs ... rsyslog is our current syslog of choice and
it has both of these problems.

> But if somebody else has made it work, they're welcome to comment.

We use the syslog logger.  It has some drawbacks, mostly because it
creates only one configuration instance that is shared.  Probably the
issue for the OP is they have another instance of syslogging from the
same process that is created later and overwrites the destination.

If you can live within the constraints of the syslog logger it works
well.  (I know this may not work for a Windows user, but) we send all
the SP logs to localhost and then rsyslog writes them both to local disk
based on string matching and over the network to our centralized log
processing service.

Here are some diffs relative to the Debian defaults:


--- /etc/shibboleth/native.logger.orig  2013-06-18 15:28:42.391040157 -0700
+++ /etc/shibboleth/native.logger       2014-03-06 19:05:45.941357897 -0800
@@ -51,4 +51,6 @@
 log4j.appender.native_log=org.apache.log4j.LocalSyslogAppender
 log4j.appender.native_log.syslogName=shibboleth-sp
 log4j.appender.native_log.facility=3
-log4j.appender.native_log.layout=org.apache.log4j.BasicLayout
+#log4j.appender.native_log.layout=org.apache.log4j.BasicLayout
+log4j.appender.native_log.layout=org.apache.log4j.PatternLayout
+log4j.appender.native_log.layout.ConversionPattern=%p %c %x: %m%n


--- /etc/shibboleth/shibd.logger.orig   2013-06-18 15:28:51.991122948 -0700
+++ /etc/shibboleth/shibd.logger        2014-03-06 19:05:37.701284666 -0800
@@ -1,5 +1,5 @@
 # set overall behavior
-log4j.rootCategory=INFO, shibd_log, warn_log
+log4j.rootCategory=INFO, shibd_log
 
 # fairly verbose for DEBUG, so generally leave at INFO
 log4j.category.XMLTooling.XMLObject=INFO
@@ -35,27 +35,21 @@
 
 # define the appenders
 
-log4j.appender.shibd_log=org.apache.log4j.RollingFileAppender
-log4j.appender.shibd_log.fileName=/var/log/shibboleth/shibd.log
-log4j.appender.shibd_log.maxFileSize=1000000
-log4j.appender.shibd_log.maxBackupIndex=10
+log4j.appender.shibd_log=org.apache.log4j.LocalSyslogAppender
+log4j.appender.shibd_log.syslogName=shibd
+log4j.appender.shibd_log.facility=3
+#log4j.appender.shibd_log.layout=org.apache.log4j.BasicLayout
 log4j.appender.shibd_log.layout=org.apache.log4j.PatternLayout
-log4j.appender.shibd_log.layout.ConversionPattern=%d{%Y-%m-%d %H:%M:%S} %p %c %x: %m%n
+log4j.appender.shibd_log.layout.ConversionPattern=%p %c %x: %m%n
 
-log4j.appender.warn_log=org.apache.log4j.RollingFileAppender
-log4j.appender.warn_log.fileName=/var/log/shibboleth/shibd_warn.log
-log4j.appender.warn_log.maxFileSize=1000000
-log4j.appender.warn_log.maxBackupIndex=10
-log4j.appender.warn_log.layout=org.apache.log4j.PatternLayout
-log4j.appender.warn_log.layout.ConversionPattern=%d{%Y-%m-%d %H:%M:%S} %p %c %x: %m%n
-log4j.appender.warn_log.threshold=WARN
-
-log4j.appender.tran_log=org.apache.log4j.RollingFileAppender
-log4j.appender.tran_log.fileName=/var/log/shibboleth/transaction.log
-log4j.appender.tran_log.maxFileSize=1000000
-log4j.appender.tran_log.maxBackupIndex=20
+log4j.appender.tran_log=org.apache.log4j.LocalSyslogAppender
+# This is broken in Shibboleth's forked log4cpp.
+#log4j.appender.tran_log.syslogName=shibd-transaction
+log4j.appender.tran_log.syslogName=shibd
+log4j.appender.tran_log.facility=3
+#log4j.appender.tran_log.layout=org.apache.log4j.BasicLayout
 log4j.appender.tran_log.layout=org.apache.log4j.PatternLayout
-log4j.appender.tran_log.layout.ConversionPattern=%d{%Y-%m-%d %H:%M:%S} %p %c %x: %m%n
+log4j.appender.tran_log.layout.ConversionPattern=%p %c %x: %m%n
 
 log4j.appender.sig_log=org.apache.log4j.FileAppender
 log4j.appender.sig_log.fileName=/var/log/shibboleth/signature.log


And some rsyslog configuration:


root at server:~# cat /etc/rsyslog.d/50_action_templates.conf 
$EscapeControlCharactersOnReceive off
$template showPrio,"%PRI-text%: %timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n"


root at server:~# cat /etc/rsyslog.d/60_tagselect.conf 
# native log from Shibboleth SP Apache module
:programname, isequal, "shibboleth-sp"                  /var/log/shibboleth/shib-sp.log
& ~

# transaction log from Shibboleth SP shibd
:msg, contains, "Shibboleth-TRANSACTION"                /var/log/shibboleth/transaction.log
& ~

# shibd log from Shibboleth SP shibd
:programname, isequal, "shibd"                          /var/log/shibboleth/shibd.log
& ~


-Jacob

-- 

Jacob Lundberg
Director, IT Services
jacob at collegenet.com
503.290.0100 (voice)
503.973.5252 (fax)
503.901.8343 (cell)