<style>
/* Changing the layout to use less space for mobiles */
@media screen and (max-device-width: 480px), screen and (-webkit-min-device-pixel-ratio: 2) {
#email-body { min-width: 30em !important; }
#email-page { padding: 8px !important; }
#email-banner { padding: 8px 8px 0 8px !important; }
#email-avatar { margin: 1px 8px 8px 0 !important; padding: 0 !important; }
#email-fields { padding: 0 8px 8px 8px !important; }
#email-gutter { width: 0 !important; }
}
</style>
<div id="email-body">
<table id="email-wrap" align="center" border="0" cellpadding="0" cellspacing="0" style="background-color:#f0f0f0;color:#000000;width:100%;">
<tr valign="top">
<td id="email-page" style="padding:16px !important;">
<table align="center" border="0" cellpadding="0" cellspacing="0" style="background-color:#ffffff;border:1px solid #bbbbbb;color:#000000;width:100%;">
<tr valign="top">
<td bgcolor="#ffffff" style="background-color:#ffffff;color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;line-height:1;"><img src="https://shibboleth.net/images/shib_240x83.png" alt="" style="vertical-align:top;" /></td>
</tr><tr valign="top">
<td id="email-banner" style="padding:32px 32px 0 32px;">
<table align="left" border="0" cellpadding="0" cellspacing="0" width="100%" style="width:100%;">
<tr valign="top">
<td style="color:#505050;font-family:Arial,FreeSans,Helvetica,sans-serif;padding:0;">
<img id="email-avatar" src="https://issues.shibboleth.net/jira/secure/useravatar?avatarId=10202" alt="" height="48" width="48" border="0" align="left" style="padding:0;margin: 0 16px 16px 0;" />
<div id="email-action" style="padding: 0 0 8px 0;font-size:12px;line-height:18px;">
<a class="user-hover" rel="esbitnjxxmhh96nfwtqbwsj6aw4=@https://signon.deakin.edu.au/idp/shibboleth" id="email_esbitnjxxmhh96nfwtqbwsj6aw4=@https://signon.deakin.edu.au/idp/shibboleth" href="https://issues.shibboleth.net/jira/secure/ViewProfile.jspa?name=esbitnjxxmhh96nfwtqbwsj6aw4%3D%40https%3A%2F%2Fsignon.deakin.edu.au%2Fidp%2Fshibboleth" style="color:#326ca6;">Aaron Howell</a>
created <img src="https://issues.shibboleth.net/jira/images/icons/bug.gif" height="16" width="16" border="0" align="absmiddle" alt="Bug"> <a style='color:#326ca6;text-decoration:none;' href='https://issues.shibboleth.net/jira/browse/SSPCPP-550'>SSPCPP-550</a>
</div>
<div id="email-summary" style="font-size:16px;line-height:20px;padding:2px 0 16px 0;">
<a style='color:#326ca6;text-decoration:none;' href='https://issues.shibboleth.net/jira/browse/SSPCPP-550'><strong>native.log </strong></a>
</div>
</td>
</tr>
</table>
</td>
</tr>
<tr valign="top">
<td id="email-fields" style="padding:0 32px 32px 32px;">
<table border="0" cellpadding="0" cellspacing="0" style="padding:0;text-align:left;width:100%;" width="100%">
<tr valign="top">
<td id="email-gutter" style="width:64px;white-space:nowrap;"></td>
<td>
<table border="0" cellpadding="0" cellspacing="0" width="100%">
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Issue Type:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<img src="https://issues.shibboleth.net/jira/images/icons/bug.gif" height="16" width="16" border="0" align="absmiddle" alt="Bug"> Bug
</td>
</tr> <tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Affects Versions:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
2.5.1, 2.5.0 </td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Assignee:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<a class="user-hover" rel="cantor.2@osu.edu" id="email_cantor.2@osu.edu" href="https://issues.shibboleth.net/jira/secure/ViewProfile.jspa?name=cantor.2%40osu.edu" style="color:#326ca6;">Scott Cantor</a>
</td>
</tr> <tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Components:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
Logging </td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Created:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
12/Feb/13 10:41 PM
</td>
</tr> <tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Description:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<p style='margin-top:0;margin-bottom:10px;'>We have configured log4j settings in Shibboleth to use FileAppend - as we use logrotate for log rotation - (which ties into an automated shipping to a fileshare process as we need to retain logs for 2 years)</p>
<p style='margin-top:0;margin-bottom:10px;'>When the file is moved and apache is reloaded, the file handle is retained and a new file created. Apache needs to be restarted to actually drop file handles.</p>
<p style='margin-top:0;margin-bottom:10px;'>In fact when apache is reloaded the current handles aren't dropped at all, but new ones are created to the same file. These just keep growing</p>
<p style='margin-top:0;margin-bottom:10px;'>The reason we noticed this was /var filled up on one of our server - but then further investigation showed this occurring on all servers. For the time being we are planning to work around part of the issue by copy-truncating with logrotate or using the RollingFileAppender - however this only addresses the growing disk space - file handles are still growing with every reload - which will presumably eventually lead to issues with the max open file limit.</p>
<p style='margin-top:0;margin-bottom:10px;'>11:45:46$ sudo service httpd restart<br/>
Stopping httpd: [ OK ]<br/>
Starting httpd: [ OK ]<br/>
$:/var/log/httpd<br/>
11:51:14$ sudo lsof | grep native.log<br/>
httpd 10402 root 16w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10402 root 18w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10404 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10404 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10405 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10405 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10408 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10408 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10410 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10410 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10412 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10412 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10414 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10414 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10416 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10416 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10419 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
httpd 10419 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log<br/>
11:51:17$ sudo mv native.log native.log.1<br/>
11:51:34$ sudo lsof | grep native.log<br/>
httpd 10402 root 16w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10402 root 18w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10404 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10404 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10405 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10405 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10408 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10408 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10410 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10410 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10412 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10412 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10414 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10414 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10416 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10416 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10419 apache 16w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
httpd 10419 apache 18w REG 8,3 71260 917534 /var/log/httpd/native.log.1<br/>
11:51:36$ sudo service httpd reload<br/>
Reloading httpd: [ OK ]<br/>
11:52:17$ sudo lsof | grep native.log<br/>
httpd 10402 root 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10402 root 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10402 root 20w REG 8,3 33668 917582 /var/log/httpd/native.log<br/>
httpd 10488 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10488 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10488 apache 20w REG 8,3 33668 917582 /var/log/httpd/native.log<br/>
httpd 10490 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10490 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10490 apache 20w REG 8,3 33668 917582 /var/log/httpd/native.log<br/>
httpd 10492 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10492 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10492 apache 20w REG 8,3 33668 917582 /var/log/httpd/native.log<br/>
httpd 10494 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10494 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10494 apache 20w REG 8,3 33668 917582 /var/log/httpd/native.log<br/>
httpd 10496 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10496 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10496 apache 20w REG 8,3 33668 917582 /var/log/httpd/native.log<br/>
httpd 10498 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10498 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10498 apache 20w REG 8,3 33668 917582 /var/log/httpd/native.log<br/>
httpd 10504 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10504 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10504 apache 20w REG 8,3 33668 917582 /var/log/httpd/native.log<br/>
httpd 10506 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10506 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1<br/>
httpd 10506 apache 20w REG 8,3 33668 917582 /var/log/httpd/native.log<br/>
13:49:50$ sudo rm native.log.1<br/>
13:53:03$ sudo lsof | grep native.log<br/>
httpd 10402 root 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10402 root 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10402 root 20w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10402 root 22w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10536 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10536 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10536 apache 20w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10536 apache 22w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10538 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10538 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10538 apache 20w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10538 apache 22w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10539 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10539 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10539 apache 20w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10539 apache 22w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10540 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10540 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10540 apache 20w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10540 apache 22w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10542 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10542 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10542 apache 20w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10542 apache 22w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10546 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10546 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10546 apache 20w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10546 apache 22w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10548 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10548 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10548 apache 20w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10548 apache 22w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10550 apache 16w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10550 apache 18w REG 8,3 74736 917534 /var/log/httpd/native.log.1 (deleted)<br/>
httpd 10550 apache 20w REG 8,3 70812 917582 /var/log/httpd/native.log<br/>
httpd 10550 apache 22w REG 8,3 70812 917582 /var/log/httpd/native.log</p>
<p style='margin-top:0;margin-bottom:10px;'>13:57:00$ sudo service httpd restart<br/>
13:57:00$ sudo lsof | grep apache | wc -l<br/>
1104<br/>
13:57:07$ sudo lsof | grep native.log | wc -l<br/>
18<br/>
13:57:08$ sudo lsof | grep native_warn.log | wc -l<br/>
18<br/>
13:57:17$ sudo service httpd reload<br/>
Reloading httpd: [ OK ]<br/>
13:57:25$ sudo lsof | grep native.log | wc -l<br/>
27<br/>
13:57:26$ sudo lsof | grep native_warn.log | wc -l<br/>
27<br/>
13:57:27$ sudo lsof | grep apache | wc -l<br/>
1120</p>
</td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Environment:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<p style='margin-top:0;margin-bottom:10px;'>RHEL 5 & 6<br/>
Apache prefork (default)</p>
</td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Project:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<a style="color:#326ca6;" href="https://issues.shibboleth.net/jira/browse/SSPCPP">Shibboleth SP - C++</a>
</td>
</tr> <tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Labels:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
log4shib
</td>
</tr>
<tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Priority:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<img src="https://issues.shibboleth.net/jira/images/icons/priority_minor.gif" height="16" width="16" border="0" align="absmiddle" alt="Minor"> Minor
</td>
</tr> <tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Reporter:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
<a class="user-hover" rel="esbitnjxxmhh96nfwtqbwsj6aw4=@https://signon.deakin.edu.au/idp/shibboleth" id="email_esbitnjxxmhh96nfwtqbwsj6aw4=@https://signon.deakin.edu.au/idp/shibboleth" href="https://issues.shibboleth.net/jira/secure/ViewProfile.jspa?name=esbitnjxxmhh96nfwtqbwsj6aw4%3D%40https%3A%2F%2Fsignon.deakin.edu.au%2Fidp%2Fshibboleth" style="color:#326ca6;">Aaron Howell</a>
</td>
</tr> <tr valign="top">
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 10px 10px 0;white-space:nowrap;">
<strong style="font-weight:normal;color:#505050;">Security Level:</strong>
</td>
<td style="color:#000000;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:12px;padding:0 0 10px 0;width:100%;">
Standard (Standard bug, may impact functionality but does not represent a security vulnerability ) </td>
</tr>
</table>
</td>
</tr>
</table>
</td>
</tr>
</table>
</td><!-- End #email-page -->
</tr>
<tr valign="top">
<td style="color:#505050;font-family:Arial,FreeSans,Helvetica,sans-serif;font-size:10px;line-height:14px;padding: 0 16px 16px 16px;text-align:center;">
This message is automatically generated by JIRA.<br />
If you think it was sent incorrectly, please contact your JIRA administrators<br />
For more information on JIRA, see: <a style='color:#326ca6;' href='http://www.atlassian.com/software/jira'>http://www.atlassian.com/software/jira</a>
</td>
</tr>
</table><!-- End #email-wrap -->
</div><!-- End #email-body -->