IDP freezes, JettyService.exe crashed and false system disk capacity alert - possibly heap related ?

[Karen Murphy]

Hi all,

I run an IdP (version 2.3.8) on Windows Server 2008 under Jetty 7.3.1.

Yesterday the IdP failed - Jetty crashed (see below), and at the same 
time false system alerts on disk space occurred (seen by email alerts).  
Disk capacity was only at 50% of total 80GB capacity.  I got the IdP 
going again by restarting Jetty, but need to understand what happened.

There didn't appear to be any general network issues.   Nothing unusual 
in the Shibboleth logs, and nothing about heap size in the Jetty logs.
The event logs contained:

Error - Application Error - 14:28:52
Faulting application name: JettyService.exe, version: 1.0.10.0, time 
stamp: 0x4f4600a2
Faulting module name: ntdll.dll, version: 6.1.7601.17725, time stamp: 
0x4ec4aa8e
Exception code: 0xc0000005
Fault offset: 0x0000000000053210
Faulting process id: 0x214
Faulting application start time: 0x01cedf8048792d0c
Faulting application path: 
c:\jetty-distribution-7.3.1.v20110307\bin\JettyService.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: 12552a2a-56a7-11e3-b01b-0050568b0d9a

Warning - srv - 26/11/2013 14:36
The C: disk is at or near capacity.  You may need to delete some files.

I'm guessing this may be heap related.  My Java options are set as 
followings:

-Xms512M
-Xmx4096M
-Xincgc
-XX:MaxPermSize=128M
-XX:+DisableExplicitGC
-XX:+UseParallelGC
-XX:+UseParallelOldGC

Which I think are compatible with recommendations ?

I'd like to understand better what may have caused this so as to avoid 
it in future.  Initially I suspected disk corruption but a full scan was 
good.

Can anyone shed any further insight as to what may have happened ?

Many thanks,

Karen


-- 
Karen Murphy
Systems Analyst - Bibliographic Services
The Library at Queen's
Queen's University Belfast
Belfast BT7 1LP
Tel: 028 90976260
Email: k.l.murphy at qub.ac.uk