SOLVED: Apache HTTPd Crashes or Goes Offline Exactly At Midnight

After updating Apache httpd on a server running FreeBSD, it went offline at exactly midnight the following night, producing a log entry like this in httpd-errors:

[Tue Aug 10 00:00:01 2010] [notice] SIGHUP received. Attempting to restart
[Tue Aug 10 00:00:01 2010] [notice] seg fault or similar nasty error detected in the parent process

The server would start right back up using an “apachectl start” command and everything seemed to work fine unless httpd was told to restart or restart gracefully (“apachectl graceful”) and then it will crash.

In this case we had newsyslog(8) under FreeBSD rotating Apache httpd’s log files at midnight. After it moves the files it then tells the server to restart so it established new file handles. This restart command was triggering the seg fault and ensuing crash.

I eventually tracked down that it was a conflict between Apache HTTPd itself and the PHP extension/module “mhash.so”.

This symptom can have different causes but the remedy is usually just recompiling the offending code.  Here is how to isolate what needs to be recompiled/reinstalled.

Verify The Issue

Make sure you can reproduce the issue reliably.  Running “apache graceful” should trigger it to crash.  If not then there is another cause for this symptom which is not covered in this article (leave a note if you know of anything else that triggers this).

Check Apache

First, comment out the PHP module line(s) in httpd.conf (ie; turn off PHP loading) to see if the issue is inside the Apache, PHP or other modules.

STILL CRASHES:  If it still crashes when you “apache graceful” then the problem is within Apache or another module being loaded.  Usually recompliling Apache will fix this.

DOESN’T CRASH: It’s time to look at the PHP modules or the core itself (see below).

Check PHP’s Modules

Comment out all the lines inside your extensions.ini file (add semicolons (;) in front of each line) and try gracefully restarting.

STILL CRASHES: If Apache still crashes then the PHP core itself is the issue, recompiling it should fix it (which might mean you now need to recompile all the modules as well because you have a new core).

DOESN’T CRASH: If Apache doesn’t crash anymore then the problem is one of the modules.  One by one uncomment them until you isolate which one.  Recompiling it should fix the issue.