FastCGI timeout during CiviCRM WordPress plugin install

FastCGI requires a high IO timeout value for certain scripts to run. You will otherwise see a 500 Internal Server Error.

I had a client running into problems installing the CiviCRM plugin on his WordPress site.

I am running FastCGI on my server, and I noticed time-outs in the logs –

Looking at /usr/local/apache/logs/error_log –

[Sat Oct 24 12:21:23.345285 2015] [fcgid:warn] [pid xxxxx:tid xxxxx] [client xx.xx.xx.xx:xxxxx] mod_fcgid: read data timeout in 60 seconds, referer: http://xxxxxx.xxx/wp-admin/options-general.php?page=civicrm-install
[Sat Oct 24 12:21:23.345327 2015] [fcgid:warn] [pid xxxxx:tid xxxxx] (110)Connection timed out: [client xx.xx.xx.xx:xxxxx] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function, referer: http://xxxxxx.xxx/wp-admin/options-general.php?page=civicrm-install

Doing searches on the errors seemed to suggest that various FCGI resource limits needed to be increased, but the solution here was very simple. The installer says it will take several minutes to complete. It only required the FcgidIOTimeout directive to be increased to accommodate the install.

From the apache docs here –

https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html#fcgidiotimeout

Description: Communication timeout to FastCGI server
Syntax: FcgidIOTimeout seconds
Default: FcgidIOTimeout 40
Context: server config, virtual host
Status: External
Module: mod_fcgid

So, default is 40 seconds (unless you’ve defined otherwise in your apache config.) I had increased mine to 60 seconds, but this wasn’t sufficient.

Increasing temporarily to 600 seconds allowed the install to complete. This was done by adding the following directive to post_virtualhost_global.conf –

FcgidIOTimeout 600

Just go to WHM -> Apache Configuration -> Includes Editor -> Post VirtualHost Include -> All Versions

If you have FCGI installed on your server, you should see a block that looks something like the following –


FcgidMinProcessesPerClass 0
FcgidMaxProcessesPerClass 20
FcgidMaxProcesses 300
FcgidIdleTimeout 60
FcgidProcessLifeTime 120
FcgidIdleScanInterval 30


Just add the following directive in between the If tags –

FcgidIOTimeout 600

If FCGI is installed and this block does not exist, just add the following –


FcgidIOTimeout 600

I would recommend that after the install you simply change the timeout value back to the default or to something like 60 seconds unless you have other good resource controls in place.


How to remove old kernels – Centos / RedHat

If you’ve got limited space on your /boot partition and want to tidy things up by removing old kernels, it’s really easy –

First you need to install yum-utilities –

yum install yum-utils

Now you can just use the “package-cleanup” command and it will do all the work for you –

package-cleanup --oldkernels --count=2

Notice at the end of the command line “–count=2” – in this example, I’ve told it to leave the most recent 2 kernels in place. You can just as easily change this to whatever (non-zero!) number of kernels you prefer to leave in place on your /boot partition.

If you prefer for yum to manage this for you, you can set a limit in /etc/yum.conf –

installonly_limit=2

Setting that value inside yum.conf will have yum automatically delete older kernels when installing new ones to keep your limit to the desired number.

If you ever want to view the current kernel version you are running, just use the following –

uname -r

If you want see all kernels on your system, you can use –

rpm -q kernel

How to kill a user’s ssh session

Ever have the need to kill an active ssh session?  It’s pretty simple with the pkill command.  The format is as follows –

pkill <ssh session>

Here’s an example:

First use the “w” command (or the “who” command if you prefer) to show all connected users –

root@server [~]# w
 23:00:16 up 4 days, 22:33,  4 users,  load average: 0.43, 0.42, 0.43
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    24.191.207.228   Mon15    2days  0.05s  0.05s -bash
root     pts/1    208.75.121.102   22:47    6:42   0.14s  0.04s mysql
root     pts/2    71.168.151.89    23:00    0.00s  0.01s  0.00s w
root     pts/3    10.1.80.100      21:47   59:05   0.05s  0.05s -bash

Let’s kill the last user on tty pts/3 –

root@server [~]# pkill pts/3
root@server [~]# w
 23:03:25 up 4 days, 22:36,  3 users,  load average: 0.47, 0.45, 0.44
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    24.191.207.228   Mon15    2days  0.05s  0.05s -bash
root     pts/1    208.75.121.102   22:47    9:51   0.15s  0.04s mysql
root     pts/2    71.168.151.89    23:00    0.00s  0.01s  0.00s w

As you can see, we’ve successfully killed the session for pts/3.


Fatal error, run database recovery // Thread died in Berkeley DB library // corrupt RPM database

I received two notifications, the first being a failed cpanel update warning due to not having the needed RPMs, and the second being an RPM update failure warning.

This was due to RPM database corruption and is easy to fix.

1. Navigate to /var/lib/rpm where your rpm database files are located

2. Delete or rename the current __db files – they will look something like this –

-rw-r–r–   1 root root    24576 Oct 27 23:00 __db.001
-rw-r–r–   1 root root   262144 Oct 27 23:00 __db.002
-rw-r–r–   1 root root  1318912 Oct 27 23:00 __db.003
-rw-r–r–   1 root root   770048 Oct 27 01:31 __db.004

3. Do “rpm –rebuilddb” – this will build brand new __db files just like the above – note that it takes a while for them to be rebuilt.

4. You can then proceed with your updates (ie. “YUM UPDATE”) – my understanding is that you can skip step 3 and a YUM UPDATE with no __db files shoudl rebuild them for you as well

I also subsequently ran SMART tests on my drives to make sure there were no drive issues, just to be sure.

The full error messages I was seeing looked something like the below –

Thread died in Berkeley DB library error: db3 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages database in /var/lib/rpm


Experiencing Sleep Issues?…READ THIS POST!

Welcome to our new blog, dedicated to helping cpanel admins and those that can’t sleep…as well as those that won’t sleep or can’t sleep because they are experiencing cpanel admin issues.

As a non-expert admin, I find myself encountering issues for which I wish I had quick solutions at hand.  My goal is to use this blog as my notepad for not only my own future reference, but for yours as well.  I hope this will be helpful to other admins going through similar learning experiences.