Fix problems caused by DISABLE_WP_CRON

OVERVIEW

Many online tutorials recommend disabling WP-CRON and using a “real” cron job to improve WordPress performance. This doesn’t always work. You may notice after following this advice that automated tasks provided by plug-ins, and WordPress itself, stop working. This article discusses what can go wrong when using the DISABLE_WP_CRON constant, and how to fix it.

BACKGROUND

There are plenty of online tutorials giving the same advice on disabling WP-CRON in WordPress and using a cron job in its place. What does this mean? WordPress and its plug-ins have tasks they need to perform in the background on a regular schedule. For example, the WP Super Cache plug-in needs to delete old cached web pages and generate new cached versions of web pages from time to time.  The WP-CRON system in WordPress handles this by checking for, and performing, any tasks that are scheduled.

The wp-cron.php file has code in it to run these scheduled tasks at the appropriate times. To make installation of WordPress easier on various operating systems, WordPress developers decided that all installations will run the code in wp-cron.php during every single web page request. There are three problems with this:

  • If your web site isn’t very popular, scheduled tasks may be late in running. This is because scheduled tasks can only run when a web page is requested from your web site. In the real world this is not a problem on a public web site. There are plenty of ‘bots accessing your web site 7/24. But if you are using a robots.txt file or other methods of stopping robots from crawling your site, it could still be a problem.
  • A variation of the first problem can occur if you are using an external caching service like CloudFlare. On a very strongly cached web site, wp-cron.php might not get called for days.
  • Common sense would seem to indicate that running wp-cron.php on each and every web page request is not very efficient. Can you get better performance if there were a better way to take care of these background tasks?  The short answer is, most people won’t notice a difference in performance after setting up a real cron job for WordPress.

It is the promise of better performance that most online tutorials cite when trying to convince you to use DISABLE_WP_CRON with a cron job. However, during a normal request for a web page to a WordPress web site, wp_cron() will exit if there are any other calls to wp_cron() running, or if there are no crons jobs currently scheduled, or if any of several invalid or special conditions exist. Also, if a final decision is made to execute the code in wp-cron.php, the file is loaded as a spawned request so that it will not delay processing of the original user web page request.


BASIC SETUP

The first step in switching to using a real cron job is to add the following line to your wp-config.php file, somewhere near the top, exactly as shown here.

define('DISABLE_WP_CRON', true);

The next step is to add a cron job to your system. How this is done varies depending on your web hosting provider. Some web hosting providers don’t give you an option to create cron jobs. In this case, stop reading now, and go find a better web hosting service.

If you have a co-located server or a virtual private server (VPS, or sometimes referred to as a Cloud Server), then familiarize yourself with using crontab to add cron jobs to your system.

If your web site runs on Windows, familiarize yourself with the AT command or the Task Scheduler.

TROUBLESHOOTING TOOLS

  • Remove command output redirects
    Online tutorials all suggest adding “> /dev/null 2>&1” to the end of each suggested command line. This discards any regular output, and any error messages, from the command. If everything is working correctly, this is not a problem. However, if anything is wrong, you won’t see any indication of it. While troubleshooting cron jobs it is best to remove these directives from the command. This will allow you to review any output from your command line. Use the crontab -e command and the MAILTO= setting to set an email address to send any cron command output to.
  • Test cron jobs from the shell command line
    You can test your cron job command line by logging in to your web server and running it from the shell command line. However, if your web server runs under a different user account, you may not be seeing the same errors that occur when cron runs the command. Unless otherwise configured, Linux with Apache web server runs under an account named www-data and that account is configured to have no shell login. To temporarily change that, use the su (switch user) command and specify a default shell:
    su -s /bin/bash www-data -c 'your commands here'

    With shared web server hosting plans, you most likely are logging in using the same account that is assigned to your web site. So running a cron job at the command line in your user account will be exactly like running it as a cron job.

  • Enable WordPress debugging and logging
    Enable WordPress debugging and logging so that you can review any PHP errors that occur when your cron job runs. Add the following lines to your /wp-config.php file.
    // Enable WP_DEBUG mode
    define('WP_DEBUG', true);
    
    // Enable Debug logging to the /wp-content/debug.log file
    define('WP_DEBUG_LOG', true);
    
    // Disable display of errors and warnings
    define('WP_DEBUG_DISPLAY', false);
    
  • File Permissions and Ownership
    If you have automatic updates to WordPress, your plugins, and/or theme that never seem to update with the cron job, it could be a file permissions or ownership problem. This can be a problem if you are doing file operations while logged into a shell account that is different from the account your web site runs in (e.g. on a VPS or “cloud server”). A good troubleshooting tool to have is a command alias that makes fixing the permissions and ownership of your web site files and directories easier, as shown below. Adjust the paths, permissions, and owner as needed for your site and server…
    alias fixperms='find /var/www/vhosts/website.com/ -type d -exec chmod 755 {} \; && find /var/www/vhosts/website.com/ -type f -exec chmod 644 {} \; && chown -R www-data:www-data  /var/www/vhosts/website.com/'
    

COMMON CRON COMMANDS AND THEIR PROBLEMS

Using wget

One of the most common cron job commands suggested in online tutorials looks like this…

wget -q -O - https://www.website.com/wp-cron.php?doing_wp_cron > /dev/null 2>&1

There is no need to add the ?doing_wp_cron query string.  WordPress uses this query string value to pass a lock code for WP-CRON from the spawn_cron() function.  It won’t hurt, but it also doesn’t do anything.  Simply remove the query string, like this..

wget -q -O - https://www.website.com/wp-cron.php > /dev/null 2>&1

However, there can still be problems with this cron command, such as…

  • Problems with the default settings used by wget.
  • An error while verifying an SSL certificate.
  • Security software on your web server, or a WordPress firewall plugin, interpreting a request lacking typical browser data as a hacking attempt.

The following command line will prevent the NinjaFirewall plugin from blocking a call by wget to wp-cron.php. It will also prevent problems with self-signed certificates, timeouts, and other problems.

wget -q --ignore-length --inet4-only --no-check-certificate --no-cache \
   --timeout=60 --user-agent="Mozilla/5.0" --header="Accept: */*;q=0.8" \
   --header="Accept-Language: en-US,en;q=0.5" \
   https://www.website.com/wp-cron.php > /dev/null 2>&1

 

Using PHP-CLI or WP-CLI

Some online tutorials suggest one of two other cron command lines. One uses the command line version of php to directly call wp-cron.php.

/usr/bin/php /var/www/vhosts/website.com/wp-cron.php > /dev/null 2>&1

The other uses wp-cli (a command line interface for WordPress).

/usr/local/bin/wp cron event run --due-now --quiet --path=/var/www/vhosts/website.com > /dev/null 2>&1

Note that both of these commands require your cron job to be running on the same server as your web site (or at least have access to the file system of your web site).

Both of these methods have the same problem. Since the code is called from the command line, no web browser is involved, and no web server software is involved. Because of this, commonly used global variables such as $_SERVER[], $_REQUEST[], $_GET[], $_POST[] do not exist.

An indicator of this problem is if your WordPress debug log has lines that contain: Undefined index on $_SERVER. Unfortunately, many plugin developers assume that the global array $_SERVER always exists and is always valid. Even some parts of the WordPress source code still use these global arrays without first checking for validity.

Another problem to watch out for with these two methods is the user account being used to run these commands.  You should not use the root account.  Ideally, you should execute these commands with the same account that your web site process is running in.  However, you can use the su -s /bin/bash www-data -c method mentioned above to switch users before running the cron commands.

These, and other problems, caused by command line access to WordPress files are explained here.

If you are lucky, the only global variables that are needed are $_SERVER[‘HTTP_HOST’] and $_SERVER[‘SERVER_NAME’]. If that is true, you can get your cron job working again by exporting environment variables HTTP_HOST and SERVER_NAME before your command line as follows (replace www.website.com and the file paths with the correct settings for your site)…

su -s /bin/bash www-data -c 'export HTTP_HOST="www.website.com"; export SERVER_NAME=$HTTP_HOST; /usr/bin/php /var/www/vhosts/website.com/wp-cron.php > /dev/null 2>&1'

For the wp-cli method use…

su -s /bin/bash www-data -c 'export HTTP_HOST="www.website.com"; export SERVER_NAME=$HTTP_HOST; /usr/local/bin/wp cron event run --due-now --quiet --path=/var/www/vhosts/website.com > /dev/null 2>&1'

When the Apache web server receives a request for a web page, it places information about the request into environment variables for other programs to use. Exporting these two environment variables tricks PHP into thinking that the Apache web server received a real request and set these environment variables. PHP will then place them into the $_SERVER global variable array. Then the PHP code in WordPress can access them without errors.

If a plugin is causing problems because it is using more than these two variables, try contacting the developer and asking if they can test their plugin with WP-CLI. If the developer won’t help you, you can create a custom PHP file to set up the global variables needed, and include the wp-cron.php file as the last line of the file. Then change your cron command to call the new PHP file instead.

For example, create a new file “my-cron.php” in your web site root directory and add the following code to it…

<?php
  $_SERVER = array();
  $_GET = array();
  $_POST = array();
  $_COOKIE = array();
  $_REQUEST = array_merge($_GET, $_POST, $_COOKIE);

  $_SERVER["PATH"] = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin";
  $_SERVER["CONTENT_LENGTH"] = "0";
  $_SERVER["SCRIPT_NAME"] = "wp-cron.php";
  $_SERVER["REQUEST_URI"] = "/wp-cron.php";
  $_SERVER["QUERY_STRING"] = "";
  $_SERVER["REQUEST_METHOD"] = "GET";
  $_SERVER["SERVER_PROTOCOL"] = "HTTP/1.1";
  $_SERVER["GATEWAY_INTERFACE"] = "CGI/1.1";
  $_SERVER["REMOTE_PORT"] = "12345";
  $_SERVER["SCRIPT_FILENAME"] = "/var/vhosts/website/wp-cron.php"; // Change for your site
  $_SERVER["SERVER_ADMIN"] = "[email protected]"; // Change for your site
  $_SERVER["CONTEXT_DOCUMENT_ROOT"] = "/var/vhosts/website"; // Change for your site
  $_SERVER["CONTEXT_PREFIX"] = "";
  $_SERVER["REQUEST_SCHEME"] = "https";
  $_SERVER["DOCUMENT_ROOT"] = "/var/vhosts/website"; // Change for your site
  $_SERVER["REMOTE_ADDR"] = "192.168.1.1"; // Change for your site
  $_SERVER["SERVER_PORT"] = "443";
  $_SERVER["SERVER_ADDR"] = "127.0.0.1"; // Change for your site
  $_SERVER["SERVER_NAME"] = "www.website.com"; // Change for your site
  $_SERVER["SERVER_SOFTWARE"] = "Apache";
  $_SERVER["SERVER_SIGNATURE"] = "";
  $_SERVER["HTTP_COOKIE"] = "";
  $_SERVER["HTTP_UPGRADE_INSECURE_REQUESTS"] = "1";
  $_SERVER["HTTP_ACCEPT_LANGUAGE"] = "en-US,en;q=0.5";
  $_SERVER["HTTP_ACCEPT"] = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
  $_SERVER["HTTP_USER_AGENT"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0";
  $_SERVER["HTTP_ACCEPT_ENCODING"] = "gzip";
  $_SERVER["HTTP_CONNECTION"] = "close";
  $_SERVER["HTTP_HOST"] = "www.website.com"; // Change for your site
  $_SERVER["SSL_TLS_SNI"] = "www.website.com"; // Change for your site
  $_SERVER["HTTPS"] = "on";
  $_SERVER["SCRIPT_URI"] = "https://www.website.com/wp-cron.php"; // Change for your site
  $_SERVER["SCRIPT_URL"] = "/wp-cron.php";
  $_SERVER["PHP_SELF"] = "wp-cron.php";
  $_SERVER["REQUEST_TIME_FLOAT"] = microtime(true);
  $_SERVER["REQUEST_TIME"] = time();

  include "wp-cron.php";

Then change your cron command line to request your new PHP file instead of wp-cron.php

su -s /bin/bash www-data -c '/usr/bin/php /var/www/vhosts/website.com/my-cron.php > /dev/null 2>&1'

This should set up all the global variables needed by any WordPress plugin during a WP-CRON call. Note that this solution only works with the PHP method of directly calling wp-cron.php. It does not work with the WP-CLI method, since WP-CLI has no option to directly call a specific PHP file.

ANATOMY OF WP-CRON EXECUTION WITH DISABLE_WP_CRON UNDEFINED

The following traces a normal page request on a WordPress site without DISABLE_WP_CRON defined. No matter what page is requested, Apache directives in the site’s .htaccess file send the request to the /index.php file for execution, which loads the /wp-blog-header.php file as shown here…

/index.php: line 17
require( dirname( __FILE__ ) . '/wp-blog-header.php' );

Within the /wp-blog-header.php file, the /wp-load.php is loaded. A direct call to the /wp-cron.php file will load this same file, so the remainder of these steps also happen when a cron job is set up to directly execute /wp-cron.php…

/wp-blog-header.php: line 13
require_once( dirname( __FILE__ ) . '/wp-load.php' );

Within the /wp-load.php file, the /wp-config.php is loaded…

/wp-load.php: line 37
require_once( ABSPATH . 'wp-config.php' );

Inside the /wp-config.php file, the wp-settings.php file is loaded…

/wp-config.php: line 96
require_once(ABSPATH . 'wp-settings.php');

Within the wp-setting.php file, the /wp-includes/default-filters.php file is loaded…

/wp-settings.php: line 130
require( ABSPATH . WPINC . '/default-filters.php' );

At this point, something directly related to WP-CRON finally happens. In the following code, we see that, if DOING_CRON has not been defined already, then a call to wp_cron() is added to the ‘init’ action list…

/wp-includes/default-filters.php: line 332
if ( ! defined( 'DOING_CRON' ) ) {
	add_action( 'init', 'wp_cron' );
}

After all the code in /wp-includes/default-filters.php has finished executing, it returns to /wp-settings.php where the ‘init’ action list is executed…

/wp-settings.php: line 525
do_action( 'init' );

The ‘init’ action list will, among other things, call the wp_cron() function, which is located in the /wp-includes/cron.php file. In the first line of code for this function, we see that if /wp-cron.php was directly called, or DISABLE_WP_CRON has been defined as non-zero, then this function will return immediately.

/wp-includes/cron.php: line 746
function wp_cron() {
	// Prevent infinite loops caused by lack of wp-cron.php
	if ( strpos( $_SERVER['REQUEST_URI'], '/wp-cron.php' ) !== false ||
		( defined( 'DISABLE_WP_CRON' ) && DISABLE_WP_CRON ) ) {
			return 0;
	}

Since we are examining this code execution as a regular page request, the code continues executing. The code then checks to see if any WordPress cron jobs currently scheduled to run. If none are ready to run, the code returns immediately.

/wp-includes/cron.php: line 752
$crons = wp_get_ready_cron_jobs();
if ( empty( $crons ) ) {
	return 0;
}

The remainder of the wp_cron() function loops through the list of WordPress cron jobs that are ready to run and executes them.

The call_user_func( $schedules[ $hook ][‘callback’] ) segment of the code, seen below, seems to be an undocumented or obsolete method of immediately calling a scheduled callback function.  The $schedules array is created by a call to wp_get_schedules() which does not define any callback functions with the returned array of schedules. However, a plugin or theme developer could add a callback function to the $schedules array by using the ‘cron_schedules’ filter. If this happens, then wp_cron() could have a detrimental effect on page load times.

There is no documentation on how to use this feature in the ‘cron_schedules’ filter documentation.  Even the highly popular WP-Crontrol plugin is currently not capable of displaying callbacks embedded in the $schedules array.  As long as you keep WordPress, your theme and your plugins up to date, it is unlikely that you will encounter this type of cron job on your WordPress site.

The real work of wp_cron() happens in the call to spawn_cron(). Notice the break 2; statement following the call to spawn_cron(). This means that both foreach() loops are broken out of immediately; effectively reducing this entire section of code down to:

if ( array_key_first($crons) <= $gmt_time )
     $results[]  = spawn_cron( $gmt_time );

Here is the actual, more complex, code.

/wp-includes/cron.php: line 765
foreach ( $crons as $timestamp => $cronhooks ) {
	if ( $timestamp > $gmt_time ) {
		break;
	}
	foreach ( (array) $cronhooks as $hook => $args ) {
		if ( isset( $schedules[ $hook ]['callback'] ) && ! call_user_func( $schedules[ $hook ]['callback'] ) ) {
			continue;
		}
		$results[] = spawn_cron( $gmt_time );
		break 2;
	}
}

The function spawn_cron() is also located in /wp-includes/cron.php. This is also a fairly complex function with a lot going on in it, as seen below.

The code inside the test for ALTERNATE_WP_CRON being defined can be ignored. It is not recommended to run cron jobs this way; it rewrites the requested URL to include a GET parameter that triggers execution of cron jobs at the same time that it displays the requested page.

The rest of the function essentially verifies there are cron jobs ready to run, uses set_transient() to create a lock to prevent more than one job per minute from running, then calls the wp_remote_post() function to do a non-blocking web request to the /wp-cron.php file!

The /wp-cron.php file will handle any remaining cron jobs that need to run. Note that the wp_remote_post() argument named ‘blocking’ is set to false. This will cause the request to spawn into a separate process and execution of the code in spawn_cron() can continue immediately.

It is also important to note that the URL has a doing_wp_cron parameter added as a query string. But unlike most tutorials that suggest a command line that adds ?doing_wp_cron as a query string parameter with no value, this parameter is assigned a timestamp value which is used to validate the call to wp_cron(). If this validation fails, the cron job does not run.

/wp-includes/cron.php: line 629
function spawn_cron( $gmt_time = 0 ) {
	if ( ! $gmt_time ) {
		$gmt_time = microtime( true );
	}

	if ( defined( 'DOING_CRON' ) || isset( $_GET['doing_wp_cron'] ) ) {
		return false;
	}

	/*
	 * Get the cron lock, which is a Unix timestamp of when the last cron was spawned
	 * and has not finished running.
	 *
	 * Multiple processes on multiple web servers can run this code concurrently,
	 * this lock attempts to make spawning as atomic as possible.
	 */
	$lock = get_transient( 'doing_cron' );

	if ( $lock > $gmt_time + 10 * MINUTE_IN_SECONDS ) {
		$lock = 0;
	}

	// don't run if another process is currently running it or more than once every 60 sec.
	if ( $lock + WP_CRON_LOCK_TIMEOUT > $gmt_time ) {
		return false;
	}

	//sanity check
	$crons = wp_get_ready_cron_jobs();
	if ( empty( $crons ) ) {
		return false;
	}

	$keys = array_keys( $crons );
	if ( isset( $keys[0] ) && $keys[0] > $gmt_time ) {
		return false;
	}

	if ( defined( 'ALTERNATE_WP_CRON' ) && ALTERNATE_WP_CRON ) {
		// ... not used.
	}

	// Set the cron lock with the current unix timestamp, when the cron is being spawned.
	$doing_wp_cron = sprintf( '%.22F', $gmt_time );
	set_transient( 'doing_cron', $doing_wp_cron );

	$cron_request = apply_filters(
		'cron_request',
		array(
			'url'  => add_query_arg( 'doing_wp_cron', $doing_wp_cron, site_url( 'wp-cron.php' ) ),
			'key'  => $doing_wp_cron,
			'args' => array(
				'timeout'   => 0.01,
				'blocking'  => false,
				/** This filter is documented in wp-includes/class-wp-http-streams.php */
				'sslverify' => apply_filters( 'https_local_ssl_verify', false ),
			),
		),
		$doing_wp_cron
	);

	$result = wp_remote_post( $cron_request['url'], $cron_request['args'] );
	return ! is_wp_error( $result );
}

CONCLUSION

As shown, a deeper understanding of why your cron job is failing can help you fix it. But keep in mind that if all else fails getting your WordPress web site to runs its scheduled tasks with a real cron job, removing DISABLE_WP_CRON and disabling the cron job is a viable alternative.