Fix problems caused by DISABLE_WP_CRON

OVERVIEW

Many online tutorials recommend disabling WP-CRON and using a “real” cron job to improve WordPress performance. This doesn’t always work. You may notice after following this advice that automated tasks provided by plug-ins, and WordPress itself, stop working. This article discusses what can go wrong when using the DISABLE_WP_CRON constant, and how to fix it.

BACKGROUND ON DISABLE_WP_CRON

Many online tutorials give the same advice on disabling WP-CRON in WordPress and using a cron job instead. What does this mean? WordPress and its plug-ins have tasks they need to perform in the background on a regular schedule. For example, the WP Super Cache plug-in needs to delete old cached web pages and generate new cached versions of web pages from time to time.  The WP-CRON system in WordPress handles this by checking for, and performing, any scheduled tasks.

The wp-cron.php file has code in it to run these scheduled tasks at the appropriate times. To make WordPress work on different operating systems, developers decided to execute code in wp-cron.php for every request processed. There are several problems with this:

  • If your web site isn’t very popular, scheduled tasks may be late in running. Scheduled tasks only run when your web site receives a web page requests. In the real world this is not a problem on a public web site. There are plenty of ‘bots accessing your web site 7/24. But if you are using a robots.txt file or other methods of stopping robots from crawling your site, it could still be a problem.
  • A variation of the first problem can occur if you are using an external caching service like CloudFlare. On a very strongly cached web site, wp-cron.php might not get called for days.
  • Common sense would seem to indicate that running wp-cron.php on each and every web page request is not very efficient. Can you get better performance if there were a better way to take care of these background tasks?  Most people won’t notice a difference in performance after setting up a real cron job for WordPress.
  • On a web site with very heavy traffic, preventing wp-cron.php from running on every web page request can improve performance.
  • Since wp-cron.php spawns a new web request, it can double the number of requests to your web site. This is an important consideration if your hosting plan has monthly limits on the number of requests served. Switching to a cron job that executes wp-cron.php every 10 minutes could significantly cut requests to your web site.

Better performance is what most online tutorials cite to convince you to use DISABLE_WP_CRON with a cron job. However, during a normal request for a web page to a WordPress web site, wp_cron() will exit if there are any other calls to wp_cron() running, or if there are no crons jobs currently scheduled, or if any of several invalid or special conditions exist. Also, if a final decision is made to execute the code in wp-cron.php, the file is loaded as a spawned request so that it will not delay processing of the original user web page request.


BASIC SETUP FOR DISABLE_WP_CRON

First, add the following to your wp-config.php file, somewhere near the top. Make sure DISABLE_WP_CRON is not set anywhere else in wp-config.php.

define('DISABLE_WP_CRON', true);

The next step is to add a cron job to your system. How you do this varies depending on your web hosting provider. Some web hosting providers don’t give you an option to create cron jobs. In this case, stop reading now, and go find a better web hosting service.

If you have a co-located server or a virtual private server (VPS, or sometimes referred to as a Cloud Server), then familiarize yourself with using crontab to add cron jobs to your system.

If your web site runs on Windows, familiarize yourself with the AT command or the Task Scheduler.

TROUBLESHOOTING DISABLE_WP_CRON PROBLEMS

  • Remove command output redirects

    Online tutorials all suggest adding > /dev/null 2>&1 to the end of each suggested wget command, or -r -nd −−delete-after before the url parameter. Both of these discard any standard output, and any error messages, from the command. This is not a problem if everything is working correctly. However, if anything is wrong, you won’t see any indication of it. While troubleshooting cron jobs it is best to remove these directives from wget commands. This will allow you to review any output from wget. Use the crontab -e command and the MAILTO= setting to set an email address to send any cron command output to.
    Another option is to add -O – to your wget command to send all output to stdout.  This will allow you to examine any output from wget as it happens.  Just remember to remove the > /dev/null output redirect from your command line.

  • Test cron jobs from the shell command line

    You can test your cron job command line by logging in to your server and executing it from the shell. However, if your web server runs under a different account, you may not see errors that occur when cron runs the command. Unless otherwise configured, Linux with Apache web server runs under an account named www-data and no login shell is defined for that account. To temporarily change that, use the su (substitute user) command and specify a default shell:

    su -s /bin/bash www-data -c 'your commands here'

    With shared web server hosting plans, you most likely are logging in using the account your web site executes in. So running a cron job at the command line in your account will be exactly like running it as a cron job.

  • Enable WordPress debugging and logging

    Enable WordPress debugging and logging so that you can review any PHP errors that occur when your cron job runs. Add the following lines to your /wp-config.php file.

    // Enable WP_DEBUG mode
    define('WP_DEBUG', true);
    
    // Enable Debug logging to the /wp-content/debug.log file
    define('WP_DEBUG_LOG', true);
    
    // Disable display of errors and warnings
    define('WP_DEBUG_DISPLAY', false);
    
  • File Permissions and Ownership

    If you have automatic updates to WordPress, your plugins, and/or theme that never seem to update with the cron job, it could be a file permissions or ownership problem. This can be a problem if you are doing file operations while logged into a shell account that is different from the account your web site runs in (e.g. on a VPS or “cloud server”). A good troubleshooting tool to have is a command alias that makes fixing the permissions and ownership of your web site files and directories easier, as shown below. Adjust the paths, permissions, and owner as needed for your site and server…

    alias fixperms='find /var/www/vhosts/website.com/ -type d -exec chmod 755 {} \; && find /var/www/vhosts/website.com/ -type f -exec chmod 644 {} \; && chown -R www-data:www-data  /var/www/vhosts/website.com/'
    
  • WP-CRON plugins and WP-CLI

    Sometimes plugins or themes don’t clean up after themselves when you uninstall them. They leave invalid wp-cron jobs in the queue. This could cause errors or timeouts; especially if a wp-cron job is trying to access an update web site that no longer exists. Several plugins, such as the popular WP Crontrol, let you examine the list of jobs in wp-cron’s queue, and delete invalid jobs. You can also use the wp cron event list and wp cron event unschedule commands if you have access to WP-CLI on your web server.

  • cURL error 6: Could not resolve host:

    If you are using the WP Crontrol plugin and see a notice like this…

    The More information link gives you this advice…

    In this case, make sure that your web server’s DNS settings are correct. You should be able to ping your web server, by hostname, from a shell on the server and see the correct IP address and hostname for your web server. For example…

    ping my.domain.com
    
    PING my.domain.com (192.168.1.100) 56(84) bytes of data.
    64 bytes from my.domain.com (192.168.1.100): icmp_seq=1 ttl=64 time=0.013 ms
    64 bytes from my.domain.com (192.168.1.100): icmp_seq=2 ttl=64 time=0.011 ms
    64 bytes from my.domain.com (192.168.1.100): icmp_seq=3 ttl=64 time=0.013 ms
    

    If you don’t see output like this, check your /etc/resolv.conf file to make sure a DNS server is configured for hostname resolution by your web server. For example…

    cat  /etc/resolv.conf
    
    nameserver 1.1.1.1
    options timeout:1

    But there is a more subtle possibility here. Are you running your web server on a network within a private IP address range? Is our web server behind a firewall or NAT device?

    In these cases, the problem may be that there is no DNS record, or no correct DNS record that your WordPress site can retrieve. In this case, add a new record to your web server’s /etc/host or C:\Windows\System32\drivers\host file (depending on your platform). Adding my.domain.com to your host file may look like this…

    cat /etc/host
    
    127.0.0.1 localhost
    192.168.1.100 my.domain.com

    Test again with ping to verify it is working. After this, the notice in WP Crontrol should disappear and your WP-Cron jobs might start working again.

  • Could Not Complete A Loopback Request

    If the Site Health screen in WordPress has an error message of “Your site could not complete a loopback request” such as the following…

    …this will prevent WP_CRON from functioning. If you have SSH access to a shell on your web server, try the following command, replacing mydomain.com with your web site’s host name…

    curl -v https://mydomain.com/

    The output from this command should provide you with more information to help you solve this problem. Common problems could be issues with DNS resolution, security plugins, firewall rules, lack of HTTP/2 support, or your web hosting provider blocking loopback requests or the use of cURL.

    Another strange problem that has been seen on shared web hosting accounts is that WP_CRON requests might not come from your web server’s IP address. In this case, whitelisting your web server IP address won’t not allow WP_CRON or loopback requests past your firewall if it looks suspicious. If you have access to your web site’s log files, search the logs for requests with a user-agent in the following format…

    "WordPress/6.2.2; https://www.mydomain.com"

    If the IP address for these requests is in your hosting company’s network, then you can try whitelisting that IP address in your firewall setup and test your loopback requests again.

COMMON CRON COMMANDS AND THEIR PROBLEMS

Using wget

One of the most common cron job commands suggested in online tutorials looks like this…

wget -q -O - https://www.website.com/wp-cron.php?doing_wp_cron > /dev/null 2>&1

There is no need to add the ?doing_wp_cron query string.  WordPress uses this query string value to pass a lock code for WP-CRON from the spawn_cron() function.  It won’t hurt, but it also doesn’t do anything.  Simply remove the query string, like this..

wget -q -O - https://www.website.com/wp-cron.php > /dev/null 2>&1

However, there can still be problems with this cron command, such as…

  • Problems with the default settings used by wgetGlobal defaults for wget could vary by distribution or be overridden by defaults set in the home directory for the user account executing the wget command
  • An error while verifying an SSL certificate.
  • Security software on your web server, or a WordPress firewall plugin, interpreting a request lacking typical browser data as a hacking attempt.

The following command line will prevent the NinjaFirewall plugin from blocking a call by wget to wp-cron.php. It will also prevent problems with self-signed certificates, timeouts, and other problems.

wget -q -O - --ignore-length --inet4-only --no-check-certificate --no-cache \
   --timeout=60 --user-agent="Mozilla/5.0" --header="Accept: */*;q=0.8" \
   --header="Accept-Language: en-US,en;q=0.5" \
   https://www.website.com/wp-cron.php > /dev/null 2>&1

 

Using PHP-CLI or WP-CLI

Some online tutorials suggest one of two other cron command lines. One uses the command line version of php to directly call wp-cron.php.

/usr/bin/php /var/www/vhosts/website.com/wp-cron.php > /dev/null 2>&1

The other uses wp-cli (a command line interface for WordPress).

/usr/local/bin/wp cron event run --due-now --quiet --path=/var/www/vhosts/website.com > /dev/null 2>&1

Note that both of these commands require your cron job to be running on the same server as your web site (or at least have access to the file system of your web site).

Both of these methods have the same problem. Since the code runs from the command line, no web browser or web server software is involved. Because of this, commonly used global variables such as $_SERVER[], $_REQUEST[], $_GET[], $_POST[] do not exist.

An indicator of this problem is if your WordPress debug log has lines that contain: Undefined index on $_SERVER. Unfortunately, many plugin developers assume that the global array $_SERVER always exists and is always valid. Even some parts of the WordPress source code still use these global arrays without first checking for validity.

Another problem to watch out for with these two methods is the user account that runs these commands.  You should not use the root account.  Ideally, you should execute these commands with the same account that your web site process is running in.  You can use the su -s /bin/bash www-data -c method mentioned above to substitute users before running the cron commands.

These, and other problems, caused by command line access to WordPress files are explained here.

If you are lucky, the only global variables that are needed are $_SERVER[‘HTTP_HOST’] and $_SERVER[‘SERVER_NAME’]. If that is true, you can get your cron job working again by exporting environment variables HTTP_HOST and SERVER_NAME before your command line as follows (replace www.website.com and the file paths with the correct settings for your site)…

su -s /bin/bash www-data -c 'export HTTP_HOST="www.website.com"; export SERVER_NAME=$HTTP_HOST; /usr/bin/php /var/www/vhosts/website.com/wp-cron.php > /dev/null 2>&1'

For the wp-cli method use…

su -s /bin/bash www-data -c 'export HTTP_HOST="www.website.com"; export SERVER_NAME=$HTTP_HOST; /usr/local/bin/wp cron event run --due-now --quiet --path=/var/www/vhosts/website.com > /dev/null 2>&1'

When the Apache web server receives a request for a web page, it stores request information into environment variables. Exporting these two environment variables tricks PHP into thinking that the Apache web server received a real request and set these environment variables. PHP will then place them into the $_SERVER global variable array. Then the PHP code in WordPress can access them without errors.

If a plugin is causing problems because it is using more than these two variables, try contacting the developer and ask if they can test their plugin with WP-CLI. If the developer won’t help you, you can create a custom PHP file to set up the global variables needed, and include the wp-cron.php file as the last line of the file. Then change your cron command to call the new PHP file instead.

For example, create a new file “my-cron.php” in your web site root directory and add the following code to it…

<?php
  $_SERVER = array();
  $_GET = array();
  $_POST = array();
  $_COOKIE = array();
  $_REQUEST = array_merge($_GET, $_POST, $_COOKIE);

  $_SERVER["PATH"] = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin";
  $_SERVER["CONTENT_LENGTH"] = "0";
  $_SERVER["SCRIPT_NAME"] = "wp-cron.php";
  $_SERVER["REQUEST_URI"] = "/wp-cron.php";
  $_SERVER["QUERY_STRING"] = "";
  $_SERVER["REQUEST_METHOD"] = "GET";
  $_SERVER["SERVER_PROTOCOL"] = "HTTP/1.1";
  $_SERVER["GATEWAY_INTERFACE"] = "CGI/1.1";
  $_SERVER["REMOTE_PORT"] = "12345";
  $_SERVER["SCRIPT_FILENAME"] = "/var/vhosts/website/wp-cron.php"; // Change for your site
  $_SERVER["SERVER_ADMIN"] = "[email protected]"; // Change for your site
  $_SERVER["CONTEXT_DOCUMENT_ROOT"] = "/var/vhosts/website"; // Change for your site
  $_SERVER["CONTEXT_PREFIX"] = "";
  $_SERVER["REQUEST_SCHEME"] = "https";
  $_SERVER["DOCUMENT_ROOT"] = "/var/vhosts/website"; // Change for your site
  $_SERVER["REMOTE_ADDR"] = "192.168.1.1"; // Change for your site
  $_SERVER["SERVER_PORT"] = "443";
  $_SERVER["SERVER_ADDR"] = "127.0.0.1"; // Change for your site
  $_SERVER["SERVER_NAME"] = "www.website.com"; // Change for your site
  $_SERVER["SERVER_SOFTWARE"] = "Apache";
  $_SERVER["SERVER_SIGNATURE"] = "";
  $_SERVER["HTTP_COOKIE"] = "";
  $_SERVER["HTTP_UPGRADE_INSECURE_REQUESTS"] = "1";
  $_SERVER["HTTP_ACCEPT_LANGUAGE"] = "en-US,en;q=0.5";
  $_SERVER["HTTP_ACCEPT"] = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
  $_SERVER["HTTP_USER_AGENT"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0";
  $_SERVER["HTTP_ACCEPT_ENCODING"] = "gzip";
  $_SERVER["HTTP_CONNECTION"] = "close";
  $_SERVER["HTTP_HOST"] = "www.website.com"; // Change for your site
  $_SERVER["SSL_TLS_SNI"] = "www.website.com"; // Change for your site
  $_SERVER["HTTPS"] = "on";
  $_SERVER["SCRIPT_URI"] = "https://www.website.com/wp-cron.php"; // Change for your site
  $_SERVER["SCRIPT_URL"] = "/wp-cron.php";
  $_SERVER["PHP_SELF"] = "wp-cron.php";
  $_SERVER["REQUEST_TIME_FLOAT"] = microtime(true);
  $_SERVER["REQUEST_TIME"] = time();

  include "wp-cron.php";

Then change your cron command line to request your new PHP file instead of wp-cron.php

su -s /bin/bash www-data -c '/usr/bin/php /var/www/vhosts/website.com/my-cron.php > /dev/null 2>&1'

This should set up all the global variables needed by any WordPress plugin during a WP-CRON call. Note that this solution only works with the PHP method of directly calling wp-cron.php. It does not work with the WP-CLI method, since WP-CLI has no option to directly call a specific PHP file.

ANATOMY OF WP-CRON EXECUTION WITHOUT DISABLE_WP_CRON DEFINED

The following traces a normal page request on a WordPress site without DISABLE_WP_CRON defined. For all page requests, Apache directives in the site’s .htaccess file send the request to the /index.php file for execution, which loads the /wp-blog-header.php file as shown here…

/index.php: line 17
require( dirname( __FILE__ ) . '/wp-blog-header.php' );

The /wp-blog-header.php file, loads the /wp-load.php file. A direct call to /wp-cron.php also loads this file. So the remaining steps also happen when a cron job directly executes /wp-cron.php…

/wp-blog-header.php: line 13
require_once( dirname( __FILE__ ) . '/wp-load.php' );

The /wp-load.php file then loads the /wp-config.php file…

/wp-load.php: line 37
require_once( ABSPATH . 'wp-config.php' );

Then the /wp-config.php file loads the wp-settings.php file…

/wp-config.php: line 96
require_once(ABSPATH . 'wp-settings.php');

The wp-setting.php file loads the /wp-includes/default-filters.php file…

/wp-settings.php: line 130
require( ABSPATH . WPINC . '/default-filters.php' );

Finally, something directly related to WP-CRON happens. In the following code, we see that, if DOING_CRON has not been defined already, then a call to wp_cron() is added to the ‘init’ action list…

/wp-includes/default-filters.php: line 332
if ( ! defined( 'DOING_CRON' ) ) {
	add_action( 'init', 'wp_cron' );
}

After all the code in /wp-includes/default-filters.php has finished executing, it returns to /wp-settings.php where the ‘init’ action list executes…

/wp-settings.php: line 525
do_action( 'init' );

The ‘init’ action list will, among other things, call the wp_cron() function, which is located in the /wp-includes/cron.php file. The first line of code in this function tests if /wp-cron.php was directly called, or if DISABLE_WP_CRON is non-zero. If either is true then this function ends immediately.

/wp-includes/cron.php: line 746
function wp_cron() {
	// Prevent infinite loops caused by lack of wp-cron.php
	if ( strpos( $_SERVER['REQUEST_URI'], '/wp-cron.php' ) !== false ||
		( defined( 'DISABLE_WP_CRON' ) && DISABLE_WP_CRON ) ) {
			return 0;
	}

Since we are examining this code execution as a regular page request, the code continues executing. The code then checks for any WordPress cron jobs currently due to run, or past due to run. If none are ready to run, the code returns immediately.

/wp-includes/cron.php: line 752
$crons = wp_get_ready_cron_jobs();
if ( empty( $crons ) ) {
	return 0;
}

The remainder of the wp_cron() function loops through the list of WordPress cron jobs that are ready to run and executes them.

The call_user_func( $schedules[ $hook ][‘callback’] ) segment of the code, seen below, seems to be an undocumented or obsolete method of immediately calling a scheduled callback function.  The $schedules array is created by a call to wp_get_schedules() which does not define any callback functions with the returned array of schedules. However, a plugin or theme developer could add a callback function to the $schedules array by using the ‘cron_schedules’ filter. If this happens, then wp_cron() could have a detrimental effect on page load times.

There is no documentation on how to use this feature in the ‘cron_schedules’ filter documentation.  Even the highly popular WP-Crontrol plugin is currently not capable of displaying callbacks embedded in the $schedules array.  If WordPress, your theme and plugins are up-to-date, it is unlikely you will encounter this type of cron job.

The real work of wp_cron() happens in the call to spawn_cron(). Notice the break 2; statement following the call to spawn_cron(). This means that both foreach() loops break out immediately; effectively reducing this entire section of code down to:

if ( array_key_first($crons) <= $gmt_time )
     $results[]  = spawn_cron( $gmt_time );

Here is the actual, more complex, code.

/wp-includes/cron.php: line 765
foreach ( $crons as $timestamp => $cronhooks ) {
	if ( $timestamp > $gmt_time ) {
		break;
	}
	foreach ( (array) $cronhooks as $hook => $args ) {
		if ( isset( $schedules[ $hook ]['callback'] ) && ! call_user_func( $schedules[ $hook ]['callback'] ) ) {
			continue;
		}
		$results[] = spawn_cron( $gmt_time );
		break 2;
	}
}

The function spawn_cron() is also located in /wp-includes/cron.php. This is also a fairly complex function with a lot going on in it, as seen below.

The code that tests if ALTERNATE_WP_CRON is defined can be ignored. It is not recommended to run cron jobs this way.  This method rewrites the requested URL to add a GET parameter that triggers execution of cron jobs.  Then it forwards the request to the modified URL.

The rest of the function essentially verifies there are cron jobs ready to run, uses set_transient() to create a lock to prevent more than one job per minute from running, then calls the wp_remote_post() function to do a non-blocking web request to the /wp-cron.php file!

The /wp-cron.php file will handle any remaining cron jobs that need to run. Note that the wp_remote_post() argument named ‘blocking’ is set to false. This will cause the request to spawn into a separate process and execution of the code in spawn_cron() can continue immediately.

It is also important to note that the URL has a doing_wp_cron parameter added as a query string. But unlike most tutorials that suggest a command line that adds ?doing_wp_cron as a query string parameter with no value, this parameter is assigned a timestamp value which is used to validate the call to wp_cron(). If this validation fails, the cron job does not run.

/wp-includes/cron.php: line 629
function spawn_cron( $gmt_time = 0 ) {
	if ( ! $gmt_time ) {
		$gmt_time = microtime( true );
	}

	if ( defined( 'DOING_CRON' ) || isset( $_GET['doing_wp_cron'] ) ) {
		return false;
	}

	/*
	 * Get the cron lock, which is a Unix timestamp of when the last cron was spawned
	 * and has not finished running.
	 *
	 * Multiple processes on multiple web servers can run this code concurrently,
	 * this lock attempts to make spawning as atomic as possible.
	 */
	$lock = get_transient( 'doing_cron' );

	if ( $lock > $gmt_time + 10 * MINUTE_IN_SECONDS ) {
		$lock = 0;
	}

	// don't run if another process is currently running it or more than once every 60 sec.
	if ( $lock + WP_CRON_LOCK_TIMEOUT > $gmt_time ) {
		return false;
	}

	//sanity check
	$crons = wp_get_ready_cron_jobs();
	if ( empty( $crons ) ) {
		return false;
	}

	$keys = array_keys( $crons );
	if ( isset( $keys[0] ) && $keys[0] > $gmt_time ) {
		return false;
	}

	if ( defined( 'ALTERNATE_WP_CRON' ) && ALTERNATE_WP_CRON ) {
		// ... not used.
	}

	// Set the cron lock with the current unix timestamp, when the cron is being spawned.
	$doing_wp_cron = sprintf( '%.22F', $gmt_time );
	set_transient( 'doing_cron', $doing_wp_cron );

	$cron_request = apply_filters(
		'cron_request',
		array(
			'url'  => add_query_arg( 'doing_wp_cron', $doing_wp_cron, site_url( 'wp-cron.php' ) ),
			'key'  => $doing_wp_cron,
			'args' => array(
				'timeout'   => 0.01,
				'blocking'  => false,
				/** This filter is documented in wp-includes/class-wp-http-streams.php */
				'sslverify' => apply_filters( 'https_local_ssl_verify', false ),
			),
		),
		$doing_wp_cron
	);

	$result = wp_remote_post( $cron_request['url'], $cron_request['args'] );
	return ! is_wp_error( $result );
}

CONCLUSION

As shown, a deeper understanding of why your cron job is failing can help you fix it. But keep in mind that if all else fails getting your WordPress web site to runs its scheduled tasks with a real cron job, removing DISABLE_WP_CRON and disabling the cron job is a viable alternative.