Surviving a Wordpress Traffic Surge

Published 2018-11-20

A popular blogger reached out to us recently for help while his sites were under a surge of traffic from press coverage, and simultaneously became a Denial of Service (DOS) target by some people who wanted to silence his voice.

When your site receives a sudden amount of attention, it can be difficult to distinguish a malicious DOS attack from a spontaneous outage due to the simply overwhelming amount of traffic that front page coverage can cause. If you're not prepared to handle this sort of load, your site can be come a victim of it's own success, and any publisher would hate to miss out on their 15 minutes of fame because their website crashed when it made it big in the news or in syndicated press coverage! In the tech industry have a name for this phenomenon, the Slashdot Effect, named after the popular sysadmin news website slashdot.org which was known to crash the websites of bloggers when their content suddenly hit the front page.

Please God don't let it crash 

If your site is lucky enough to become so popular, perhaps you have something controversial to say, or a not-so-friendly competitor is lurking, this kind of attention also begs notice by the so-called "Russian hackers" (pro tip: they are usually not actually Russian) and other malicious actors, which can cause a disruption to your publishing and escalate the intensity of the legitimate traffic which is already straining your application server.

Knowing Enough To Be Dangerous

This particular customer had originally tried to manage this problem himself by migrating to a larger server from his VPS company, and relying on CloudFlare services to both cache the traffic served from his site and protect him from malicious DOS traffic. Unfortunately, he had come to the end of his wits when moving to the new server actually made his site(s) go offline into timeout hell during all but the silent hours of midnight - not for failing to configure them properly to run his Wordpress domains - but for making the mistake of solving the wrong problem first.

When Your CDN Is Not Caching

If you're trying to optimize your Wordpress site, the first thing you ought to do is check your cache headers. Even though this site was running with a CloudFlare (tm) Cache enabled, it's not going to cache anything if your upstream application is sending Cache-Control: no-cache headers back to it!

Wordpress in particular can be complex to identify in the code where certain behaviors (such as sending no-cache headers) are originating, since so much configuration comes from the database and can be scattered throughout a myriad of plugin files. When your site is under heavy load right now, you don't have time to debug all of this noise to fix the cache headers on your home page. With this little patch of code (add it to your theme's functions.php), you can intercept the headers before Wordpress renders the page and insert your own cache control logic. (See the Wordpress documentation for the send_headers hook)

function my_headers_filter() {
  header_remove("Pragma");
  header_remove("Cache-Control");
  header("Cache-Control: public, max-age=3600");
}
add_action( 'send_headers', 'my_headers_filter' );

Running MySQL at 7200 RPM

One mistake our beloved customer made is that even though he migrated his server to a bigger machine, his MySQL was now running on a spinning hard drive and his performance was severely degraded. In the <Current Year> there's no reason to run a random access database on anything but a SSD.

So Much Work, So Few Cores

This should be obvious, but make sure you have enough cores to run your site if you're running a full LAMP stack on your own. A good rule of thumb: at least one core for every letter in the acronym. When you try to squeeze too many processes in to a few cores, you'll get what's called CPU 'thrashing' - the degradation in performance that happens when the CPU is overloaded with context switching between processes so much by the scheduler that no single process has much opportunity to actually get any work done before it loses it's time slot. Your CPU has a cache of it's own, which is a major major major impactor on your overall performance if those CPU caches can stay hot - you want each CPU core doing it's own dedicated work so it can take advantage of its cache.

The Apache Web Server Is Not An Attack Helicopter

While Apache was a great web server and a pioneer of the internet, there are better options out there that have been streamlined for performance with high concurrency.  Apache can perform well, but it takes a lot of intimate tuning to get right. If you're using cPanel on your server, you might have a hard time getting away from Apache.

Optimizing the PHP Runtime

Make sure to review php.ini and tailor it to your particular environment. You should be running PHP 7+ with Wordpress, so make sure that opcache.enable is set in your production instances. Also do not neglect your php-fpm pool configuration. In particular pay attention to the process manager 'pm' and 'pm.max_children' configs. You don't want so many php-fpm processes to spawn that your CPU will start thrashing out of control, but you also don't want to under-provision and leave your CPUs idle either. This number should reflect a balance between the amount of i/o each php process has to do (more i/o -> more processes) and the amount of cores you have available.

Running Isolated PHP Containers

If you're hosting multiple sites on your server, consider running each of your sites in it's own Docker container. This will provide a harder level of isolation in case any hacker manages to exploit one of your sites, and it does happen from time to time, since there is a large attack surface for Wordpress with all of the plugins, and the opportunities for vulnerability are there even in Wordpress core. The last thing you want is an attacker getting a shell on one site only to pwn all of your other domains and databases at the same time. You'll also get the benefits of managing your PHP installations via a Dockerfile which can make your servers more neatly organized, portable, and upgradeable.

Load Testing Tools

ab (ApacheBench) and httperf are two easy (and free!) to use command line http load testing tools you can use to analyze the performance impact of your changes. Make good use of them! Measurement is key to effective optimization.

Conclusion

Running a Wordpress site (or any other blog software) is something that's become very simple for an end user to set up, and it's a great tool for people to start publishing with. On this site, we are using Ghost software to power our blog. If you're running a business-critical site, or expecting to hit a large surge of publicity, make sure you are prepared to handle the traffic when the timing is critical. The benefits of prepping your site to handle lots of traffic extend into making your site load very fast as well - and you and your audience will notice and appreciate the swift loading times.

Professional Support

Arctype Corp. is here if you need emergency support with your Wordpress or other web application software in a time of need, or want the help of an expert advisor to tune your servers for better performance at an affordable price. Give us a shout at sales@arctype.co.

Discussion