10 ways to improve your Web server for better Drupal performance

10 improvements and 10 time wasters. Will take some time to transfer my notes from testing logs.

10 improvements

1. Switch from shared hosting to a VPS

There is almost nothing you can do to improve performance on shared Web site hosting because there is very little you can control and any change you make for your site can be buried in the workload created by other sites.

A VPS gives you more control because you have your own mini server within a virtual server. You can make most of the changes mentioned on this page when you upgrade to a VPS. Some changes might be easier when you expand to a dedicated server.

Most VPS installations start with a Linux installation then the Virtual Server software runs under Linux and your VPS runs under the VM software. You get whatever Linux is used for the server. The hosting company should update their Linux with the latest security patches and those updates should flow through to your Linux. Everything else should be under your control.

The commercial WHM/cPanel and the open source Webmin/Usermin/Virtualmin families are the two main choices for software to manage a VPS. You can make many changes within the administration pages provided by those families of software.

2. Switch on APC

A PHP opcode cache is the best way to speed up PHP. See PHP performance. You can easily switch one on using cPanel and equivalents. Your Web hosting supplier should offer to switch it on if they installed PHP. Some bare metal hosting does not have PHP or anything installed and your first step is to install Webmin then Apache/MySQL/PHP then switch on the right options for PHP, including an opcode cache.

3. Choose the right hosting provider

Some Web hosting providers overload their servers to squeeze out some extra profit. Some do not help you with configuration or finding bottlenecks. The difference between a good service provider and a bad one can be as little as $10 per month. The good service might handle twice as many users before you have to upgrade to a larger account, saving you more than you spend on the better service.

4. Switch on Apache compression

Drupal can compress Web pages if Apache, or your choice of Web server, does not compress pages. Drupal cannot compress things outside the pages delivered by Drupal. your Web server can compress everything that will benefit from compression. Switch on compression in your Web server and switch off page compression in Drupal.

5. Tune your database

Databases have an optimise function. It is usually run once when you first create a database but never again. Run optimise after you load in a lot of data. You might start with one optimise per week then move to once per month.

The next step is to look through common database accesses and make sure those accesses use suitable indexes. I can do this by hand based on extensive experience. There are tools to help you and they vary by database. The free open source databases usually have cryptic reports you can run by hand then the commercial analysis tools simply run the free reports and reformat the reports for readability.

There are programs to perform more extensive and complex investigations but they typically look at stuff you only need to change when you have giant databases spread over many servers. Your Web site hosting company will help you with the basics and they will stop you docing anything radical that might damage performance, reliability, or security for their other customers.

6. Transfer the contents of .htaccess files to the Apache .conf file

Apache has several configuration files linked together. The main one is httpd.conf and that includes Web site configurations from httpd-vhosts.conf. httpd-vhosts.conf is read once when Apache starts up. .htaccess files are read for every page request. You save file reads by removing the .htaccess files.

Some Drupal modules install special .htaccess files in some directories and you have to do something to remove those modules or modify those modules to not use .htaccess files. You can also set up special directory level definitions in httpd.conf.

Ignore those silly Web pages that talk about hybrid schemes where you move some settings out of some .htaccess files but not the whole file. The authors clearly have never worked on Apache. do not know the power of the httpd.conf file, and have never measured the impact of altering .htaccess files.

9. Use a local hosting service

Use a hosting service with a data centre local to your customers. They will get better response times for page accesses. Analyse the geographic locations of your customers or users. Try for the best response for the most users during the peak workload.

10. Content Delivery Network

A CDN, Content Delivery Network, is expensive and is one way to offload delivery of videos, if you have a lot of videos on your Web site. When you have to deliver large files, you set up a server in each of your major use areas. Think of a Web site with a big following in Germany, California, and Tasmania. You set up one server in Berlin, one in Los Angeles, and one in Hobart. You replicate the large files to all three servers. Easy. Now think of the same site serving 80 countries. Passing the video to an external service could be easier than setting up you own servers and might be cheaper than hiring someone to manage the capacity of each server.

The last time I looked at a CDN for a major project, the main CDNs were mainly USA based and charged a lot of money to deliver content from USA, a place where servers were, at the time, relatively cheap. None of the CDNs matched the usage pattern required for that project. Some of the CDNs are now in more countries. Check carefully exactly which countries they service from local servers and the response time in those countries where you need good service.

If you do not get the right match, the CDNs create almost as many problems as they solve. Usage reporting can be a problem. Watch out for sudden cost overruns. They may not charge the same rate in every country and your service may not cover the cost in some countries.

If your site is popular, photographs and other small files are often cached in all the reverse proxy servers around the world. Videos and Linux distribution CD images are too big to be cached. Linux distribution CDs are already replicated to mirror servers all around the world. That leaves videos as the big market and training videos, with a reasonable fee attached, are profitable enough to pay for the use of a CDN.

10 this should already be done items

1. Serve through a Reverse Proxy server

Reverse proxy servers are everywhere on the Web. They service static pages from a local cache in memory or on disk. Your Web browser contains something similar and stores a local copy of each Web page, CSS file, image, and Javascript file to save the overhead of rereading the data across the Internet. Little records, called HTTP headers, tell the cache to save the page for a minute or a day. The whole process is repeated throughout the Web.

Your ISP uses a giant cache to reduce expensive downloads from the Web. When you access a Web site, you often see a local copy of content unless the Web site uses incorrect HTTP header settings. If you set the right HTTP headers in Drupal, your Web pages, CSS files, Images, and other downloads should be cached for most anonymous users.

Pressflow, Mercury, Pantheon and other magic performance packages often achieve much of their effect purely by changing the default settings for HTTP headers.

2. Use Memcache

When you access a Web page, the contents might be cached in your Web browser. If not, they might be cached in your ISP's reverse proxy server. If not, Drupal might cache the content. Whatever Drupal decides to do, if the page makes a database request, the request will be cached in the database. Why would you want to add another cache?

If you use a Web hosting service where you cannot control every detail and you pay a lot of money for the service, you could end up with more memory than you use. Some of that memory can be used to cache parts of your Web site in place of things you cannot control.

If your Web hosting service provider does not offer a reverse proxy server, you can use memcache to provide a similar service. That is the main readon I put Memcache after the proxy service option. investigate proxy servers first.

Memcache can cache other things including things that could be cached by the operating system, the Web server, and the database if they had enough memory. Tune their settings first to make maximum use of those features. You can control those other items on dedicated servers and many Virtual Private Servers but not shared hosting. Memcache does not work on shared hosting, almost nothing works on shared hosting, so switch from shared hosting to a VPS.

3. Tune your Web server

A good Web hosting supplier will recommend Web server software and tune the software. Some of the lesser hosting companies throw all sorts of hosting workloads on one server in the hope that they will balance out. The better ones place similar size accounts on the same server and occasionally move sites around to balance out workloads from different time zones. They set Apache, or their recommended Web server software, to the right settings for an account.

Some articles tell you to tune Apache yourself. The say things like a simple calculation then hit you with (total memory - OS memory - MySQL memory) / size of an Apache process. Yeah easy to read but where do you get the information. Your hosting account provides the total memory. Your Web hosting provider can find the others. When you ask them to point you to the other figures, tell them what you are interested in achieving and ask them for a recommendation. A good Web hosting provider will have the main Apache settings preset to close to the right values for the type of account you purchased.

4. Switch from MySQL to InnoDB

Hard to place this here or in time wasters. InnoDB is an alternative to MySQL, performs a little bit faster some of the time and slower other times, with MySQL being the best performer in most cases. Pressflow is a version of Drupal with a lot of database requests changed to make better use of InnotDB, making InnoDB the slightly better performer for Pressflow. Drupal 7 uses InnoDB and the Pressflow style improvements, making InnoDB one of those things that should be done for you by the conversion to Drupal 7.

If your Web site software does not specify a specific database, your Web site hosting supplier might provide one preconfigured. Let them do it for you and use whatever they provide. They can then put in all the work to tune the database.

I prefer PostgreSQL to InnoDB and Drupal 7 will support PostgreSQL. PostgreSQL also supports great features for really big databases. During the life of Drupal 7, I will switch some of my biggest sites to PostgreSQL. I will look for a Web hosting service with experience in providing PostgreSQL and use whatever tuning settings they recommend.

If you switch databases without experience or support, the results will be the same or worse. With drupal 7 so close, go with the flow, leave your database alone and use whatever the Drupal 7 conversion produces.

10 might be useful for really big sites

1. Apache Solr

Apache Solr is a replacement for the Drupal search facility. Solr runs as a separate service hopefully on a separate server because it uses Java and does not fit in with a Web server tuned for Drupal. If you use a VPS, you could use a different VPS for Solr.

There is nothing in Solr that could not be added to Drupal or any other content management system. The first part of the speed improvement from Solr is the simple act of splitting part of your Web workload off to another server. You could achieve the same improvement for your existing Web site by using a faster server or more memory or moving the database to another server or by moving the search database tables to a database on another server.

The second part of the speed improvement is the big investment in developing Solr, far more than was invested into the search facility for Drupal.

You then get a bit of an improvement from something with the fancy title of faceted search. All it means is you can search the results of a search instead of searching the whole database. Sub searches create as many problems as they solve because many people do not understand what is happening. You need a really good user interface to keep people informed of what is happening.

A good subsearch goes something like this. I want a taxi. I search for taxi. Bloat search says their are 70 million results and you have to think some more about your search. A good search would understand the use of taxies and would ask if you would like a taxi in your city. You are now searching just one city. The search can get really tricky when you have several options set to limit the search but cannot see why there are so many results missing. In the taxi example, a selection of Sydney as the city might produce the whole of Sydney or just the central business district.

There are sites where you can add extra selection criteria but you cannot step back one level, instead you have to restart the whole search. There are a few other problems with subsearch user interfaces. When I first tried to use Solr based searches, the user interfaces made the searches difficult.

Solr makes some searches faster because you are allocating extra hardware resources and money that might be better spent in other areas. The best feature of Solr depends on a really good user interface with means you need to invest time and research into making the best use of Solr to give your users a real benefit.

10 time wasters

These are thinks recommended by a lot of people but you rarely see them recommended from personal experience, it is almost always just something they read somewhere then put in a Web page do they could publish a 10 things list.

1. SSD

SSD is Solid State Disk. The best SSDs are faster than conventional magnetic disks. See Best SSD for your notebook computer. The average SSD is slower than using magnetic disks in a RAID 5 array. Dollar for dollar, SSDs are small compared to magnetic disks and may be too small for your Web site.

Your Web site hosting supplier will suggest a range of storage options including faster disks and RAID. They might offer SSD as a very expensive option and in most cases, the RAID option is the best option. Many SSDs advertise a fast peak performance for specific types of read operations and hide their slow performance at everything else including terrible write speeds. RAID does not suffer that performance difference. You have to spend really big dollars to get fast write performance from SSD for a Web server and those big dollars produce far more speed when spent on extra memory, RAID, and other options.

If you build your own Web servers and you have your servers already at maximum memory and your boss says you cannot buy another server but you can spend money on disks, look at some of the better SSDs for use with selected files. In a server starved of memory, you might put page files on SSD. In a server with a highly optimised database and still of limited performance, you might put the database indexes on an SSD or the whole database on SSD. Use two SSDs in a RAID 1 configuration or three SSDs in RAID 5. Pay the extra money for SSDs with good write performance.