Introduction:

One of the common tips to increasing Apache performance is to turn off the per-directory configuration files (aka .htaccess files) and merge them all into your main Apache server configuration file (httpd.conf).

Jeremy raised an interesting question about when the performance loss caused by using many htaccess files is offset by the ease of maintenance. He's arguing - and I agree - that it makes sense to keep the configuration locally inside .htaccess files, despite the performance loss as these are easier to maintain.

It's fairly logical that the multiple .htaccess file route will be slower - for every node in the request URI, the webserver has to look for an .htaccess file and merge the rules found in every one. So, we're going to have to have a filesystem seek'n'read for every subdirectory.

However, is this a major issue? How much of a performance hit is there? Let's find out...

Set-up:

Ok. Let's make two docroots each with the same structure and files.

1) htdocs_access - the .htaccess version. This has one .htaccess file in the leaf directory.

2) htdocs_config - the httpd.conf version. This has the same rule as the above, but the rule is in the server-wide httpd.conf file and htaccess support is turned OFF (AllowOverride None).

Next, we need to get the .htaccess/httpd.conf files to do something ( mainly so we can see if Apache's merged them in ). So, we'll make a number of files in the last random directory (the leaf node), and give half of them the extension .foo, and the other half .bar. We'll then tell Apache to process the .bar's with PHP, and the .foo's as text. All files will have the same content:

PLAIN TEXT
PHP:

Here's the (python) code I used to generate this structure:

PLAIN TEXT
PYTHON:
  1. #!/usr/bin/env python
  2.  
  3. import os
  4.  
  5. # where we'll place the generated structure
  6. staging = '/Users/simon/server'
  7. htdocs_access = os.path.join(staging, 'htdocs_access')
  8. htdocs_config = os.path.join(staging, 'htdocs_config')
  9.  
  10. # how deep to go!
  11. dir_depth = 10
  12.  
  13. # how many files in the leaf node of the dir.
  14. num_files = 50
  15.  
  16. # what content to put in the files
  17. content = ""
  18.  
  19. # the actual htaccess file
  20. htaccess = """
  21. AddHandler application/x-httpd-php .bar
  22. """
  23.  
  24. # make directory structure
  25. dir = ''
  26. for dirnum in range( 0, dir_depth ):
  27. dir = os.path.join( dir, str( dirnum ) )
  28.  
  29. hta = os.path.join( htdocs_access, dir )
  30. htc = os.path.join( htdocs_config, dir )
  31. os.makedirs( hta )
  32. os.makedirs( htc )
  33.  
  34. # make the files...
  35. for filenum in range( 0, num_files ):
  36. # assign the file types - half .foo, and half .bar
  37. if filenum % 2 == 0:
  38. filename = '%d.foo' % filenum
  39. else:
  40. filename = '%d.bar' % filenum
  41.  
  42. f = open( os.path.join( hta, filename ), 'w+' )
  43. f.write( content )
  44. f.close()
  45.  
  46. f = open( os.path.join( htc, filename ), 'w+' )
  47. f.write( content )
  48. f.close()
  49.  
  50. # now, add the .htaccess file inside the lead htdocs_access dir
  51. f = open( os.path.join( hta, '.htaccess' ), 'w+' )
  52. f.write( htaccess )
  53. f.close()
  54.  
  55. # and we'll place it in the root of the htdocs_config dir as
  56. # httpd.conf to remind ourselves to add it to the httpd.conf file
  57. f = open( os.path.join( htdocs_config, 'httpd.conf' ), 'w+' )
  58. f.write( htaccess )
  59. f.close()

Here's what we end up with:

PLAIN TEXT
CODE:
  1. 0/
  2. 1/
  3. 2/
  4. 3/
  5. 4/
  6. 5/
  7. 6/
  8. 7/
  9. 8/
  10. 9/
  11. 0.foo
  12. 1.bar
  13. 10.foo
  14. 11.bar
  15. (...etc...)
  16. 6.foo
  17. 7.bar
  18. 8.foo
  19. 9.bar

Where htdocs_access has a .htaccess file in 9/ and htdocs_config doesn't.

Server Configuration:

Here are the two httpd.conf files for the configurations:

htdocs_config httpd.conf:

PLAIN TEXT
CODE:
  1. ### Section 1: Global Environment
  2. ServerRoot "/usr/local/apache2"
  3. PidFile logs/httpd.pid
  4. Timeout 300
  5. KeepAlive On
  6. MaxKeepAliveRequests 100
  7. KeepAliveTimeout 15
  8. DirectoryIndex index.html
  9. AccessFileName .htaccess
  10. HostnameLookups Off
  11.  
  12. # fixes crashes on OSX??
  13. AcceptMutex fcntl
  14.  
  15. StartServers         5
  16. MinSpareServers      5
  17. MaxSpareServers      5
  18. MaxClients         100
  19. MaxRequestsPerChild  10
  20.  
  21. ### Section 2: 'Main' server configuration
  22. User nobody
  23. Group #-1
  24.  
  25. DocumentRoot "/Users/simon/server/htdocs_config"
  26.  
  27. LoadModule php5_module modules/libphp5.so
  28.  
  29. Listen 8111
  30.  
  31. <directory>
  32. Options Indexes FollowSymLinks
  33. AllowOverride None
  34. AddHandler application/x-httpd-php .bar
  35. </directory>

htdocs_access httpd.conf:

PLAIN TEXT
CODE:
  1. ### Section 1: Global Environment
  2. ServerRoot "/usr/local/apache2"
  3. PidFile logs/httpd.pid
  4. Timeout 300
  5. KeepAlive On
  6. MaxKeepAliveRequests 100
  7. KeepAliveTimeout 15
  8. DirectoryIndex index.html
  9. AccessFileName .htaccess
  10. HostnameLookups Off
  11.  
  12. # fixes crashes on OSX??
  13. AcceptMutex fcntl
  14.  
  15. StartServers         5
  16. MinSpareServers      5
  17. MaxSpareServers      5
  18. MaxClients         100
  19. MaxRequestsPerChild  10
  20.  
  21. ### Section 2: 'Main' server configuration
  22. User nobody
  23. Group #-1
  24.  
  25. DocumentRoot "/Users/simon/server/htdocs_access"
  26.  
  27. LoadModule php5_module modules/libphp5.so
  28.  
  29. Listen 8111
  30.  
  31. <directory>
  32. Options Indexes FollowSymLinks
  33. AllowOverride All
  34. </directory>

Results:

Benchmarking was done with "ab" the Apache Benchmark program, which was set to access one page 1,000 times with 10 concurrencies. Each configuration was benchmarked five times in random order (to minimise the effect of any running background processes etc).

htdocs_config htdocs_access
Test: Time Taken (s): Requests per Second: Time Taken (s): Requests per Second:
1 12.683213 78.84 13.21618 75.66
2 12.854491 77.79 13.574916 73.67
3 11.777676 84.91 13.163296 75.97
4 13.668398 73.16 12.26475 81.53
5 13.76753 76.47 13.264527 75.39
AVERAGE: 12.9 78.23 13.1 76.4

So - we're looking at a difference of around 2.3% extra requests per second when htaccess files are disabled. This is really quite trivial, and should only be worried about when you're really loaded.

Issues:

There are a number of areas where this could be improved:

  • Try different directory depths i.e. the more nested the directory is, the slower it should be under the .htaccess scenario. In contrast, if there's only 2 or 3 levels then it should be faster.
  • Have multiple .htaccess files in the intermediate nodes to see how Apache handles the merging of these files. Here we've just used one .htaccess file, and we should probably see further slowdowns if Apache has to merge some complicated rule sets.
  • Access different files - I just requested one file repeatedly, so we might be getting a lot of interference from any caching systems (harddrive, ram, php caches etc) that I forgot about. Additionally, requesting multiple URI's is a more realistic test case for a webserver.