In my last three entries about minimal web server security, I covered
security by obscurity methods to
deter a blind attacker and very basic methods to prevent unintentional information disclosure by
blocking open directory browsing.
The best solution to prevent open directory browsing is simply turn it off. Using a .htaccess file and setting
Options -Indexes, you block undesirable directory browsing. However, the .htaccess file can do much more than just disabling directory indexing.
(The following examples are part of a presentation I gave earlier this week for the Fort Collins Internet Professional's Technical Track.)
Redirecting Errors
When you use
Options -Indexes, requests for the open directory will return a server error (403 Forbidden). However, you might want to return a nicer error code. Using your .htaccess file, you can catch and redirect errors to an HTML page or return simple HTML. For example:
ErrorDocument 403 /403.html
ErrorDocument 404 "That page was <b>not found</b>."
ErrorDocument 401 /login/unauthorized.cgi
You can also redirect based on substring matches. For example:
RedirectMatch 301 (.*/secret.txt) /~user/block.cgi?$1
The extended regular expression "(.*/secret.txt)" is a little cryptic. It says to look for any substring in the URL that contains "/secret.txt". It then redirects the user to a CGI web page and appends the offending URL to the query for the CGI.
Blocking Attacks
The .htaccess file allows you to define variables and then use the variables as conditionals. For example, my blog should not have any kind of quotes in the URL. If someone submits a quote, then they are likely trying to attack my site with an SQL or CGI injection attack. By defining a few environment variables, I can detect and block potential attacks long before they ever get to my blog software (and potentially compromise my site).
SetEnvIf Request_URI "'" bad_bot=1
SetEnvIf Request_URI '"' bad_bot=1
SetEnvIf Request_URI '`' bad_bot=1
SetEnvIf Request_URI '%22' bad_bot=1
SetEnvIf Request_URI '%27' bad_bot=1
SetEnvIf Request_URI '%60' bad_bot=1
<Limit HEAD GET POST>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>
The first part sets an environment variable called "bad_bot" if any kind of quote is seen in the URI. (The URI is the part of the URL that comes after the hostname and before any question mark.)
The second part limits who can access the web site. If you do a HEAD, GET, or POST request, and you include a quote in the URI, then you are blocked. Since casual/friendly users don't include quotes in the URL, this is not a problem for regular visitors.
Other characters you might want to block include <, >, $, !, and the backslash (\).
Depending on your site, you might even want to block them from query strings (everything after the question mark). To do this requires the mod_rewrite engine.
RewriteEngine on
RewriteCond %{THE_REQUEST} .*['"`!$<>;].* [OR]
RewriteCond %{THE_REQUEST} .*%22.* [NC,OR]
RewriteCond %{THE_REQUEST} .*%27.* [NC,OR]
RewriteCond %{THE_REQUEST} .*%60.* [NC,OR]
RewriteCond %{THE_REQUEST} .*%3C.* [NC,OR]
RewriteCond %{THE_REQUEST} .*%3E.* [NC,OR]
RewriteCond %{THE_REQUEST} .*%3B.*
RewriteRule $ - [l,F]
This looks for any of the forbidden characters in the entire HTTP request and fails access if any are seen.
Stopping Undesirables
Attacks can be technical or political. For example, let's say you don't want people to hyperlink to images on your site. How do you stop them? The answer is to look for the referrer (spelled "referer"). If the referrer for the image does not come from your site, then someone is cross-linking to you. In this example, we look for any jpeg, gif, png, or bmp request that was not linked from my site. If one is found, then a CGI script is called. Ideally, the script will generate an image that is the correct size but contains a nasty message about cross-linking.
# Identify if a Referer is used
SetEnvIf Referer "^http://www.hackerfactor.com/" from_me=1
SetEnvIf Referer "^http://hackerfactor.com/" from_me=1
SetEnvIf Request_URI .jpg has_image=1
SetEnvIf Request_URI .gif has_image=1
SetEnvIf Request_URI .png has_image=1
SetEnvIf Request_URI .bmp has_image=1
RewriteEngine on
RewriteCond %{env:has_image} 1
RewriteCond %{env:from_me} !1
RewriteRule (.*) /block_image_theft.php [l]
Of course, the most annoying type of undesirable visitor are those spam-bots. If you are a web master, then you know what I am talking about. They find forms on your site and submit tons of spam. Since many of these bots don't use a referer, we can block posts that lack a referrer.
# Identify if a Referer is used
SetEnvIf Referer "^$" no_referer=1
<Limit POST>
Order Allow,Deny
Allow from all
Deny from env=no_referer
</Limit>
Not Too Bad
With a little practice, your .htaccess file can block potential attacks against your web site. A well-configured .htaccess can deter undesirable visitors, protect you from potential exploits, and enhance your site's usability by returning informative error messages.
Your talk at FCIP earlier this month was great. Thank you for summarizing some of it here so members that couldn't attend can still make use of your excellent information.
I'm especially interested in cutting down spam-bot form submissions by checking for a referer.
Lots of useful stuff here! Thank you again for your contributions to the Fort Collins Internet community.
Thanks so much for the great presentation at FCIP.
Vi
If folks are using WordPress, you can seriously cut down on spam posts as well using some of the WordPress anti-spam widgets... I've found those working fairly well.
Thanks!
Sincerely,
James