7G Firewall : Log Blocked Requests

This tutorial explains how to log requests that are blocked by the 7G Firewall. This is useful for testing, debugging, and just keeping an eye on things. Learn how to log requests from Apache mod_rewrite and download my custom 7G logging script. It’s a complete example that shows how to log rewrite requests via PHP. All open source and free :)

If you’re using 7G, you can find any errors using the free logging script.

Contents

• How it works
• Log Example
• Usage
• Configure 7G for logging
• Preparing the Log file
• Reading the log file
• Preflight Check
• Download 7G Logging script
• Troubleshooting 7G Firewall
• Bonus: IP Logging with 7G

How it works

The 7G Firewall includes built-in logging directives. Once logging is enabled (as explained below), mod_rewrite will send request data to our handy little PHP script. The logging script then parses the data and writes it to the log file. The result is a log file that looks very similar to Apache defaults.

Under the hood, here is what happens for any requests blocked by 7G:

1. 7G checks the request
2. If not blocked, the request continues normally
3. If blocked, the request data is sent to the logging script
4. The logging script then writes the data to the log file
5. The script exits with a simple message (configurable)

So regular visitors and legit traffic will continue normally, while any blocked requests are redirected to the logging script. Conceptually simple.

Log Example

Here is an example showing what the log entries look like:

012.012.012.01 - 2019/01/24 01:47:12 - GET - HTTP/1.0 - /www.php - boot.ini [boot.ini] - - - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6)
187.255.215.91 - 2019/01/24 21:19:01 - GET - HTTP/1.0 - / - - - - Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) [Exabot] 
222.333.444.00 - 2019/01/24 00:15:33 - POST - HTTP/1.0 - /xertive.php [xertive] - - - - curl/7.54.0

In log entries, matching firewall patterns are indicated via brackets like [this]. In the above example, notice the following blocked patterns:

• boot.ini — from 7G QUERY STRING rules
• Exabot — from 7G USER AGENT rules
• xertive — from 7G REQUEST URI rules

So for each request, the location of the bracketed/match string tells us the related section of 7G. More details on this later in the post.

Usage

To log any/all requests blocked by 7G, follow the steps:

1. Add 7G Firewall to root .htaccess (or Apache config)
2. Configure 7G Firewall for logging (see next section)
3. Unzip 7G Logging script and upload files to root web directory
4. Make log.txt writable (and protect via .htaccess)

Test well and leave feedback below in the comments or direct via my contact form. Please do not report bugs in the comment section, best to communicate via email/contact form, thank you for understanding.

Configure 7G for logging

The 7G Firewall comprises the following sections:

# 7G:[CORE]
# 7G:[QUERY STRING]
# 7G:[REQUEST URI]
# 7G:[USER AGENT]
# 7G:[REMOTE HOST]
# 7G:[HTTP REFERRER]
# 7G:[REQUEST METHOD]

Each of these sections contains a set of mod_rewrite rules. For each section (except CORE rules), you want to disable (comment out) the default RewriteRule, and enable (un-comment) the logging RewriteRule. For example, in the QUERY STRING section:

# 7G:[QUERY STRING]
<IfModule mod_rewrite.c>
        
        .
        .
        .
        
        RewriteRule . - [F,L]
        
        # RewriteRule . /7g_log.php?log [L,NE,E=7G_QUERY_STRING:%1___%2___%3]
        
</IfModule>

We want to change that to this:

# 7G:[QUERY STRING]
<IfModule mod_rewrite.c>
        
        .
        .
        .
        
        # RewriteRule . - [F,L]
        
        RewriteRule . /7g_log.php?log [L,NE,E=7G_QUERY_STRING:%1___%2___%3]
        
</IfModule>

So now the QUERY STRING rules are sending request data to our 7g_log.php log file. Repeat this same “rule swap” for each section (except CORE rules) in 7G. Once this is done, 7G is configured for logging and will send all blocked requests and data to the 7G Logging script.

Preparing the Log file

To prepare the log file, you want to make sure it is writable by the server (e.g., change CHMOD permissions on the file). Then you also want to make sure the file is protected from outside access. This is prevent sensitive information from falling into public domain. As we’re already working with .htaccess, here is a simple code snippet that protects our log file from all outside access:

# Apache < 2.4
<IfModule !mod_authz_core.c>
        <Files ~ "7g_log\.txt">
                Deny from all
        </Files>
</IfModule>

# Apache 2.4+
<IfModule mod_authz_core.c>
        <Files ~ "7g_log\.txt">
                Require all denied
        </Files>
</IfModule>

Add that to your root .htaccess file and done. No changes need to be made. Note however that both sections are not necessary; if you know your version of Apache, it is safe to use only the relevant code.

Reading the log file

For each log entry, the following request data is recorded (if available):

• IP Address
• Date/Time
• Request Method
• Request Protocol
• Request URI
• Query String
• Remote Host
• Referrer
• User Agent

So each log entry/line records these fields in the following order:

IP Address - Date/Time - Request Method - Request Protocol - Request URI - Query String - Remote Host - Referrer - User Agent

Note that empty/blank values are simply left as-is. So when you see stuff like this in your log file it’s totally normal:

- / - - - - User agent 1234..

It just means that no data was available for each of the blank items. It is common especially for Remote Host and Referrer fields to be blank.

Preflight Check

Here is a checklist of requirements for logging to work:

• 7G Firewall installed and configured for logging
• 7G_log.php and 7G_log.txt both added to root web directory
• 7G_log.txt is writable by the server
• 7G_log.txt is protected against outside access

Once everything is in place and ready, 7G logging should be enabled on your site. Note that the 7G_log.php script includes several variables/options that you can modify if desired:

• SEVENGLOGPATH = Path to the log file
• SEVENGLOGFILE = Name of the log file
• SEVENGUALENGTH = Length of User Agent log entry
• SEVENGEXIT = Message displayed to blocked visitors
• date_default_timezone_set('UTC') = Default Timezone

These variables are predefined to work according to this tutorial, out of the box. So no changes need to be made to the logging script, but there are few things that you can tweak as needed.

Download 7G Logging script

Here is the latest version of the 7G Firewall Logging script. License: GPL v3.

7G Firewall: Log Blocked Requests – Version 1.0 (2KB zip)

Important: 7G Logging is recommended for testing/development purposes only. Not recommended for live/production sites.

Disclaimer: Capturing HTTP requests via mod_rewrite can be challenging, because each host/server/setup/config is unique. The 7G Logging script is developed in several different Apache/server environments, but it is not guaranteed to work out of the box for everyone.

Troubleshooting 7G Firewall

Once logging is enabled on your site, troubleshooting and testing the 7G Firewall becomes quick and easy. This section gives some ideas and tips that should be useful for anyone wanting to debug, test, and so forth.

While using 7G, if any legitimate page or resource fails to load, or if some feature stops working, simply take a look at the 7G log file. Locate the most recent (and/or relevant) entries. For each related entry, the offending firewall pattern will be indicated with [brackets]. So you will know immediately which 7G rules/pattern(s) are responsible for any issues.

If you discover any offending patterns, simply disable (comment out) or remove. Also please report any bugs or patterns that are causing problems. That way I can update the firewall and keep things as error-free as possible.

Troubleshooting Tips

Here is a working list of things to check when troubleshooting 7G.

Locating patterns

Some sections of 7G contain a lot of “tightly packed” rules. This can make locating offending patterns difficult, even when you know the matching pattern. In such cases, the quickest way to identify the pattern is to use the the halving method.

Code placement

In your .htaccess file, Make sure that 7G code before any existing mod_rewrite rules (e.g., WordPress Permalinks).

Server error

If you get a server error after installing 7G, double-check that your site meets the requirements.

Encoded Characters

By default, special characters are passed to the log file unencoded. To instead pass the encoded equivalents, remove the “No Escape” flag NE from each of the 7G rewrite rules.

More tips will be added as they arrive.

Bonus: IP Logging with 7G

Users familiar with nG Firewall/Blacklist may have noticed that IP blocking is removed from 7G. As covered in previous posts, there are better, more effective ways to prevent site access based on IP address. Nonetheless, blocking by IP remains useful in a variety of scenarios. So with that in mind, here is a bonus IP ADDRESS section that may be added to 7G (or any .htaccess file):

# 7G:[IP ADDRESS]
<IfModule mod_rewrite.c>
        
        # RewriteCond %{REQUEST_URI} !(7g_log.php) [NC]
        
        # RewriteCond %{REMOTE_ADDR} ^(000\.000\.000\.000)$ [OR]
        # RewriteCond %{REMOTE_ADDR} ^(100\.100\.100\.100)$ [OR]
        # RewriteCond %{REMOTE_ADDR} ^(200\.200\.200\.200)$ 
        
        # RewriteRule . /7g_log.php?log [L,NE,E=7G_IP_ADDRESS:%1]
        
</IfModule>

As written, this code does nothing, because each directive is disabled (commented out with a hash/pound sign #). To enable these rules, remove the hash from each line. Then replace the “dummy” placeholder IP values with any that you would like to block and log. Once implemented, these rules will result in any blocked IP addresses indicated in the log file with [brackets].

Tip: For the above IP ADDRESS section, each RewriteCond should end with an [OR] flag, except the last RewriteCond. If you only want to block one IP address, use only one RewriteCond and omit the [OR] flag.

May the log files be with you!

from Perishable Press http://bit.ly/2G0zV0y