Apache logs, Load Balancer and X-Forwarded-For

In most normal configurations Apache’s web server logs look like this:

75.104.128.36 - - [13/Aug/2008:14:06:32 -0700] "GET /index.html HTTP/1.1" 200 21279 "http://khaitan.org/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16"

This is achieved by using the following log format in the apache virtual host config, which looks like this:

%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i"

However, when you move apache behind a Load Balancer say F5 BigIP, the logs start showing the IP of the load balancer instead of the actual client IP. This may result to error in reporting viz. uniques, and other issues if your application relies on knowing the client IP. It is also possible that any geo based code may also stop working. There is an easy way to fix it.

Good news is that most of the modern load balancers already have a mechanism of sending the client IP. This is done by inserting an HTTP header X-Forwarded-For. It may look something like this: UA-CPU: x86
Accept-Encoding: gzip, deflate
X-Forwarded-For: 67.161.42.194

First, Make sure that your Load Balancer is sending the X-Forwarded-For header. Drop this small php file on your server which is behind the load balancer and make sure that the IP of the machine from where you are connecting to shows up in the header as shown above.


<?php
$headers = apache_request_headers();
foreach ($headers as $header => $value) {
echo "$header: $value
\n";
}
?>

If you do not see that header, change your load balancer settings (Google X-Forwarded-For for your specific load balancer) or better still call your Sys admin to do it for you.

Finally, modify your log directive in apache by replacing %h to %{X-Forwarded-For}i

%{X-Forwarded-For}i %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i"

Tags: , ,