How to Remove Testomato from Your Access Logs

Don’t let Testomato fill up your access logs. 

Uptime monitoring and periodical testing can end up adding a crazy amount of useless data to your access logs since they list all the requests for individual files that people have requested from your site.

We pulled together a short guide to show you how to remove Testomatobot  from your Nginx and Apache access logs to avoid generating a lot of unimportant data.

The Problem with Bots

In case you don’t already know, Testomatobot is the toolset we use to download page content, send web forms, and access other resources to test according to your project configuration.

There are various reasons different bots may be accessing your site: search engines looking to index your site, a program updating the content in a newsreader, or services like Testomato that check your server or site speed.

Depending on personal and project preferences, you may actually want to see the bots in your raw server log data.

But it’s not always information that everyone wants to track, and in some cases, bots can be problematic.

Here’s why you might want to block them:

• You have to dig through a mountain of data when you want to check your logs.
• Your data can end up showing weird spikes and skew your analytics insights.
• Your web host charges by the hit on the server based on your logs.

An automated testing service like Testomato can end up pinging your server regularly to get information. For example, when you enable Uptime Monitoring, we check your web projects every 15 seconds to make sure they’re online and how quickly they’re responding.

Every time Testomato accesses your site, we show up in your log. This could amount to as many as 240 times per hour, which comes out to over 5,000 times per day.

So, while some of you may choose to have Testomatobot show up in your server access logs, we wanted to give you the option to remove it.

How to Remove Testomatobot

You can use the following lines of code to remove Testomatobot from your Nginx and Apache logs.

For Nginx < 1.7.0

Add the following code to your Nginx settings:

map $remote_addr $dontlog {
  default      0;
  "10.2.100.2" 1;
  "8.8.8.8" 1;
}

And change your virtual server settings access_log with the code if ($dontlog) { access_log off; } :

server {
  listen 80;

  server_name www.server.com;

  access_log /var/www/server/logs/http/access.log combined;
  error_log /var/www/server/logs/http/error.log warn;

  location / {
    if ($dontlog) {  access_log off; } # off log for testomatobot
    try $uri /index.php?$uri&$query_string;
  }

  location ~ (^/img/|^/css|^/js) {
    if ($dontlog) {  access_log off; } # off log for testomatobot
    try $uri /error.php?$uri&$query_string;
  }

}

For Nginx >= 1.7.0

Add the following code to your Nginx settings:

map $remote_addr $dontlog {
  default      0;
  "10.2.100.2" 1;
  "8.8.8.8" 1;
}

And change your virtual server settings access_log with the code if=$donotlog;:

server {
  listen 80;

  server_name www.server.com;

  access_log /var/www/server/logs/http/access.log combined if=$dontlog; # off log for testomatobot
  error_log /var/www/server/logs/http/error.log warn;

  location / {
    try $uri /index.php?$uri&$query_string;
  }

  location ~ (^/img/|^/css|^/js) {
    try $uri /error.php?$uri&$query_string;
  }

}

Apache

The configuration for Apache is even simpler:

SetEnvIf Remote_Addr "10\.2\.100\.2" dontlog
SetEnvIf Remote_Addr "8\.8\.8\.8" dontlog

CustomLog "/var/www/server/logs/http/access.log" combined env=!dontlog

Have more questions about how to remove Testomatobot?

You can reach our team any time at us support@testomato.com! You can also get in touch with us on Facebook or Twitter.