Envtesting: Environment Testing to Verify Settings
Do you run your application on multiple servers? Do you have separate environments for production and development?
How many have you found errors in your application or had it break completely as a result of a different setting or condition in your environment?
Trying to detect errors in your environment can end up costing you or your admin long unproductive hours.
As you search for bugs, you’ve probably asked yourself any number of questions:
- Is the application actually connected to the database?
- Did I remember to adjust the access rights correctly?
- Am I allowed to write to the /tmp directory or log?
- Did I forget to add PHP support for PDO?
- Did I add support for MongoDB?
Most of these questions can be answered automatically using a simple test.
In terms of development, testing your environment is as important as unit and integration testing. And it’s especially appropriate for those of you using continuous delivery.
Environment testing was something that we struggled with at Wikidi and after we were unable to find any suitable PHP solutions – we created Envtesting.
Envtesting is used for simple automated environment testing and verifying the minimum requirements needed to run PHP applications.
One of the big things we kept in mind when we were writing Envtesting is that it need to be a tool that was not just for programmers, but for administrators as well. Essentially, it had to be a tool that didn’t require a strong background in programming.
We also want to make sure creating a test was as simple as possible. Similarly, it needed to be easy to add already existing tests to a new Envtesting process. Our last requirement was how fast we could deploy it to a new project.
Let’s take a closer look at what you can do with Envtesting.
How to Download Envtesting
Envtesting is a single minified PHP file and you can easily download it using a command line tool, such as curl.
curl https://raw.github.com/wikidi/envtesting/master/Envtesting.php > Envtesting.phpYou can find the entire source code on Github. We minified Envtesting using a script from David Grundl in nette/build-tools. Thanks, David!
Writing Your First Test
To write your first test, all you have to do is create a PHP file and an instance object \envtesting\Suite. From there, you can add virtually any test:
<?php
require_once __DIR__ . '/Envtesting.php';
$suite = new \envtesting\Suite('my great envtest');
$suite->addTest('booltest', function() {
if (true != false) {
throw new \envtesting\Error('Tento test je vzdy spatne');
}
}, 'boolean');
// spusti vsechny testy a vypise jejich vysledek
echo $suite->run();The addTest function expects the second parameter to be an anonymous function, a callback, or an instance object that implements the __invoke function.
Note that anonymous functions always throw \envtesting\Error. This exception occurs when you execute $suite->run();. It’s captured and converted as a failed test mybooltest.
If you run the code generated by the command line, you’ll get the following result:
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
my great envtest
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
ERROR | booltest | boolean | | | This test is always wrongWhen you run tests from a command line, it’s possible to come across different PHP settings for Apache and PHP CLI. However, you can start generating the results as an HTML page with a little code editing.
Instead of echo $suite->run();write $suite->run()->render();:
<?php
require_once __DIR__ . '/Envtesting.php';
$suite = new \envtesting\Suite('my great envtest');
$suite->addTest('booltest', function() {
if (true != false) {
throw new \envtesting\Error('Tento test je vzdy spatne');
}
}, 'boolean');
// spusti vsechny testy a vypise jejich vysledek jako HTML
$suite->run()->render();The results should look something like this:
Please note the CSV buttons – this is the third format you can use to receive the results for tests. This format may be useful for automatically processing output to a monitoring tool (e.g. Nagios).
More Complex Tests
Envtesting is part of a larger collection of several basic envtests. This collection includes, among other things, a test for testing the connection to MySQL (test writing, reading, deleting records) and tests for MongoDB.
The tests are divided into three groups: application tests, libraries, and testing services.
The simplest tests are located in envtests/library. These test are used to verify whether some PHP extensions are present.
For example, this test is used to verify the accessibility of Gettext:
<?php
namespace envtestslibrary;
// verify gettext and get text function is present
\envtesting\Assert::true(
\envtesting\Check::lib('gettext', 'gettext'),
'Gettext library not found'
);
echo 'Gettext found';It’s easy to add this test to your envtestingSuite:
<?php
require_once __DIR__ . '/Envtesting.php';
$suite = new \envtesting\Suite('my great envtest');
$suite->addTest(
'Gettext',
'envtests/library/Gettext.php',
'library'
);
$suite->run()->render();Internally, the Gettext.php script is wrapped in an anonymous function using include while running the test. The test either fails and throws an exception, or passes, and the output for all the content is then displayed as a final message report.
The next group of complex tests are used to verify the functionality of a service.
For example, this test checks whether you are connected to the Memcache server.
Here’s how to add it to your Suite:
<?php
require_once __DIR__ . '/Envtesting.php';
$suite = new \envtesting\Suite('my great envtest');
$suite->addTest('memcache', new envtestsservicesmemcacheConnection('127.0.0.1', 11211), 'service');
$suite->run()->render();The last group are application tests, which are designed to verify, for instance, whether or not you can write to MySQL tables. Again, this is a trickier example since, in this case, MySQL requires a special table.
The following example validates the accuracy of the access rights to the slave server database:
<?php
require_once __DIR__ . '/Envtesting.php';
$suite = new \envtesting\Suite('my great envtest');
$slaveOperations = new envtestsapplicationmysqlOperations(
"mysql://slave:slave@localhost:3306/database"
);
$suite->addTest(
'mysql:select',
array($slaveOperations, 'selectAllow'),
'application'
)->setNotice('slave');
$suite->addTest(
'mysql:insert',
array($slaveOperations, 'insertNotAllow'),
'application'
)->setNotice('slave');
$suite->addTest(
'mysql:delete',
array($slaveOperations, 'deleteNotAllow'),
'application'
)->setNotice('slave');
$suite->addTest(
'mysql:update',
array($slaveOperations, 'updateNotAllow'),
'application'
)->setNotice('slave');
$suite->run()->render();Here’s an example of multiple tests:
A Note About Transmission Parameters
Occasionally, it’s possible for a test to throw an option to pass for some parameters when it runs. Envtesting takes even these instances into account.
You can add parameters directly to tests at the same time you add tests to \envtesting\Suite using the function \envtesting\Test->withOptions():
<?php
require_once __DIR__ . '/Envtesting.php';
$suite = new \envtesting\Suite('my great envtest');
$suite->addTest('test', function ($param1, $param2) {
return sprintf('Predal jsi mi "%s" a "%s"', $param1, $param2);
})->withOptions('a', 'b')->setType('library');
$suite->run()->render();Test Groups
You can arrange tests in groups. It’s possible to mix these test groups together using the method \envtesting\Suite->shuffle().
In particular, you can set \envtesting\Suite so that if a single test fails, all the following tests are marked with the same error without running the entire group.
You can do this using the function \envtesting\Suite->failGroupOnFirstError().
<?php
require_once __DIR__ . '/Envtesting.php';
$suite = new \envtesting\Suite('my great envtest');
// group 1
$suite->group1->addTest(
'APC',
'envtests/library/Apc.php'
)->setType('library')->setNotice('1/3');
$suite->group1->addTest(
'GD',
'envtests/library/Gd.php'
)->setType('library')->setNotice('2/3');
$suite->group1->addTest(
'Gettext',
'envtests/library/Gettext.php'
)->setType('library')->setNotice('3/3');
// group 2
$suite->group2->addTest(
'PDO', function() {
throw new \envtesting\Error('Die with me!');
}
)->setType('library')->setNotice('1/3');
$suite->group2->addTest(
'PDO',
'envtests/library/Pdo.php'
)->setType('library')->setNotice('2/3');
// alternative note added to the test group
$suite->to('group2')->addTest(
'Mongo',
'envtests/library/Mongo.php'
)->setType('library')->setNotice('3/3');
$suite->setName('Group die test')
->shuffle()
->failGroupOnFirstError()
->run()
->render();// fail group on first errorA Few More Things About Envtesting
Envtesting is capable of doing much more, so here are a few other bits for you to take away:
\envtesting\Suite is able to implement ArrayAccess and IteratorAggregate, which comes in handy if you prefer to write any tests yourself. Most functions provide a fluent interface. Static functions like Suite::instance()return single objects of\envtesting\Suite, which is sufficient – in most cases, you’ll only need one instance Suite.
Failed tests do not necessarily result in \envtesting\Error or Exception being thrown. Envtesting also recognizes one state of test and rejects the warning \envtesting\Warning. This is useful if you’re checking a less critical requirement.
If a test goes well, Envtesting will automatically process the return value of an anonymous function or a callback function as a report of the test results. Or if you’re interested in a particular object, you can use the__toString() method. The method is again used to compile a report about the results of passing this test.
The last thing worth mentioning is \envtesting\Throws. One of note is a simple error_handler, which captures all warnings and notices, and then, converts them to the classic exception Exception.
This article was originally published in Czech on Zdrojak by Testomato developer Roman Ožana. Be sure to check out his blog nabito.net for more It has been translated and posted here with his permission.
