A year in the life of the IJDb by Colin E.

Creating multilingual websites by Colin E.

Juggling Simulators - An Introduction by Colin E.

Learn to Juggle 3 Balls by Colin E.

An interview with Ben Beever by Colin E.

Introduction

A few months ago I had a change of career, I now work as a software engineer alongside a small team of developers in a comfortable little office overlooking the river Tyne (near Newcastle, UK). I have been writing software for many many years and consider myself to be a pretty competent programmer, however when I started this new job I quickly discovered that there is a lot more to software engineering than just been able to program. Testing is one of the many disciplines that all good software engineers should be well versed in. Where I work, testing is seen as an integral and essential part of software development and is an activity that we spend a considerable amount of time on.

I must admit that I have not approached the testing of my personal websites with a similar rigour. For small, non-critical developments a relaxed approach to testing works fine. However, I have found that for larger websites casual testing becomes very time consuming ... a more rigorous approach is required.

This article describes the development of an object oriented system testing framework for PHP. You can download the framework from teh following link:

Download: SiteCheck.zip (7 KBytes)

Testing in PHP

Software tests typically belong to one of the following categories:

• Unit tests - these are used to exhaustively test simple components that have few or no dependencies.
• Integration tests - these are used to test the functionality of a number of units working together.
• System tests - these are the top level tests, they drive the software through the same interface as the users.

Unit testing is very fashionable at the moment, this is due in part to the popularity of Extreme Programming which promotes test first programming. The PHP community also embrace unit testing, with a number of PHP unit test frameworks available for developers to use. However, I have yet to find any PHP framework which can be used to perform system tests.

The need for system tests

I have been actively developing a PHP driven web site, the Internet Juggling Database which has expanded considerably over the years. It is the very first web site I created using PHP, and my skills as a PHP developer have grown along with it. Consequently there are many parts of the code which I feel need a little attention. When I have the time, I like to make a few improvements, creating object oriented modules which I can re-use across this, and other, websites. However, each time I make some improvements there is a good chance that I will find an email from a kind visitor to my site informing me that they have found a page which bears a message similar to this one:

Fatal error: main(): Failed opening required '../lib/string.php' (include_path='.:') in /news/rss.php on line 23

The reason that these errors have occurred is almost always the same. I have done something like moved a file, changed a function interface, or renamed a database table without changing all the files that depend on it. No amount of unit testing will catch this type of error. The error has happened when I have broken the integration between two components of the web site, therefore what is needed is an integration or system test.

Unfortunately I have been unable to find any tools, PHP or other, which will perform the type of tests which I require. Most system testing tools geared towards web sites allow you to take snapshots of web pages and compare them to a static baseline. However, with a truly dynamic web site there is no such thing as a static snapshot. The only solution to this problem was to write my own system test framework ...

Finding errors in PHP pages

What I needed was a framework that would check all the pages on my site for PHP warning or error messages. This is basically a three step process, retreive the page, test for errors, write details to the report. PHP is well equipped to retreive web pages, see the manual entries for fopen and fsockopen, and generating reports in HTML or other formats is a simple task. The more interesting task is the identification of PHP errors within the body of your HTML pages.

For those of you who familiar with regular expressions, you will know that they offer the only practical solution to this problem. However, regular expressions are not the easiest of techniques to work with. They have such a horrific syntax that most developers I know agree that they cannot understand a one of their own regular expressions which they wrote just a week ago! Let's look at how we might develop a regular expression to match errors in PHP pages ...

When writing regular expressions I always start with something that I know is going to work, therefore I would start with a concrete match:

ereg("<b>Fatal error</b>: main(): Failed opening required '../lib/string.php' (include_path='.:') in /news/rss.php on line <b>4</b>",)

This will of course only match this specific error, but at least if that works, we have a good starting point. The next step is to make the match a little more general. The 'Fatal error' message is always just a couple of words, so we can replace this with "[a-zA-Z ]*", which just means any number of letters and spaces. This may seem a little too general, but remember that it is going to be used in conjunction with other patterns to form a more complex match. If we look at the other elements of our error message we can do the same, replacing the specific piece of text with a more general pattern, until we have an expression which is general enough to match any error that PHP might throw at us:

 $error = "[a-zA-Z ]*";
 $message = "[;=':_.<>\(\)/a-zA-Z0-9 ]*";
 $file = "[_.\(\)/a-zA-Z0-9 ]*";
 $line = "[0-9]*";
		
 if( ereg("<b>($error)</b>: ($message) in <b>($file)</b> on line <b>($line)</b>", $page, $matches))
 {
    ...
 }

Putting it all together

I have created a simple object oriented framework that tests for PHP error on specified pages. A summary of the classes that make up this framework are as follows:

• SiteCheck - the main class which retrieves pages from an instance of the URLSource class and checks them for errors using the technique described above. The results are sent to the ReportWriter class.
• URLSource - this class provides URLs to the SiteCheck class. I have provided a simple implementation, URLSourceArray, where you provide a list of URLs as an array. An alternative implementation could crawl your website hunting out links then passing each of these to the SiteCheck class.
• ReportWriter - this class receives messages from SiteCheck and assembles a report. The ReportWriterHTML outputs the report in HTML, whereas the ReportWriterEmail sends and email if any error are found.

An example of how SiteCheck is used is shown below:

 require ("SiteCheck.php");
 require ("URLSourceArray.php");
 require ("ReportWriterHTML.php");
 
 $urlSource = new URLSourceArray(
    array (		
           //front page
           "index.php", 			
           // links section
           "links/",
           "links/index.php?parent=4",
           "links/index.php?child=20",
           "links/submit.php",			
           // clubs section
           "clubs/",
           "clubs/index.php?country=14",           
           // news section
           "news/search.php"
           )
     );
 $siteCheck = new SiteCheck($urlSource, new ReportWriterHTML(), 80, "www.jugglingdb.com", "");
 $siteCheck->runCheck();

And an example of its output is shown here:

You do not have to exhaustively list every URL within your web site. If we do not know how the code behind this part of the site works, we perform 'black box' which is typically quite exhaustive, however if we wrote the code ourselves we are able to perform 'white box' tests. With this test technique we try to test all the paths of the code, hence the above example visits "links/index?parent=4", rather than "...?parent=1" , "...?parent=2", etc... If one works, we can be pretty sure that they all will.

Whenever I make changes to my web site I run a script similar to the above to see if I have inadvertently broken another part of the site with the changes I have made. I also have a cron job set up which runs the same script, but provides the SiteCheck class with an instance of the ReportWriterEmail class which will send me an email if anyone else makes a change to the site which results in PHP errors.

With the above in place I can now make significant structural changes with confidence and make bold updates, safe in the knowledge that my system tests are keeping an eye on things for me.

Download: SiteCheck.zip (7 KBytes)

Colin E. (webmaster@jugglingdb.com)