Chapter 7

Evaluation and Testing




Introduction

This chapter will outline the strategy employed to test the system both during and after development. The chapter is divided into four sections:

7.1 Testing Strategy
7.2 Development Testing
7.3 Field Testing
7.4 Testing Results


7.1. Testing Strategy

The strategy used tested the system at three different levels, as described in [LAYZELL 89].

Firstly, logic testing was undertaken. This is testing at the lowest level of detail. This tests that each individual function and programme element performs the function it is required to perform based upon the design. This helps to ensure that individual algorithms do what they are supposed to do, and do so as efficiently as possible.

Next, program testing was carried out to ensure that when the individual functions and programme elements are combined, they operate together in the correct way. In particular, this tests if the correct information is being passed between the individual modules and objects within the system.

Finally, program testing tests that the complete programme operates in the correct way, and meets the requirements of the user. This incorporates field tests with actual users.


7.2. Development Testing

The main problem encountered during logic testing was a problem with string lengths going over their boundaries. Unfortunately in C++ if a string is copied into another string that is not long enough to contain it the error produced can be very difficult to track down.
The worst culprit was the URL splitting algorithm, which takes URLs from the targets of links on web pages, and splits them into a domain, a path, and a filename.
To overcome this problem, specific limits were set on the lengths of each of the elements (domain, path, filename), and every time a string is concatenated or copied a test is done before to ensure that the new string is long enough to handle the data being inserted into it.
In order to test that this worked correctly, a testing stub was used. This is a small piece of code that sends the function being tested various test data, and checks that its results are correct. The results of one set of test data are shown in figure 7.2a.

Results from Test Stub
Test results from url_splitter:

URL to test: http://www.tesurl.com
Valid?: Yes
Output: http://www.tesurl.com/
Error:

URL to test: ftp://testftp.com
Valid?: No
Output: ftp://testftp.com
Error: ERROR - Can only download pages using HTTP.

URL to test: mailto:test@email.com
Valid?: No
Output: mailto:test@email.com
Error: This is a link to an e-mail address.

URL to test: http://www.testwierdchars&abc.123/456%2,aa-f*.html
Valid?: Yes
Output: http://www.testwierdchars&abc.123/456%2,aa-f*.html
Error:

URL to test: http://www.testlongdomain56789012345678901234567890.com
Valid?: Yes
Output: http://www.testlongdomain56789012345678901234567890.com/
Error:

URL to test: http://www.testlongpath.com/
12345678901234567890123456789012345678901234567890/
Valid?: Yes
Output: http://www.testlongpath.com/
12345678901234567890123456789012345678901234567890/
Error:

URL to test: http://www.testlongpath.com/
12345678901234567890123456789012345678901234567890/a.htm
Valid?: Yes
Output: http://www.testlongpath.com/
12345678901234567890123456789012345678901234567890/a.htm
Error:

URL to test: http://www.testlongfname.com/
12345678901234567890123456789012345678901234567890.htm
Valid?: Yes
Output: http://www.testlongfname.com/
12345678901234567890123456789012345678901234567890.htm
Error:
Figure 7.2a – Test Results using Test Stub

A similar method was used to resolve some other problems that were encountered during implementation. In many cases the debugger provided with Microsoft Visual Studio was sufficient to solve the problems encountered.

A list of the tests carried out on the system, along with the results of the tests is included in Appendix G.


7.3. Field Testing

To really ensure that the system works properly, it should not only be tested by the programmer, but also by users of the system. Often users can discover problems that the programmer did not consider or even think possible.

The system was developed as a set of incremental prototypes, with new features being added to each version before it was released for field tests. Colleagues at the University tested the early versions of the system, with only the final version tested by blind users.

One of the main problems highlighted by field-testing was a problem with the WinSock interface provided with Windows. It was discovered that only three concurrent Internet connections can be used, if more are used the extra connections will not receive any data. To prevent this occurring a limit of three was set on the number of concurrent connections that can be made.
This does have the side effect that the user of the system cannot use Internet Explorer to browse the web while the system is downloading the news otherwise some of the pages in the newspaper will be blank. This is an unavoidable problem, and is documented in the user instructions (Appendix C).


7.4. Testing Results

Despite development and field-testing, some problems still remain with the system. These are largely due to the complexity of the system and the timescale of the project.
The system works well enough to demonstrate its usefulness, but it still suffers from changes to the design of the web pages it uses to compile the newspaper.
Some problems were never resolved, for example the system does not work at all with BBC web pages. Given more time this problem could easily be resolved.