Oxford Computer Consultants

Welcome to the seventh edition of the OCC eNewsletter with its insight into how emerging Information Technology will impact on your business.

Our aim is to inform business managers and technical directors in clear language about topical aspects of IT. Every quarter we'll explain how businesses are using IT to gain a competitive advantage and improve their business processes.

In this issue, Cornelius Grotjahn explains how User Acceptance Testing defines the success of a project, what's involved for the customer, and how a software manufacturer like OCC can help with the process.

Manufacturer and Customer Responsibilities for User Acceptance Testing

A new piece of bespoke software is usually written when a business identifies a need for it. Having identified roughly what it needs the new software for, the business typically commissions a software manufacturer to write a new system and thus becomes the customer.

The manufacturer then goes through some development methodology to write the software. On the way to the finished product the steps followed will include:

• In a requirements analysis the manufacturer finds out from the customer exactly what the customer needs the software to do. In the process he also learns the customer's domain knowledge, i.e. how the customer's business operates.
• They then write a specification, which details how the software will operate and which features it will have.
• The manufacturer now designs the system architecture, which specifies how the system breaks into components, what these components will do in detail, and which technologies will be used in the components.
• In the implementation phase the components are coded and tested individually in unit tests.
• Then the system is tested as a whole.

At the point of delivery and before deploying the software, the customer usually performs a User Acceptance Test to make sure that the new software works.

User Acceptance Testing

As is obvious from the term, user acceptance testing (UAT) is a form of testing. Testing makes sure that software works properly. Problems like defining what “working properly” means, or to what extent it is possible to ensure that there are no errors in software, are well understood in the software industry, even if their solutions are more elusive.

“User” suggests that the testing in question is performed by the software's users. This does not only refer to the people who will actually key data into the software, say, but also to the customer's company as a whole. The company needs to ascertain whether the software fulfils the need that led to its being commissioned. The users are more interested in whether the software is nice to use, which to the company is only relevant in as much as it improves efficiency.

“Acceptance” refers to the process of taking delivery of new software. By accepting the software, the customer may acknowledge that the manufacturer has fulfilled his side of the contract, so acceptance has legal consequences.

User acceptance testing is usually thought of as quality assurance testing executed by the customer that happens at the end of the software cycle (if you disregard maintenance) before the deployment of the new software. Passing the user acceptance test determines the success of delivering the software and can be used as a milestone for the contract between the customer and the manufacturer.

Validation

Why does the customer need to do this testing? After all, he wants someone else to do the job of developing the software, which is what he is paying for. The manufacturer already has the responsibility to perform testing as part of development. But the customer must still make sure that the manufacturer has done his job properly, and produced a usable system, without serious errors.

Such testing is known as validation: The customer validates (as far as possible) that the software meets the specification, does not contain bugs, and that it provides good usability.

He may, of course, trust the manufacturer, particularly one who has already delivered successful software, and the cost of testing outweighs the risk of the software being unacceptable. Contractually, the customer must accept the software – with or without testing – soon after delivery. He should not surprise the manufacturer with problems later, when the manufacturer may have moved on from the project.

If the software fails the UAT, the customer will get the manufacturer to improve it before testing again. Thus the customer employs UAT to protect himself against the risk of deploying a faulty system.

Verification

Validation testing only does half the job. The software may meet the specification, but is the specification correct? Will the specified software fulfil the customer's business requirements? Testing for that is called verification.

Verification must also be done by the customer because only he can do it. Only the customer has the necessary domain knowledge to judge whether the software is specified correctly.

Unlike validation, it is not sufficient to rely on the manufacturer. This is not so much a question of doubting the manufacturer, as of mistrusting communication between customer and manufacturer. When the customer explains his domain knowledge and business processes so that the manufacturer can draw up the requirements specification, any communication problems may leave the manufacturer with the wrong idea of the customer's exact requirements. Only the customer can pick up such problems. Therefore the customer must verify the specification.

Verification happens much earlier than validation. For that reason it is not usually considered part of UAT. For the same reason verification is important. If a problem is introduced early, at the project stage of specification, then waiting until project delivery to find the problem would greatly increase the cost of fixing it. When verification fails, the manufacturer will revise the specification before implementing it. Thus the customer uses verification to protect himself against the risk of having the wrong system implemented at great cost.

Helping the Customer

Software testing is not easy. Scripts must be written that cover as many borderline cases as possible and executed meticulously. The customer cannot test individual components since he does not have access to their interfaces and specifications. Therefore the whole system must be tested, which makes testing more complex. The customer might not have the expertise for this. After all, he does not develop software, which is why he is the customer.

The manufacturer can help his customer by proposing at the beginning of the project to take over some of the UAT burden. Firstly, he can advise the customer on the testing methodology to help him overcome his lack of experience in software development. He can also extend the specification document, adding to each feature a test that shows it works. This has the advantage that the test provides an explanation of the feature, which reduces misunderstanding. To alleviate the black-box problem the manufacturer can select borderline cases for test scripts based on his knowledge of how the software works internally; or he can provide information about the internals of the software that allows the customer to choose better test cases. The manufacturer can also guide the customer through the process of verification to make sure that the right questions are asked of the specification. The manufacturer may not criticise the specification himself since it already reflects his idea of the domain knowledge, which is potentially wrong.

In order to accept this help, the customer needs to trust the manufacturer not to let any knowledge from the analysis, design, and implementation phases of the project influence the testing. Otherwise, any flaws in the manufacturer's knowledge could break the testing that is supposed to detect the effects of these flaws in the implementation. If the manufacturer is sufficiently detached, his help with user acceptance testing will make sure that misunderstandings in the specification and mistakes in the implementation are detected.

Using Consultants

There is another way for the customer to get help with user acceptance testing: he can use the services of a third party, a consultant. This would prevent pollution of testing methodology by inside knowledge of the project and its implementation.

There is, as for the manufacturer, the danger that the consultant misunderstands the customer's domain knowledge. If the two make the same mistake then the consultant will fail to detect problems with the manufacturer's specification. But the likelihood of this is reduced if the consultant has his own, independent communication channel to the customer. Any mistakes or misunderstandings by the consultant will be different from the manufacturer's and hence all problems will be detectable.

Another advantage of using a third party is that the consultant does not require the same high level of software engineering competence as the manufacturer. Instead, he can be chosen to have more expertise in the customer's domain.

Conclusion

Verification near the beginning of a project is even more important than user acceptance testing proper at the end. While the customer is responsible for both, he should make good use of help and expertise offered by the manufacturer and he should possibly consult a third party to ensure that verification and validation are carried out thoroughly.

Top

What is OCC?

The purpose of OCC (http://www.oxfordcc.co.uk/) is to create original, robust and flexible IT solutions. Our aim to add value to customers' businesses by enabling them to grasp the opportunities of Information Technology and the Internet. In so doing, we aim to give our staff challenging jobs and competitive rewards. We work in the IT field because we enjoy the technology, because we're good at it and because we can see the positive impact IT has on both business and society. We aim to achieve our purpose by:

• recruiting and retaining highly skilled staff. We believe that the intelligence and skill of our developers is one of our competitive advantages,
• working closely with our customers. It is always our aim to allow any prospective client to contact any of our previous or ongoing clients for a reference,
• using our results and reputation to win repeat business and generate new business. We believe that our reputation should speak for itself; and
• undertaking leading edge R&D because this is the life blood of innovative companies. This ensures that we have expertise in emerging as well as current software technologies.

What Does OCC Do?

OCC promotes itself as having a strong ability to grasp a client’s business needs and to use technology to “add value” to client processes. Our strengths are reflected in the quality of our development staff, our high levels of repeat business (over 93% of clients buy again from OCC), and our knowledge and experience in specific sectors such as energy, engineering, local government and health.

OCC’s Services and Expertise

Software Services

A complete range of design, development and support services for:

• Custom software applications;
• Re-engineering of software in legacy applications; and
• Content Management System based web sites and web applications based on standard and emerging web services technology.

Industry Sectors

Over 16 years of experience, reference sites and testimonials from our customers in:

• Online Training and eLearning (http://www.oxfordcc.co.uk/Doc18191.html)
• Electronics (http://www.oxfordcc.co.uk/Doc20185.html)
• Energy and Oil (http://www.oxfordcc.co.uk/Doc18197.html)
• Environment (http://www.oxfordcc.co.uk/Doc18183.html)
• Financial Services (http://www.oxfordcc.co.uk/Doc18174.html)
• Healthcare Software Solutions (http://www.oxfordcc.co.uk/Doc18172.html)
• Manufacturing (http://www.oxfordcc.co.uk/Doc20195.html)
• Telecommunications (http://www.oxfordcc.co.uk/Doc18195.html)
• Public Sector (http://www.oxfordcc.co.uk/Doc20351.html)

Technologies

All mainstream and emerging technologies including:

• Database applications;
• Mathematical modeling software with graphical outputs; and
• Web applications based on Content Management technologies, web services and the semantic web.

Socially Responsible Business Practice

Oxford Computer Consultants adheres to socially responsible business practice (http://www.oxfordcc.co.uk/Doc21009.html). The company has formal environment and ethics policies that are communicated to all staff.