Testing & Evaluation

March 18, 2009


Testing, Evaluation & Documentation

A checklist of checklistsLaunching a flawed application onto the market can be a very costly mistake.  Make sure you have got it right, your job and your reputation are at stake! Also, don’t do all this at the end of the project, anticipate problems at the early design stages, its a  lot cheaper and easier to fix them then than towards the end of a project when the design is relatively closed and expensive to change.

Usability testing

What is usability testing?

Usability testing generally involves measuring how well test subjects respond in four areas: time, accuracy, recall, and emotional response. The results of the first test are the baseline or control measurement; all subsequent tests are compared to the baseline.  A typical usability test would cover the following for around five to ten users

Time on Task — How long does it take people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)
Accuracy — How many mistakes did people make? (And were they fatal or recoverable with the right information?)
Recall — How much does the person remember afterwards?
Emotional Response — How does the person feel about the tasks completed? (Confident? Stressed? Would the user recommend this system to a friend?)
Short article on Practical usability testing
US Gov comprehensive usability resource
Series of tutorials from IBM on design for ease of use

Screen Testing

Each screen of an application should be tested to ensure that it complies with the design and works properly.  A checklist can be used to work through each item.  Below is a typical example which could be adapted to your project

Does the layout match the design?
Do all the buttons work?
Do all the texts, fonts colours and sizes match the original design spec?
Do  text scroll functions or animations work?
Do the non functioning controls grey out when not needed?
Is there a visible continuity between screens?
If there are any applets, SWF’s or other plugins and do they operate correctly?
Has the text been spell checked?
Do video and audio files load quickly enough and play smoothly?

Navigation testing

Is your navigation scheme logical and fully functioning?  A typical test checklist would include

Do all the links within the application work?
Do all the hyperlinks function correctly?
Are there any broken links?
Is there a smooth transition between screens?
Is there consistency between screens?

Accessibility

How many different types of people can use your application without modification?  Is your design inclusive?  Can it be used by as many people as possible regardless of their age, ability or situation?

W3C web accessibility guidelines
RNIB good design advice.  There are two million people with sight problems in the UK. Good design can make websites, information, products, services and buildings accessible to them.
About Inclusive Design at the Design Council

Security

Are there any security issues that have to be covered?

Dancing pigs (warning this link could be insecure, your bank account could be cleared out and all your children be born with tails)

Evaluation

Any multimedia application should be evaluated especially in terms of its fitness for purpose.  Have you met all the requirements of the design brief?  Will you get paid?  Some contracts require the developer to produce an evaluative report prior to product launch.

Does the application do what it is intended to do?
Has the designer designed the best solution for the problem?
Does the application work in an efficient manner?
Is the application robust?
Is the application easily maintained?
Has a technical and user guide been provided?

Documentation

People need to know how to use your application and that it will work on their machine.  Do not assume they will know what to do or have any technical knowledge.  If they can find a way to break it they will!

Instructions – “……….press any key”
User – ” Where’s the any key………?”

You will need to establish the platform and lowest specification that will play your application.  A typical specification would include

Operating system
Clock speed of CPU
RAM
Hard drive space
Optical drive type
Sound card  configuration
Video card configuration
Screen resolution
Internet access

Instructions for using the application and sources of support also need to be included.  For example

Insert into DVD drive
If auto run does not work navigate to “game” folder
Left double click on game icon
How to write instructions for busy, grouchy people.

Copyright

Is it all your own work? Any multimedia application must take into account the Copyright, Designs and Patents Act. The creator of a multimedia application must ensure that all appropriate licences and permissions have been obtained for their particular purposes.

General copyright issues
ver 12/5/6 cb

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: