Amela Karahasanović Research Scientist, SINTEF IKT Adjunct Associate Professor, DESIGN

Transkript

1 Amela Karahasanović Research Scientist, SINTEF IKT Adjunct Associate Professor, DESIGN Sintef IKT, Forskningsveien 1 User modeling Usability and user experience Methods for empirical evaluation in HCI and SE Software maintenance Program comprehension

2 Technology for a better society Largest Scandinavian independent research organisation More than 2100 employees from 67 countries Generated knowledge for 2.8 billions of NOK in 2010 Eight research divisions About 16% of our turnover is derived from international projects 2

3 HCI research group Psychology Software Engineering Media science 3

4 Wednesday 19 th September 10:15 12:00 Bobble experiment (3D technology) Living Lab studies Collaboration tool 4

5 Plan for today Chapter 10 + some examples What is usability testing Usability testing versus traditional research Types of usability testing User-based testing Questions for discussion and homework 5

6 What is usability testing? Involving representative users, representative tasks, representative environments Testing paper prototypes Wizard of Oz Testing working version before it is released Testing working system Different devices (smart phones, laptops ) 6

7 Goal of usability testing Improve the quality of an interface by finding flaws in it Flaws that cause problems for the majority of people 7

8 8

9 Usability versus research? Approach similar as one used in classic research Different goals Usability testing is involved in building a successful product (resources, time, trade-offs) Practical, large impact Optimize interface after each iteration Usability study Research study - Research on usability methods 9

10 Usability engineering Activities aiming to improve the ease of use of an interface Expert-based testing (usability inspection) Automated testing (usability inspection) User-based testing (usability testing) 10

11 Expert-based testing Structured inspections done by interface experts Before tests with users Confusing wording, inconsistent layout, obvious flaws Heuristic review Compare interface with the rules Consistency inspections Series of screens or web pages inspected Cognitive walkthrough Experts perform the tasks (high-frequency and important/seldom) Guidelines review Web Content Accessibility Guidelines 11

12 Automated usability testing Software that compare interface with the guidelines Produce report and/or fix the code Manual check often needed <alt> tag but not if the text is appropriate Number of fonts, avg. font size, deepest level of a menu RAMP, InFocus 12

13 User-based testing Select representative users Select the setting Decide what tasks users should perform Decide what type of data to collect Before the test session (informed consent, etc.) During the test session Debriefing after the session 13

14 Questions Part I Go to the alarm app on you phone Difference between a summative and a formative usability test of this app Should we first test it with users or with experts? What do you think? Why? What are the differences between a usability test and an experiment with this app? 14

15 Stages Users perform tasks (early or later in the development) Formative testing Low-level fidelity prototypes How the user perceives an interface component? Low-cost of paper prototypes, users comfortable to criticize Summative testing Evaluate the effectiveness of specific design choices High fidelity prototype Validation test Before release, compare to benchmarks Formal approach 15

16 Usability evaluation Talhonia time 16

17 Usability evaluation Talhonia cont. 17

18 How many users? 5 users will find approximately 80% of problems 7 for small projects, 15 for medium-large projects Goal to find the major flows, that will cause the most problems and must be fixed How many users can we afford? How many users can we get? How many users do we have time for? 18

19 Location Lab Portable usability lab Remote studies (time, place) 19

20 Tasks Clear; no need for further explanation Tested One clear answer way to do things Tasks that are performed often Critical tasks logging No private, financial information Must be clear how to go to the next task 20

21 Oppgave 4: Du er hjemme og du ser NRK Nett- TV på din PC ( ). Se på episoden 10:12 av Matlyst og svar følgende spørsmål: Hva heter kvinnen som er kledd i lilla genser og brunt forkle? Når en potet frøs om vinteren, hva ble den da? 21

22 Subject_id 15 Task 2 Measurement Task performance Time performance User satisfaction (validated survey tool) Average time to recover from an error Time spent using help Number of visits to the search feature Time spent on specific web page Typing speed Qualitative data Sum of Seconds Minutes Visited Unified Library Application - Quit Unified Library Application Title Information ReturnItem Reserve Title Lend Item Insert Title Window Find Title Compilation 22

23 Starten av dagen: Sjekke at alt er på plass i laben; gavekort, utstyr, godteri, brus, batterier, pen og papir osv. Skrive ut eksperimentet material som skal deles til deltagerne Skrive ut tlf.nr. til de som kommer (mail fra Nordstat) Informere resepsjonen at vi venter deltakere. Hvis deltaker kommer veldig tidlig, be resepsjonisten henvise dem til sofa i resepsjonen. Logge seg på PC og teste alt virker (ingen updates skal dukke opp i løpet av eksperimentet) Gå inn på facebook og twitter og sjekk at thomtor er logget ut Heng IKKE FORSTYR på døra Ha alltid mobilen med deg Eksperimentet (før deltakeren kommer) Start SINTEF verktøyet (r2d2exp.jar). Det kan ta litt tid IKKE KLIKK FLERE GANGER! Start opp camtasia studio (dobbelt klikk) Start opp ulog (høyre klikk på ikonet i system tray og start) Eksperiment (hent deltaker) Ta med deltageren ned; hilse; introduksjon Konfidensialitet skjema signeres Husk å sette mobilen på vibrasjon Start opptak i camtasia studio (høyreklikk, klikk record the screen, klikk record button) 23

24 24

25 Making sense of the data Write up the results and help influence the design of the specific interface Presentation to developers and managers Report should Include all flaws Priorities For each flaw: describe the problem, present the data, priority, suggest a fix, estimate time/efforts for the fix Report structure How you did usability testing and how you prepared What happened during the testing The implications and recomendations 25

26 Questions Part II Alarm app usability test What is important to remind participants before the usability test? What is the "think aloud" protocol? Should it be used in summative or formative studies? Properties of good tasks? 26

27 Assignment Make a PLAN for a summative evaluation of the Contacts functionality on your mobile phone in a usability lab with 10 participants. 27