![Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Empfohlen
Weitere ähnliche Inhalte
Andere mochten auch
Andere mochten auch (14)
Ähnlich wie Designing a Successful Eye-Tracking Study UPA 2008
Ähnlich wie Designing a Successful Eye-Tracking Study UPA 2008 (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Designing a Successful Eye-Tracking Study UPA 2008
- 1. Designing a Successful Eye Tracking Usability Study Step-by-Step Andrew Schall Senior Usability Specialist Human Factors International UPA 2008, Baltimore, MD Thursday, June 19 th 2008 Anne Washington Ph.D. Candidate George Washington University
- 6. How Eye Tracking Works
- 7. How the Eye Tracker Locates Your Eyes Cornea Iris Pupil Near-infrared light
- 8. Fixations are Mapped to Points On-screen X Y
- 10. What Can Eye Tracking Tell Us?
- 12. Courtesy of Bunnyfoot by way of Future Now
- 13. Can we get them to change where they look?
- 15. What are users interested in (or not interested in)?
- 16. What are users interested in (or not interested in)?
- 21. How quickly do they learn?
- 22. Measuring efficiency Average Fixation Count Before After
- 24. Planning for Analysis ^ Ahead
- 25. Eye Tracking Produces lots of Raw Data
- 27. Plot Where & When They Looked
- 28. Categorize the Areas for Analysis
- 29. Aggregate Where People Looked
- 34. Using a Pre-task Question What are some of the things that go into your decision when you are looking for a seminar for continuing education credit?
- 35. Designing Task Instructions Find a seminar that you would like to enroll in. Enroll in a seminar available this week in San Jose on the topic of taking depositions.
- 36. Using Retrospective Think-aloud I noticed that you looked at _____several times… When you were looking for _____ you didn’t seem to notice _____ ? What were your impressions of that task?
- 37. End of Session Team Debriefing
- 39. Making it Happen
- 40. Eye Tracking Lab Configurations
- 41. How Many Participants? How many people do you need for consistent & reliable results?
- 42. 1 Participant (Low Validity) Images courtesy of www.realeyes.it
- 45. 4 Participants (Low Validity)
- 48. 10 Participants (Low Validity)
- 49. 12 Participants (Low Validity)
- 50. 15 Participants (Medium Validity)
- 51. 17 Participants (Medium Validity)
- 52. 20 Participants (Medium Validity)
- 53. 22 Participants (Medium Validity)
- 54. 25 Participants (Medium Validity)
- 55. 27 Participants (Medium Validity)
- 56. 30 Participants (High Validity)
- 57. 32 Participants (High Validity)
- 58. 35 Participants (High Validity)
- 59. 37 Participants (High Validity)
- 60. 40 Participants (High Validity)
- 61. 42 Participants (High Validity)
- 62. 45 Participants (High Validity)
- 63. 47 Participants (High Validity)
- 64. 53 Participants (High Validity)
- 65. Some claim 8 people are enough to analyze visual attention…
- 66. 8 people Sample A
- 67. Sample B 8 people
- 68. Sample C 8 people
- 69. Sample D 8 people
- 70. Sample E 8 people
- 71. What about a sample of 15 people?
- 72. 15 people Sample A
- 73. Sample B 15 people
- 74. Sample C 15 people
- 75. Sample D 15 people
- 76. What about samples of 30?
- 77. Sample A 30 people
- 78. Sample B 30 people
- 79. Sample C 30 people
- 80. So what does that mean?
- 81. Qualitative, informal, formative Quantitative, statistically significant, summative > 30 > 30 > 15 < 15
- 82. Don’t Recruit These People
- 84. Listen
- 85. Observe
- 86. Ask
- 88. Be Sure Your Data is Collected Accurately Good Calibration Poor Calibration
- 89. Checking the Eye Tracking Status Good lock on both eyes Unable to track eyes Lock on one eye
- 90. Note-taking Strategies Task 1 Watch an animation that demonstrates how a wind turbine works. Left navigation Top navigation Center links Feature box Task Where initially looked Comments Participant quickly glanced at feature boxes but didn’t notice the one about wind power Task 2 Find out which team won this year’s Solar Decathlon. Left navigation Top navigation Center links Feature box Participant quickly noticed and clicked on the Solar Decathlon link in the top navigation
- 91. Need a Volunteer
- 92. Vs .
- 95. Positioning the Participant Replace this slide with an audience participation activity. Have the participant demonstrate some of the movements that can affect the eye tracker
- 96. Pre-task Question What are some of the things that go into your decision when you are looking for a course or seminar for continuing education credit? Replace this slide with an audience participation activity. Ask the participant the pre-task question.
- 97. Task Instructions Enroll in a seminar available in San Jose on June 9 on the topic of taking depositions. Find a seminar that you would like to enroll in. Replace this slide with an audience participation activity. Ask the participant to perform the task.
- 100. But…We Don’t Have an Eye-tracker
- 101. Q & A
- 102. Thank You Andrew Schall [email_address] Anne Washington [email_address]
- 103. Room Layout – Conference Room Participant Note-taker Moderator
- 104. Room Layout – Traditional UT Lab Participant Note-taker Moderator
- 105. Room Layout – HFI Baltimore Lab Participant Note-taker Moderator
Hinweis der Redaktion
- These images are not in fact medieval torture devices, but old-style eye-trackers that some of you may have seen before.
- Newer eye-trackers that can be built right into a flat screen monitor and do not require the participant to wear anything or force them to restrict their movement.
- The Tobii system is accurate to within a half a degree It takes only about 30 seconds to calibrate a participant promote more natural user behavior by not placing unnatural restrictions on participants Test administrators can quickly learn the basics without expensive training Results are produced immediately and often do not require extensive data analysis and number-crunching
- Most modern eye-trackers are binocular and video-based. This means that the eyetracker uses two high resolution video cameras that records reflections of near-infrared light off each eye. When infrared light is shone onto the eye, a reflection occurs on the boundaries of the lens and cornea. These camera systems are connected to a high-end pc with sophisticated software for calibration and stabilization of the image. ------------------------------------------------------------------------------------------------------------------------------------ Camera A high-resolution camera with a large field-of-view is used to capture images of the subject’s eyes. NIR-LEDs Near infra-red light-emitting diodes (NIR-LEDs) are used to generate even lighting of the subject and reflection patterns in the eyes of the subject. Optical filters Optical filters are used to block sun-light and other sources of interfering light. Control electronics Control electronics are built-into the screen to control the monitor, the camera and the NIR-LEDs
- One of the most useful analysis tools is the eye-gaze replay. It allows you to see where the participant looked during the test session. The eye-tracking software examines the video footage in real-time. As it analyzes the video, it records the x and y coordinates of where the user looks, usually referred to as fixations. (Animate) Each fixation appears as a dot on the screen. The screen makes it look like a game of connect the dots as the computer draws lines from one fixation to another. The computer also tracks how long a user remains focused on a particular point, often referred to as the fixation duration. The quick movements from dot to dot are called saccades.
- Now I’ll demonstrate the eye--tracker with help from someone in the audience…
- Which design is more effective?
- These two designs were evaluated using eye tracking. These visualizations are heatmaps representing where most people looked. It has been proven that people are attracted to images of faces, so it makes perfect sense that both images have received attention. Notice the heat that is drawn from the woman on the right as she looks directly at the product. People followed her gaze and looked directly at the brand label. So, how does this relate to user experience?
- Here is another example where users are looking everywhere but where we want them to. This area contains lots of unnecessary text that people ignored and spent most of their time looking for a way off this page.
- By redesigning this page to contain more actionable items, people took the time to read the links and even clicked on a few of them.
- What area or areas on this page do you think users were least interested in?
- I’m sure that this greatly disappointed the designers when their big fancy Flash piece received little attention from users. The animation took too long to load in and make its point. By then, most people had written it off as ad and didn’t bother to look at it.
- Library of Congress Study for Legislative Information System (LIS). Tool used by congressional staffers. Need to find information quickly and easily. High turnover, frequent new users to system. Left image shows the advanced search page with boxes highlighting the areas most users will need to use. The right image show the aggregated heatmap showing where users looked.
- Visual design: Tests visual hierarchy, whether users notice specific items or spend too much time look at. It is often difficult to choose a visual design for a website. Individuals in the customer’s organization, designers and developers often have strong opinions on what will work or won’t work in the visual design of a specific site. These opinions are just that, opinions. They are often not backed up by any proof at all. It is extremely difficult to convince someone that their opinion of how a page should be laid out is incorrect. Eye tracking provides conclusive, measurable and repeatable performance measures of a particular visual layout. Even with small sample sizes and an early prototype page you should be able to effectively choose and justify the visual designs to your customer.
- Users’ look patterns change the more they use an interface Better designs make it quicker & easier for people to know where to look in order to accomplish tasks Users learned where the links were located and didn’t bother to look at other areas on the page
- Testing Design Effectiveness Where are my users looking? Discover where users look most of the time, and where they look the least Discover where they look initially upon page load Discover where they look during a given task
- **Supplement this slide with a paper printout of raw data with several pages to demonstrate the enormous amount of data** The eyetracker generates a lot of raw data. Luckily, there are tools that can take all of this data and convert it into something that a person can readily understand. Horizontal & vertical coordinates (where) Time of fixation (when) Duration of fixation (for how long) Screen recording Pupil size Keyboard & mouse clicks URLs Audio Participant video
- Here you will see a video clip from a recent eye-tracking study. You will first notice that the eye moves around the screen very quickly. To really grasp what the user is seeing, I found it helpful to review segments of the footage several times. Reviewing in slow motion can also help. You might have also noticed that some fixations dots are larger than others. The larger the dot, the longer the user focused on that particular area.
- Static display of eye-gaze points and scan paths Shows the exact sequence that the user’s eye followed Can be adjusted to show one or more participants, and specific periods of time
- Defined by the researcher after the session Automatically performs analysis based on defined regions: # of fixations Duration Frequency Time to first fixation Areas of interest can be defined after a session or study is over. This tool allows you to define specific regions of the screen to Duration= how long a person looked at this region Frequency=how often they looked back at this region
- Automatically generated diagram of the evaluated page with an overlay of frequently looked at areas Heatmaps can display data from a single user or a combination of users Areas of the page with the greatest frequency of eye fixations are overlaid with a red and orange “hot spot.” Yellow and green &quot;hot spot“ overlays are used to show smaller frequencies of eye fixations. Screen locations without “hot spot” overlays represent screen locations with no eye fixations. Purple X’s represent mouse clicks Red line indicates page fold
- Retrospective interviewing with or without scan path replay. Having the eye-tracking information as a supplement to think-aloud and video of the user’s expressions can provide a level of understanding that is difficult to replicate without the eye tracking data. One could think of the eye tracking information as providing a context to the expressions and think-aloud information one is trying to understand. You can see exactly which piece of text or button is causing them to frown, what they are looking at when explaining their frustration. This can provide vital clues to understanding the breakdown event and proposing an optimal solution to the issue. Validate (or invalidate) user supplied data
- Sessions should last no more than a total of 1 hour from calibration to debriefing 1. The Tee Up --Start at a search engine ---Look at how users visualize and select results 2. Short period of open exploration “ Feel free to look around on this page without clicking on anything just yet” --To find out where the user looks without any direction or specified task -- Collect 15 seconds worth of data ---- Natural visual hierarchy without being influenced ----- Homepage -------- Landing pages 3. Pre-determined tasks -- Be as specific as possible --- Eyetracking data is difficult to analyze when tasks are performed without a specific goal… 4. Retrospective think-aloud between tasks -- Validate the eyetracking -- Discuss any unique or special eye movements that you noticed 5. Debriefing
- Develop a pre-task question to better understand the user’s mental model as it relates to the task
- Ask specific tasks without room for misinterpretation. Specific tasks are especially important when using eye tracking because you will not be using think aloud. Have a clear indication of task completion or failure GOOD: Participant clicks on a link and says out loud “I’m done” or “I found what I was looking for” BAD: Participant has found what she wanted but continues to look around the screen If possible, randomize the task order to reduce bias
- Traditional think-aloud protocol can interfere with the accuracy of eye tracking data collection Retrospective think-aloud allows participants to concentrate on the task Immediately after the task, the moderator can prompt the participant to talk about their experience performing the task RTA has been proven to provide accurate accounts of participants’ experiences (Guan, Zhiwei, et al)
- End of Session Team Debriefing What did we learn from watching this session? Are we seeing any patterns amongst users? Did we discover any new issues with the design? Why do this? -- Helps to make sure that everyone “saw” the same things -- Gets all aspects of the team (developers, management, designers) involved -- Helps to find trends
- The f ollowing slides show how many participants are needed to get a reliable visual analysis of a web page
- The f ollowing slides show random sets of 8 people from the same study shown on previous slides.
- Variation is still too large for samples of 15 people.
- Heatmaps of random 8 people out of all 53 participants were very different. It means sample of 8 can give a very misleading picture of true visual attention.
- Some people can unknowingly be poor participants Very thick eyeglasses (high prescription) “ Trendy” eyeglasses (small frames) Mascara (makeup issues) Bi-focals or tri-focals (progressives are ok) Poor vision without correction (sit too close to screen) Scarring on the eye due to surgery or injury
- Be quiet while the participant is performing the task. Talking to the participant will affect what they look at and will contaminate the eyetracking data. Also discourage participants from talking to you will they are performing the task.
- Actively pay attention to what the participant is looking at and make note of anything that you find interesting and that you want to address with the participant at the end of the task.
- At the conclusion of the task, ask the participant about their experience with the task and about anything that you noted of interest that you would like them to explain.
- Most modern eye-trackers use software to calibrate each participant’s vision with the eye-tracking system. This is a necessary step before proceeding with your test because the calibration routine “teaches” the eye-tracker how the eyes of a particular participant look and behave. (Begin animation) The drawing on the screen represents the monitor that the participant will use in the test. The typical calibration routine includes between 8 and 16 dots. The participants are instructed to look at the dots as they appear on the screen. Most participants actually find this exercise kind of fun. In my experience, most participants are curious about the eye-tracker but typically forget that they are being tracked after a few minutes.
- Why do you need a good calibration? --If the calibration is significantly off, so will your data. --If you can’t get a good calibration, do not use the participant’s eye tracking data. Good calibrations have several short lines within each of the circles Poor calibrations have long lines and in some cases no lines within each circle
- The eye gaze status box shows the current tracking status for the participant’s eyes This box should be left on the moderator’s screen during the test What happens if you loose their eye tracking during the middle of the session?
- Now we will try out some of the methods I just talked about. I’d like to have someone from the audience volunteer to be our mock participant.
- We are going to compare two websites today to determine which one makes it easier to locate the price for…
- Crest Whitestrips Premium Plus So what are we trying to determine with this task? The obvious: can they locate the price of the product With eye tracking : What are some of the visual cues that they notice or more importantly do not notice that would help them to complete the task? What are some of the other things that they notice along the way? Do they seem to notice other products that they might be interested in? Pre task If you were looking for a product to whiten your teeth, where exactly would you expect to find it in your local drug store? Task Find out how much Crest Whitestrips Premium Plus costs.
- Each new participant must run through the calibration sequence. This helps the eyetracker to lock on to the participant’s eyes based on a series of calibration points.
- The eyetracker will be unable to track a person’s eyes if they sit too close or too far away from the screen Try to be proactive, and ask participants to sit naturally, but within range of the eyetracker If you notice that they are out of range, gently ask them to reposition themselves in their chair
- Develop a pre-task question to better understand the user’s mental model as it relates to the task
- Ask specific tasks without room for misinterpretation. Specific tasks are especially important when using eye tracking because you will not be using think aloud. Have a clear indication of task completion or failure GOOD: Participant clicks on a link and says out loud “I’m done” or “I found what I was looking for” BAD: Participant has found what she wanted but continues to look around the screen If possible, randomize the task order to reduce bias
- Understand the limitations of the technology – It is not a mind-reading device, must be combined with traditional methods Plan ahead on how you will use the data – Planning ahead means more than just deciding early on to include eye tracking in your study. You need to think about the specific aspects of the design and specific questions you want answered. Do not wait until after you have collected the data. Steer clear of potentially bad participants – Make sure that you properly screen participants to increase the likelihood that they will be tracked. Follow Listen, Observe, Ask Moderating Technique – To get the most accurate eye tracking data, follow the listen, observe, and then ask technique. Monitor eye tracking status– Make sure you are actually collecting data. Keep an eye on the status of the tracking and use it to anticipate problems. Always perform a pilot test – Pilot testing is especially important when including eye tracking in your study. You should make sure that you are comfortable with your test protocol and that you have thoroughly tested the equipment.
- Don’t feel left out. Not everyone has an eye tracker. There are options: Buy one (eye-trackers are expensive) Rent an eye-tracker Use a usability lab that already has an eye-tracker Hire someone to do an eye tracking study for you