I presented this deck at the Bay-CHI Birds of feather meeting at Yahoo on June 22, 2010.
The deck provides a brief history of eye-tracking and the the kinds of decisions we make using the method.
1. Prasad Kantamneni Eye-Tracking At Yahoo! Overview of the Method Bay-CHI: Eye-Tracking Birds of feather meeting. June 22, 2010
2.
3.
4.
5.
6.
7.
8. High resolution image seen by the Fovea Reduced visual acuity experienced by the parafovea Progressively reducing visual acuity from the periphery of the retina
9. Users use parafoveal preview to identify the parts most likely to have relevant information based on the location of boldfaced terms
10. Familiar summary patterns draw user attention and clicks Users are relatively blind to unfamiliar summary patterns.
11.
12. Learned to test Models with click logs Design A Conversational title style Design B To the point title (query term – property)
13. Learned when and how to introduce changes into a UI <video> Yahoo! Presentation
14. Learned when not to introduce change Rapidly Evaluating new ideas, and predicting user behavior Launched with Keywords instead of image thumbnails -- despite more positive response to thumbnails – because of cognitive overhead.
15. Learned how to quantifying the effectiveness of a UI and optimize for learning Old Yahoo! Y! with Search Assistance
We did a number of different Experiments including changing the way certain terms are bolded -- showed significant double digit % increase in clicks, and user engagement – which helped us quantify the impact, and understand more about why’s
People make a number of subconscious decisions. In this case this abstract would have significantly eroded perception of relevance, and increased user frustration.
Even though users said that they liked the design with the image thumbnails – the actual behavior indicated that users tended to avoid the images because they thought of them as ads or irrelevant content. Additionally there is a certain amount of cognitive overhead associated with switching contexts from scanning text to scanning images – as a result we decided not to try to change habits -- because in this case the behavior is hardwired into users.
We were able to quantify the effectiveness of a user interface using eye-tracking. Senior leadership was not interested in launching search assistance, until we were able to prove a 14% increase in page effectiveness – at which point the Search organization quickly agreed to support a launch. Yahoo launched Search assistance in 2007 to rave reviews. This significantly improved user experience and market share. Google followed a year later. Search assistance is now a standard feature for anybody searching
Eye tracking predicted a 6-8 week learning curve for search assistance. Once launched click logs confirmed the findings. The Click log findings on learnability were presented at SIGIR 2008 ( A longitudinal study of real-time search assistance adoption, Peter Anick, Raj Gopal Kantamneni ) Planning for learnability is critical for the web because the cost for switching is minimal. If something takes 20 weeks to learn users are likely to migrate to a different product rather than learn the new experience even if it will eventually be a better experience. At Yahoo! I usually shoot for learnability to be under 6 -10 weeks for search. Other products have different thresholds depending on the nature of user interaction.
The rich advertisement program increased ad conversions (not clicks!) significantly. Currently this program commands a premium in the Yahoo! ad marketplace.
The same heat map will mean different things depending on the user intent. This heatmap on the yahoo shopping experience is bad. As a result the page was redesigned to better match user inent.