2. Test Pilot is a opt-in platform aims to
improve the Web by collecting structured
user feedback from Firefox and other
Mozilla Labs experiments.
3. The Test Pilot platform
1.We run Test Pilot as a standalone opt-in program and not
as part of Firefox.
2.We will never collect user information without first having the
participant’s explicit permission.
3.We only transmit and store anonymous and
sanitized information in order to protect individual
privacy.*
4.We respect participants and their privacy by making test
data knowable and manageable for individuals.
4. Design question 1:
"What should the default action be when the user clicks in
the location bar for the first time?"
Design question 2:
"When user clicks a link, where this link should be opened as
a tab, at the end of the tab bar, or near the current tab?"
5. Firefox is a vehicle for 300 million people carrying their
work and lives on the Internet.
How might we help product teams make better decisions to
enhance user experience?
6. A good statistic aids a decision or shapes an
opinion.
How should we protect our user privacy while
we collect structured data for statistics?
7. A insightful statistic does not necessarily require
user private data.
Examples:
1. What are top-5 ubiquity commands people use the most?
2. How often people use the down-arrow near awesome bar and when?
3. How many tabs do people open?
8. Test Pilot Mission:
1. Improve the open Web experience by making anonymous,
aggregate usability & usage data available to developers as a
shared public resource.
2. Respect the user & their privacy by making test data
knowable and manageable for individuals.
9. Test Pilot Platform:
Collecting structured user data to improve product design.
https://testpilot.mozillalabs.com
10. Privacy and security with the utmost importance
Two focuses:
1. Establishing data privacy policy
2. Enabling participants’ controls on their data
12. Give participants’ controls in the entire process
User controls on their data:
1 2
People opt in to the People participate in a specific Test Pilot study
Test Pilot program
1. Introduce Test Pilot
and its purpose. Before a study starts During the study Study complets
2. Introduce privacy policies: 2-1 2-2 2-3
2.1 User control
1. Introduce what this 1. Reinforce that no
2.2 Data collection and storage
study is and what data PII will be shared
2.3 Data sharing
need to be collected.
People are able to People are able to
3. Clarify how this program may
2. Clarify if this program review what study review their data.
change current product
may change current they are participating.
performance:
Firefox UI. People submit the
3.1 Won't lose any existing data,
study data or leave.
e.g bookmarks.
People opt-in a study,
3.2 UI may or may not change.
or leave.
People opt-in to Test Pilot.
People are able to opt out at any time.
No data will be reported until people submit.
People can opt out of the Test Pilot program at any time.
13. Let participants see the data as it is being
collected and before it gets sent to the server.
https://testpilot.mozillalabs.com
14. First large scale privacy-sensitive usability study with
the Test Pilot framework around tab usage completed
with over 5,000 participants, results and raw data
posted under Creative Commons license
15. How Math Teachers Can Help Improve the Web?
https://testpilot.mozillalabs.com
16. Try it out and improve the
Web together!
https://testpilot.mozillalabs.com
Twitter: @MozTestPilot
Hinweis der Redaktion
IP addresses will be transmitted to Mozilla servers (as part of regular internet communications), but they will NOT be associated with test data. We may keep IP addresses as web log data, but only for a certain minimal amount of time, and only for the purposes of optimizing the service and preventing abuse.
Working in Firefox team, you can hear that people trying to find answerers for these type of questions all the time:
question 1, &2.
every small detail is important to our product, Firefox, which is .....
[goest to slide 2]
used by over 300 million people daily.
How might we help product teams make better decisions to enhance user experience?
by understanding what data are helpful, what data are private and belong to users, and how we could get helpful data and not touching user private data, Test Pilot team sets up a mission before for this project:
mission 1
mission 2
The basic framework for Test Pilot project is:
1. pre-define what questions the product team want to answer and what data are meaningful to this question, the product team and UX practitioners work together to develop the test plan and invite people to join the test.
2. after people participated in the study and agree to submit the test data, the anonymous and aggregated data will be analyzed for test results.
3. All these activities are processed in a open and transparent environment, that the public can see what happened.
During the test participation, we are aware that privacy and security are keys to the success of building a trust worthy program.
So we focused on these two areas before we can build any engineering work.
the Test Pilot team worked intensively with lawyers and engineers, to develop a data discipline that fully respect participants privacy, and all later engineering work are built around this data discipline.
No personal information.
No personally identifiable information.
Very Careful with potential personally identifiable information.
People need to opt in to the Test Pilot program;
People need to opt in to a specific study after they join the Test Pilot program.
People can opt out the study or the program at anytime.
People can choose to submit the data or not when a study completes. Unless users submit the data, no information will be transfered to Mozilla server.
People can see what data are collected in the local computer in real time.
The Test Pilot team takes openness one step further. Unlike data collected by that other browser company, the aggregated data samples collected by all of the Test Pilot plug-ins is freely available for download. This means that if you’re a Math or Statistics teacher, you can build lessons around Test Pilot data-sets from the real world that your students helped create by having installed the plug-in.
For any questions, you can contact the Test Pilot team with this email.