now with working fonts

1. Optimizing for change:
Taking risks safely & e-commerce
Kellan Elliott-McCrea
@kellan
CTO, Etsy
Monday, October 8, 12

2. Monday, October 8, 12

3. Launched June 18, 2005 in Brooklyn
875,000 monthly active sellers
33.5MM items for sale
$525MM in sales in 2011
1.43B page views, in Aug
102 engineers
74 releases, yesterday
Monday, October 8, 12

4. Take more risks.
Build a better software.
Have more fun.
Monday, October 8, 12

5. “Sure that works when you’re
building social software but
what about a real business
with $$$ involved?”
- everybody always
Monday, October 8, 12

6. Continuous
Deployment:
small changes,
pushed frequently
Monday, October 8, 12

7. you can’t avoid making
mistakes
you can avoid making
BIG mistakes
Monday, October 8, 12

8. What are you optimizing for?
MTTR MTBF
Monday, October 8, 12

9. MTTR MTBF
Monday, October 8, 12

10. 4 core techniques:
1. Put a Button On It
2. Branch in Code
3. Trunk is Always Deployable
4. Dark/Incremental Launches
Monday, October 8, 12

11. Put a Button On It.
Monday, October 8, 12

12. Branch in code:
use features flags
4 core techniques:
if ($cfg[‘awesome_new_search’]) {
# new hotness
$rsp = do_solr();
} else {
# boring old stuff
$rsp = do_grep();
}
Monday, October 8, 12

13. Branch in code:
use features flags
4 core techniques:
for free you get:
1% launches
admin only launches
dark launches
split tests
Monday, October 8, 12

14. any engineer can launch an
experiment to
1% of users
57 experiments live right now
Monday, October 8, 12

15. Metrics driven development
measure everything!
feedback loops!
Monday, October 8, 12

16. Engineers love to measure
make it ridiculously
easy
Monday, October 8, 12

17. Metrics driven development
StatsD::timing("page.render", $msec);
Monday, October 8, 12

18. Metrics driven development
Monday, October 8, 12

19. Metrics aren’t optional
a feature isn’t done
without metrics
Monday, October 8, 12

20. Make metrics visible
remove the
passwords
Monday, October 8, 12

21. Some tools:
Graphite, Ganglia, Logster*,
StatsD*, event beacons, log
files, EMR, Vertica, Splunk
Monday, October 8, 12

22. Getting started? Use StatsD
StatsD @
Instagram, Pinterest, Github,
Mozilla, LAN.com, Zynga,
Kickstarter, LivingSocial and 70+
other companies
Monday, October 8, 12

23. Step 1: your 5 core metrics:
@ Etsy:
sign ups, logins, checkout, new
listings, posts in the bugs forums
Monday, October 8, 12

24. Who watches the graphs?
Monday, October 8, 12

25. Automate your analysis
USE COMPUTERS!
Monday, October 8, 12

26. Automate your analysis
holtWintersConfidence(Upper|Lower)
Monday, October 8, 12

27. Automate your analysis
continuous integration:
unit tests, coding standards,
static analysis, risky code paths
Monday, October 8, 12

28. Make effective security easy
by default
Make insecure
patterns “grep-able”
Monday, October 8, 12

29. Actively monitor for attacks.
Spikes in 500s and failed
logins are your first clue.
Monday, October 8, 12

30. “I discovered the vuln late Friday afternoon and wasn't
quite ready to email it to them. Saturday morning, I
confirmed the hole was still there and fixed a few bugs
with my demo.
I had my girlfriend test it from her house. It didn't work
for her. I tested again and it had stopped working for
me. Sure enough, it was now properly sanitized and
had the correct JSON MIME type.
The following Monday I received a response thanking
me for reporting it, and telling me I was right. “
Monday, October 8, 12

31. Treat independent security
researches with respect.
Monday, October 8, 12

32. “Culture eats
strategy
for breakfast”*
(*possibly apocryphal)
Monday, October 8, 12

33. Thank you!
Monday, October 8, 12