Presentation on how Meetup tackles web performance. Given on: - Nov 17th, 2009 for the NY Web Performance Group (http://www.meetup.com/Web-Performance-NY/) - Jan 26th, 2010 for NYC Tech Talks Meetup Group (http://www.meetup.com/NYC-Tech-Talks/)

1. Meetup Performance
Greg Whalin, CTO Meetup (@gwhalin), Justin Cataldo, Lead UI Engineer (@jcataldo), Will Howard, Lead UI Engineer

2. Meetup
Platform for local groups
Mission is MEME (Meetup Everywhere About Most Everything)
~6.2m members
~70k Groups
~500k group joins every month
~5 million Meetups have happened
~53 million RSVPs

3. General Architecture and Back-
end Performance
(just a tiny bit - this could and
should be another presentation)

4. Data
MySQL (any RDBMs store) biggest pain
replication
use InnoDB!
smart indexing (take advantage of clustered indexes)
server side conn pools and short timeouts on connections
smart about fs choice on Linux (we use XFS after benchmarking)
Hardware - relatively expensive boxes
multi-core (8 or 16) Opteron
lots of ram (32/64GB)
fast drives (lots of spindles, RAID10)
Cache, cache, cache!
innodb buffer cache
memcache
local app server memory
Shrink data when possible
archive unused data
custom serialization when serializing
data partitioning/sharding

5. Storage
Over half a million photos uploaded to Meetup every month
Scaled and processed into 4 different sizes (plus original)

6. Storage solutions
Options for growth include NAS, SAN, or something else
NAS and SAN are single point of failure and possibly $$$
Only postpones problem

7. MogileFS
developed by Brand Fitzpatrick (i.e. Memcached)
OSS distributed filesystem (built in Perl)
any hard drive on network can easily be added to cluster
scales easily and cheaply

8. Much much much more going on
but...

9. UI Performance
(much of our focus here)

10. Why does performance matter?

11. Why does performance matter?
Slow site

12. Why does performance matter?
Slow site
Bad User Experience

13. Why does performance matter?
Slow site
Bad User Experience
Drop in Member
Activity

14. Why focus on front end performance?
Back end only accounts for 10-15% of the response time
Less time and resources
Costs less
http://developer.yahoo.net/blog/archives/2007/03/high_performanc.html

15. Case Study: Event Details

16. Event Details:
Load Time
Load time = 6.321s
www.webpagetest.org

17. Event Details: Requests
Lots of javascript being loaded

18. How do we improve
performance?

19. 3 Steps to improving performance
1. Externalize script
2. Move scripts to the bottom of the page
3. Reduce requests

20. 3 Steps to improving performance
1. Externalize script
2. Move scripts to the bottom of the page
3. Reduce requests

21. Why externalize scripts
Prevents blocking
Inline scripts prevent asynchronous downloads
Downloads must wait for the script to be executed
Caching
Inline JavaScript is downloaded every time
External scripts are cached by the browser
Reduced overall page size
Reusable
Can use the same code somewhere else on the site
easily

22. Pull out inline script
<script type="text/javascript">
>
if(typeof Meetup.EventDetails == 'undefined')
Meetup.EventDetails = {};
(function(){ EventDetails.js
var self = Meetup.EventDetails;
...
})();
</script>

23. 3 Steps to improving performance
1. Externalize script
2. Move scripts to the bottom of the page
3. Reduce requests

24. Web:script
Custom tag built in house
Moves inline and external script to the bottom of the page
Allows UI engineers to not have to worry about where they
place scripts
Compresses inline script using YUICompressor
/***** Load External Script *****/
<web:script src="/script/Meetup/packed/EventDetails.js" />
/***** Load Inline Script *****/
<web:script>
Meetup.Copy.noMembersMarkedAttended = "<trn:message key="event.attendance.noMembersMarkedAttended">No
members have been marked attended</trn:message>";
Meetup.Copy.noMembersMarkedAttendedDynam = '<trn:message key="event.attendance.
noMembersMarkedAttendedDynam"><trn:param name="GROUPING">__GROUPING__</trn:param>No members in
"{GROUPING}" have been marked attended</trn:message>';
Meetup.Copy.noMembersMarkedAbsent = "<trn:message key="event.attendance.noMembersMarkedAbsent">No
members have been marked absent</trn:message>";
</web:script>

25. 3 Steps to improving performance
1. Externalize script
2. Move scripts to the bottom of the page
3. Reduce requests

26. Reduce Requests
Concatenate as much as possible
Load only what we need upfront

27. Concatenation using Sprockets
Sprockets (www.getsprockets.com)
Created by 37Signals
Ruby library that preprocesses and concatenates JavaScript
files
Baked into our build process

28. Concatenation using Sprockets (cont.)
EventDetails.js
CommentDeleteConfirm.js
BubbleTips.js
PlacesManager.js
MaxCharactersEnforcer.js > EventDetails.js

29. Using Sprockets
/******* Sprockets Directives *******/
//= require <CommentDeleteConfirm>
//= require <DomDeco/BubbleTips>
//= require <DomDeco/PlacesManager>
//= require <DomDeco/MaxCharactersEnforcer>
/******* Begin Event Details Code *******/
if(typeof(Meetup.EventDetails) == 'undefined')
Meetup.EventDetails = {};
(function(){
var self = Meetup.EventDetails;
...........
})();
/******* Build Process *******/
<exec executable="sprocketize" failonerror="true" output="${image.dir}
/script/Meetup/packed/EventDetails.js">
<arg value="-I"/>
<arg path="${image.dir}/script/Meetup/"/>
<arg path="${image.dir}/script/Meetup/EventDetails.js"/>
</exec>

30. Lazy Loading
Defer loading of javascript files until they are needed
Reduces the initial upfront requests
Helps reduce blocking by downloading files asynchronously
Precaching

31. Lazy Loading: How it works
Inserts scripts into the head dynamically
var scriptNode = function(src) {
return createDOM("script", {
"type": "text/javascript",
"src": src
});
}
var load = function(id, src, n) {
var script = scriptNode(url);
head.appendChild(script);
script.onload = script.onreadystatechange = function() {
if (!this.readyState || this.readyState == "loaded" || this.readyState == "complete") {
script.onload = script.onreadystatechange = null;
}
}
}
Meetup.Script.include("http://static2.meetupstatic.com/script/Meetup/DomDeco/LinkDecorator.js",callback);

32. Did it make a difference?

33. Reduced requests
Javascript requests cut by 50%

34. Event Details
Page Load After
Old Load Time = 6.321s
New Load time = 4.643s

35. Event Details
Page Load After
Old Load Time = 6.321s
New Load time = 4.643s
Load time
decreased by
27%!

36. But that's not all

37. Execute early
Execute when the dom is ready

38. Execute early: DOMReady
Libraries that have a DOM ready solution:
jQuery
YUI
Prototype
Pretty much every modern JS library (not MochiKit)
Meetup uses MochiKit, so we rolled our own.

39. Execute early: DOMReady
And by rolled our own, I mean we're using the Dean
Edwards/Matthias Miller/John Resig implementation.
http://dean.edwards.name/weblog/2006/06/again/#comment5338
With a few changes.
Meetup.DOMReady.ready(function(){
Meetup.EventDetails.init();
if(Meetup.EventDetails.isCanceled != 4 && Meetup.EventDetails.rsvp != 0){
deletePopup = new Meetup.CommentDeleteConfirm();
deletePopup.pagerOffsetFieldName = "p_commentsList";
deletePopup._decorate();
}
});

40. Execute early: Even earlier
Do you need to wait for the DOM to be ready?
If you aren't manipulating the DOM, there's no reason to
wait until it's ready.

41. Automated Image Optimization
Using smush.it
http://developer.yahoo.com/yslow/smushit/
Smusher Ruby gem
http://github.com/grosser/smusher (gem install smusher)
BASH script that watches our image directories for
changes and executes smusher.
./filescan.sh /usr/local/meetup/static/img/ 'smusher -q @' 'jpg,png'&

42. Event Delegation
Run less JavaScript up front

43. Event Delegation
But first, a little bit about event bubbling...
From: http://www.quirksmode.org/js/events_order.html

44. Event Delegation
Pros Cons
Does not work well with nested elements
Much faster on load (not
Doesn't work with all events
connecting DOM elements)
Slight performance hit with execution
No need to disconnect /
reconnect with AJAX calls
Fewer memory leaks
A lot of JS libraries already have plug-ins for event
delegation (jQuery, YUI, prototype).
But, it's pretty easy to write your own (we did).

45. Event Delegation: Meetup.Dispatcher
<div id="C_page">
...
<span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href="
http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span>
...
</div>
var mdp = Meetup.Dispatcher.init("C_page", "onmouseover");
mdp.registerFunc("topic-info-hover", Meetup.UI.InfoHover.mouseOver);
Meetup.UI.infoHover.mouseOver = function(e) {
topicId = _getTopicId(e.target());
if (!topicId || topicId == "") return;
_primeCache(topicId);
var activeEl = _getActiveEl(e.target());
var pos = getElementPosition(activeEl);
...
}

46. Event Delegation: Meetup.Dispatcher

47. Event Delegation: Meetup.Dispatcher
<div id="C_page">
...
<span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href="
http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span>
...
</div>
// Inits a new instance of dispatcher
// Connects a mouseover event to the parent container "C_page"
var mdp = Meetup.Dispatcher.init("C_page", "onmouseover");
// Calls Meetup.UI.infoHover.mouseOver() when target element has "topic-info-hover" class.
mdp.registerFunc("topic-info-hover", Meetup.UI.InfoHover.mouseOver);
Meetup.UI.infoHover.mouseOver = function(e) {
topicId = _getTopicId(e.target());
if (!topicId || topicId == "") return;
_primeCache(topicId);
var activeEl = _getActiveEl(e.target());
var pos = getElementPosition(activeEl);
...
}

48. Speeding up DOM crawling with Sizzle
sizzlejs.com
Internet Explorer 7
MochiKit: 6623.94ms
Sizzle: 306.03ms
Firefox 3.5
MochiKit: 210.524ms
Sizzle: 111.553ms

49. Where do we go from here?
More concatenation and lazy loading where it makes sense
Defer image loading where it makes sense
Reduce DOM elements
Reduce CSS and improved selector efficiency
and more

50. Deployment and Serving
As it pertains to css/html/js

51. Launches
Launch multiple times a day (sometimes)
Need launches to be quick / no downtime
Optimize static resources only at deploy time and only if
modified

52. Deployment of static content
Sprockets (reduce requests)
YUICompressor for js (local mod to speed up optimizing
multiple files)
Pre-compress css and jsp
Set cache-control to be fresh for over a year (indefinite)
All links on site generate programatically and versioned

53. Link generation
Custom JSTL page function
http vs https
domain sharding and cookie free domain
content versioning
<link rel="stylesheet" href="${mfn:staticUrl( "/style/meetup.css", state.context.isSecure )}" type="
text/css" />
<img src="${mfn:imgUrl( '/img/noPhoto_50.gif', state.context.isSecure )}" alt=""
class="noPhoto" />
<link rel="stylesheet" href="http://static1.meetupstatic.
com/050991196173395491322880/style/meetup.css" type="text/css" />
<img src="http://img1.meetupstatic.com/39194172310009655/img/noPhoto_50.gif" alt="" class="
noPhoto"/>

54. Versioning static content
MD5 checksum of contents of file
Run w/ each launch
Store versions in db tied to release
mysql> select * from resource_version where filename = '/style/base.css';
| 394328 | /style/base.css | 39020083689267241 | 567 |
| 398052 | /style/base.css | 8487620432388779772669 | 568 |
| 401776 | /style/base.css | 357470606563045379 | 569 |
| 405506 | /style/base.css | 3068234199748867 | 571 |
| 409240 | /style/base.css | 024745310801291061590 | 572 |
| 412974 | /style/base.css | 024745310801291061590 | 573 |
| 416708 | /style/base.css | 09972542737049101325 | 574 |

55. Static content serving
Served off CDN (reverse proxy)
Anycast DNS for hostname resolution
Origin servers running lighttpd
Strip versioning from url using rewrite rules
Cache set to 12 months out
Compress
expire.url = (
"/" => "access plus 12 months"
)
compress.cache-dir = "/var/cache/lighttpd/"
compress.filetype = ("text/plain", "text/html", "text/css", "application/x-javascript", "text/javascript")
url.rewrite-once = (
"^/script/(.*/)?[0-9]+/(.+).js$" => "/script/$1$2.js",
"^/style/(.*/)?[0-9]+/(.+).css$" => "/style/$1$2.css",
"^/img/(.*/)?[0-9]+/(.+).(gif|png|jpg)$" => "/img/$1$2.$3",
"^/d+/script/(.+).js" => "/script/$1.js",
"^/d+/style/(.+).css" => "/style/$1.css",
"^/d+/img/(.+).(gif|jpg|png)" => "/img/$1.$2",
"^/photos/(([^/]+)/.+/(.+).jpeg)(?:?.*)?$" => "/cgi-bin/photos.fcgi?type=$2&key=$3",
"^/photos/([^/]+)/([^/]+).jpeg(?:?.*)?$" => "/cgi-bin/photos.fcgi?type=$1&key=$2",
"^/file.*" => "/"
)

56. Questions?
Also, we need help! Hiring for:
Linux Systems Administrator
Software Engineers
UI Engineers
QA Engineers
Community Specialist
PR Renegade
Sponsorship Sales
Account Coordinator
http://www.meetup.com/jobs/