3. Containers are born: March 18, 1982
• Containers are not a new idea, having originated via filesystem
containers with chroot in Seventh Edition Unix
• chroot originated with Bill Joy, but specifics are blurry; according
to Kirk McKusick, via Poul-Henning Kamp and Robert Watson:
4. Containers ca. 2000
• Seeking to provide a security mechanism, FreeBSD extended
chroot into jails:
5. • To provide workload consolidation, Sun introduced complete
operating system virtualization with zones (née Project Kevlar)
Containers ca. 2002
8. Aside: Hardware-level virtualization
• Since the 1960s, the preferred approach for operating legacy
stacks unmodified has been to virtualize the hardware
• A virtual machine is presented upon which each tenant runs an
operating system that they choose (but must also manage)
• Effective for running legacy stacks, but with a clear inefficiency:
there are as many operating systems on a machine as tenants:
• Operating systems are heavy and don’t play well with others with
respect to resources like DRAM, CPU, I/O devices, etc.!
• Still, hardware-level virtualization became de facto in the cloud
10. Containers ca. 2011, cont.
• Via SmartOS zones, Joyent has run containers in multi-tenant
production since ~2006...
• Adding support for hardware-based virtualization circa 2011
strengthened our resolve with respect to OS-based virtualization!
• We saw that the operational characteristics of OS containers —
performance, elasticity, tenancy — opened up new possibilities...
12. Containers ca. 2012/2013, cont.
• Over 2012 and early 2013, we built Manta, a ZFS- and container-
based internet-facing object store offering in situ compute
• Containers allow the compute to be moved to the data instead of
having to backhaul data to transient compute
• And as a reminder, the OS — Unix! — was built around the notion
of ad hoc computation
13. Aside: Unix
• When Unix appeared in the early 1970s, it was not merely a new
system, but a new way of thinking about systems
• The Unix philosophy, as articulated by Doug McIlroy:
• Write programs that do one thing and do it well
• Write programs to work together
• Write programs that handle text streams, because that is a
universal interface
• Four decades later, this philosophy remains as relevant as ever!
14. Aside: Doug McIlroy v. Don Knuth
• In 1986, Jon Bentley posed the challenge that became the Epic
Rap Battle of computer science history:
Read a file of text, determine the n most frequently used words,
and print out a sorted list of those words along with their
frequencies.
• Don Knuth’s solution: a purpose-built algorithm in WEB, a Pascal-
like literate programming system of his own invention
• Doug McIlroy’s solution shows the power of the Unix philosophy:
tr -cs A-Za-z 'n' | tr A-Z a-z |
sort | uniq -c | sort -rn | sed ${1}q
15. Containers ca. 2012/2013, cont.
• Manta allows for an arbitrarily scalable variant of McIlroy’s
solution to Bentley’s challenge:
mfind -t o /bcantrill/public/v7/usr/man |
mjob create -o -m "tr -cs A-Za-z 'n' |
tr A-Z a-z | sort | uniq -c" -r
"awk '{ x[$2] += $1 }
END { for (w in x) { print x[w] " " w } }' |
sort -rn | sed ${1}q"
• As with McIlroy’s solution, the container allows for not just for
terser expression but faster execution!
16. Containers ca. 2012/2013
• Manta served to strengthen our belief in the power of containers
as not merely better infrastructure, but as transformative
• Our biggest challenge with Manta was that containers were not
well understood: it felt “easy” to us, but was confusing to many
• Would the world ever really figure out containers?!
18. Containers ca. 2013/2014: Revolution at last!
• Docker used the rapid provisioning + shared underlying
filesystem of containers to allow developers to think operationally
• Operational dependencies are encoded via an image
• Images can be reliably and reproducibly deployed as a container
• Images can be quickly deployed — and re-deployed
• Coupled with the mainstream adoption of open source and the
leading adoption of microservices...
• Containers accelerate software development
20. Containers ca. 2014/2015, cont.
• Triton combines SmartOS and our cloud management software
with a Docker Remote API endpoint
• With Triton, the notion of a Docker host is virtualized: to the
Docker client, the datacenter is a large Docker host
• One never allocates VMs with Triton; one only allocates
containers — and all Triton containers run directly on-the-metal
• All of Triton is open source — you can download it and run it for
yourself (or run it on our public cloud)
26. 2016 and beyond: Frameworks v. Libraries
• The crowded container ecosystem can be viewed broadly as two
different approaches:
• A framework approach in which the framework is in control, but
alternatives are “pluggable”
• A library approach in which the developer/operator is in
control, composing functionality out of well-defined modules
• The “all in” nature of frameworks may make initial adoption easier,
but they ultimately sacrifice flexibility!
27. 2016 and beyond: Frameworks v. Libraries
• For example, a framework approach to orchestration may not
accommodate applications that demand local persistence
• We opted for orchestration that lives not in an all-knowing
framework, but rather with the application, in the container
• In-container logic allows for service registration/discovery, health
checks, etc. — but does not handle restart
• This is application-centric orchestration, and we have dubbed this
the autopilot pattern: https://autopilotpattern.io
28. 2016 and beyond: Containers on autopilot
• We have developed ContainerPilot, an embodiment of the
autopilot pattern for Docker containers
• All open source (https://github.com/joyent/containerpilot) — and
not Triton or Joyent specific; can be used anywhere
32. 2016 and beyond: Failure matters
• When deploying containers + microservices, there is an unstated
truth: you are developing a distributed system
• While more resilient to certain classes of force majeure failure,
distributed systems remain vulnerable to software defects
• We must be able to debug such systems; hope is not a strategy!
• Distributed systems are hard to debug — and are more likely to
exhibit behavior non-reproducible in development
• With container-based systems, we must think in terms of the
system, not merely the program!
33. 2016 and beyond: The Jevons paradox
• The Jevons paradox seems very likely to hold for containers:
greater efficiency will result in a net increase in consumption!
• Efficiency gains from containers are in terms of developer time...
• ...but requiring containers to be scheduled in VMs induces
operational inefficiencies: every operator must now think like a
cloud operator — maximizing density within fixed-cost VMs
• Greater consumption + operational inefficiencies threaten to slow
the container revolution — or make it explosive in terms of cost
• To realize the full economic promise of the container revolution,
we need container-native infrastructure!
34. 2016 and beyond: Public and on-prem
• Death of on-prem computing is greatly exaggerated!
• There are three key determinants for public v. on-premises:
• Economics: Rent vs. buy; OPEX vs. CAPEX
• Risk Management: Security/compliance — and also risk
factors associated with operator-as-threat
• Latency: The speed of light is a constant!
• Economics dominates: “private cloud” efforts that do not deliver
public cloud economics are doomed to (continue to) fail!
35. 2016 and beyond: #serverless
• On the one hand, this term is clearly meaningless…
• ...but on the other, it highlights that containers have liberated us to
think in terms of higher-level functionality
• Thinking in functions is great (viz. Manta), but confining functions
to containers to single-tenant VMs undermines their economics
• The virtual machine is a vestigial abstraction; we must reject
container-based infrastructure that implicitly assumes it!
• So don’t say “#serverless” when you mean...