Even though software engineering has been around for decades, there is still no clear ways to assess the strengths and weaknesses of software design.
This talk introduces a framework to assess the strength of any specific software design and steps to refactor and improve it. Both object-oriented and functional programming will be discussed as ways to improve the design.
In the talk, the speaker also proposes a software architecture that incorporates all the ideas presented as the conclusion.
About speaker:
Thành currently works at Holistics Software as Co-founder and Chief Engineer architecting the next generation DataOps driven BI platform.
Before joining Holistics as co-founder, Thanh had 8 years of experience as a software engineer and big-data consultant from multiple companies, notably Revolution Analytics which was acquired by Microsoft in 2015.
Thanh graduated from National University of Singapore in 2009 majoring in Computer Engineering with a minor in Technopreneurship.
4. Happy company
class Employee; end
class Engineer < Employee
def do_work; end
end
class ProductAnalyst < Employee
def do_work; end
end
class Company
attr_accessor :members
end
def main
thong_ng = Engineer.new
khai = ProductAnalyst.new
company = Company.new
company.members = [thong_ng, khai, ...]
company.members.each(&:do_work)
end
5. How about Son?
class Engineer; def write_code; end; end
class ProductAnalyst; def design_product; end; end
company.engineers.each(&:write_code)
company.product_analysts.each(&:design_product)
# How about Son = ???
son = SometimesEngineerSometimesProductAnalyst.new
son.write_code
son.design_product
# 3 months later...
son.becomes(ProductAnalyst) # ???
6. Agenda
1. OOP is not about class and inheritance
2. Software design is about cracking complexity
3. Functional programming is about reducing complexity
4. Functional core, imperative shell
7. ● Traditional way of thinking about OOP is
incomplete
OOP is not about class and inheritance
8. Class and inheritance are language features and
implementation details, not essential parts of object-
oriented design.
10. OOP without class
module Employee
def do_work; end
end
module Engineer
def write_code; end
end
module ProductAnalyst
def design_product; end
end
son = Object.new.extend(Employee)
son.extend(Engineer)
son.extend(ProductAnalyst)
11. OOP without inheritance
class Engineer; end
class ProductAnalyst; end
thong_ng = Employee.new(roles: [Engineer.new])
son = Employee.new(roles: [Engineer.new, ProductAnalyst.new])
“Favor composition over inheritance”
12. Alan Kay’s OOP
“OOP to me means only messaging, local retention and protection and hiding of state-
process, and extreme late-binding of all things.” -- Alan Kay
● Message passing -> Interface is the only dependency between objects
● Local retention, protection and hiding of state-process -> Objects don’t know
the internal details of each other
● Extreme late-binding of all things -> Object behaviors can be changed at any
point in time
No mention of class or inheritance at all!
13. Ruby’s messages - everything is an object
1.to_s # returns "1"
1.send(:to_s) # returns "1"
1 + 1 == 1.send(:+, 1)
# There is no Boolean class in Ruby
true.class # returns TrueCLass
if (1 == 1)
'true'
end
# There is no if/else in Smalltalk - control structures are messages
(1 == 1).ifTrue do
'true'
end
(1 == 1).send(:ifTrue, &Proc.new { 'true' })
son.send(:extend, ProductAnalyst)
Class A; end
A.is_a?(Object) # returns true - classes are objects
A whole program is just about objects sending messages to each other.
14. What this means for software design
When you design a program, don't confuse class with role (interface).
● Class: language feature to construct object with certain roles. An object can
be created from a single class.
● Role: Messages an object with this role can reply to. An object can have
multiple roles. An object’s roles can change during run time.
Start with roles first: Employee role, Engineer role, ProductAnalyst role,
Persistable role, Movable role, etc.
15. The messages
Think about the behaviors the program must support, in other words, the
messages of each role:
● do_work → WorkingRole
● write_code → EngineeringRole
● design_product → ProductAnalysisRole
And then, when we want concrete behavior, we supply them via Factory, Class or
whatever the mechanism the language supports to construct objects. In pure
OOP, class is just a type of factory.
16. Recap
● OOP is not about class and inheritance
● OOP is about message sending
22. Complexity is the root of all evil
● Readability: It makes it hard to understand and
reason about a program
● Reliability: Hard to reason about the program →
hard to debug
● Reusability: Hard to turn code into reusable
component
● Scalability: Hard to separate code that can run in
parallel
● Testability: Hard to test code with too many
dependencies
23. Two types of complexity
● Essential complexity
○ Complexity inherent to the problem which can’t be removed e.g. Slack
notification
● Accidental complexity
○ Complexity due to the choices made from a particular software design to
solve the problem
24. 5 aspects of complexity
● Shared mutable state
● Side-effects
● Dependencies
● Control flow
● Code size
25. 1st aspect of complexity - mutable state
● Hard to reason about
● Implicit time
dependency
● Explosion of state
space
● Hard to parallelize
26. 2nd aspect of complexity - side effects
● Async call
● UI render
● Network call, etc.
Main problems: asynchronous
and fallible
27. 3rd aspect of complexity - Dependencies
● Class dependency: object creation
dependency
● Interface dependency: messages
dependency
● Temporal dependency: mutable
state/side-effect dependency
Golden rule of dependency management:
Things that change more should depend
on things that change less often
28. 4th aspect of complexity - Control flow
Control flow: hard to understand, reason
about, more cases to test
● Branching: if/else, case/when, etc.
● Looping: for/while loop, recursion, etc.
● Error handling: try/catch
35. OOP isolate control flow
Objects’ internal control flow are isolated from each other. External control flow is
just about objects sending messages to each other.
Branching can be converted to polymorphism and isolated to object creation
phase.
37. The main problem with OOP is that it does not reduce global complexity, it just packages
complexity into isolated, smaller boxes.
● Increase global complexity by adding indirection to behavior
○ Raw data is inherently less complex than objects with behavior
● Increase dependencies: Adding dependencies between objects
● Increase control flow: As the control flow now are the messages between
objects
● Increase code size
OOP may even increase complexity!
38. ● Writing software is about fighting complexity
● 2 types of complexity
● 5 aspects of complexity
● How OOP isolates complexity but does not reduce it
Recap
39. Part II
Software Design and Refactoring
Seeking complexity reduction with
functional programming
40. ● Reduce shared mutable state
● Reduce side effects
● Reduce dependencies
● Reduce control flow
Functional programming helps reduce complexity
41. What is functional programming?
Programming with pure, composable functions
● Pure: Data flows from inputs to outputs
○ No mutations
○ No side-effects
● Composable: Functions are first-class, which means they can be treated as
data and transformed like data.
42. How to make tiramisu
1. Begin by assembling four large egg yolks, 1/2 cup sweet marsala wine, 16 ounces mascarpone
cheese, 12 ounces espresso, 2 tablespoons cocoa powder, 1 cup heavy cream, 1/2 cup granulated
sugar, and enough lady fingers to layer a 12x8 inch pan twice (40).
2. Stir two tablespoons of granulated sugar into the espresso and put it in the refrigerator to chill.
3. Whisk the egg yolks
4. Pour in the sugar and wine and whisked briefly until it was well blended.
5. Pour some water into a saucepan and set it over high heat until it began to boil.
6. Lowering the heat to medium, place the heatproof bowl over the water and stirred as the mixture began
to thicken and smooth out.
7. Whip the heavy cream until soft peaks.
8. Beat the mascarpone cheese until smooth and creamy.
9. Poured the mixture onto the cheese and beat
10. Fold in the whipped cream
11. Assemble the tiramisu.
○ Give the each ladyfinger cookie a one second soak on each side and then arrange it on the
pan
○ After the first layer of ladyfingers are done, use a spatula to spread half the cream mixture
over it.
○ Cover the cream layer with another layer of soaked ladyfingers.
○ The rest of the cream is spread onto the top and cocoa powder sifted over the surface to
cover the tiramisu.
12. The tiramisu was now complete and would require a four hour chill in the refrigerator.
43. Imperative: mutates data
def make_tiramisu(eggs, sugar1, wine, cheese, cream, fingers, espresso, sugar2, cocoa)
mixture = whisk(eggs)
beat(mixture, sugar1, wine)
whisk(mixture) # over steam
whip(cream)
beat(cheese)
beat(mixture, cheese)
fold(mixture, cream)
dissolve(sugar2, espresso)
soak(fingers, espresso, seconds: 2)
assemble(mixture, fingers)
sift(mixture, cocoa)
refrigerate(mixture)
mixture # it's now a tiramisu
end
45. Problems
● What are the states of the input arguments after
running this function?
● If I have two people to make this tiramisu, which
parts can be done in parallel?
● The steps are too complex, how do I refactor the
steps to sub-functions?
● If we miss a step, how do we debug the wrong
mixture state?
46. What if we can make tiramisu without mutating data?
47. The core of Functional Programming is thinking
about data-flow rather than control-flow
53. Benefits (cont.)
● Each step can be tested in isolation without mocking
● Easier to debug, just put a breakpoint between 2 steps and check the input
and output of each function
● The pipeline can be parallelized and refactored easily. Any group of steps will
have a clear set of inputs and outputs.
54. ● Reduce shared mutable state: pure functions don’t mutate state
● Reduce side effects: pure functions don’t create side effects
● Reduce dependencies: zero time dependency, explicit dependencies
between steps
● Reduce control flow: no loop, no if/else
Functional programming helps reduce complexity
60. Refactoring is about reducing or isolating complexity
-- Thanh Dinh
Object oriented programming makes code understandable
by encapsulating (isolating) moving parts (complexity).
Functional programming makes code understandable by
minimizing (reducing) moving parts (complexity).
-- Michael Feathers
63. All code can be classified into two distinct roles: code that
does work (algorithms) and code that coordinates work
(coordinators).
-- John Sonmez
64. Example
We need to design a game that has
● Human has health
● Monster can attack human
● When a group of humans is near a monster, the monster will find human with
least health to attack
● When a human is attacked, its health is reduced based on monster's Attack
attribute - human’s Defense attribute
The game also needs to handle physics, input and rendering, etc.
65. Traditional OO design
class Monster
attr_accessor id, position, health, attributes
def attack(humans)
attacked_human = humans.min { |h| abs(h.position - monster.position) }
attacked_human.health -= min(attributes.atk - attacked_human.attributes.dfs, 0)
end
def render; end
def physics; end
end
class Human
attr_accessor id, position, health, attributes
def render; end
def physics; end
end
66. Problems
● Mutable states make it very hard to debug and reason about
● Tangled dependencies: Monster class now depends on Human class
○ As most real world requirements involve multiple objects, this is unavoidable
● Bloated classes: more requirements mean the classes get bigger and bigger
67. Functional programming deals with values; imperative
programming deals with objects
-- Alex Stepanov, Elements of Programming
68. Functional Core - immutable values
class PositionData < Value.new(x, y); end
class HealthData < Value.new(health); end
class AttributesData < Value.new(atk, dfs); end
class Monster < Value.new(
id: Integer,
position: PositionData,
health: HealthData,
attributes: AttributesData
)
end
class Human < Value.new(
id: Integer,
position: PositionData,
health: HealthData,
attributes: AttributesData
)
end
69. Functional core - pure functions for behaviors
class DamageService
def self.attack(monsters, humans)
monsters.reduce([]) do |attacked_humans, monster|
attacked_human = humans.min { |h| abs(h.position - monster.position) }
attacked_humans << attacked_human
.merge_new(health: attacked_human.health - min(monster.atk - attacked_human.dfs, 0))
attacked_humans
end
return humans.merge(attacked_humans)
end
end
70. OOP shell - objects
require DamageService, InputHandler, Renderer, PhysicsProcessor # etc.
class Environment # Storing states and communicate with other objects
attr_accessor humans: HumanCollection,
monsters: MonsterCollection
def process_damage
humans = DamageService.attack(monsters, humans)
end
def process_input; end # Coordinate with InputHandler
def process_physics; end # Coordinate with PhysicsProcessor
def render; end # Coordinate with Renderer
def run
loop do
process_input
process_damage
process_physics
render
end
end
end
71. Refactoring: reducing/isolating complexity
Mutations, side effects, dependencies, control flow can either be minimized or
isolated. We are doing both with this architecture.
● Reduce mutations by changing the core to functional. The only parts that still
have mutations are in the OO shell → Mutations are isolated
● Reduce unnecessary side effects by changing the core to functional. The
only parts that still have side effects are in the OO shell → Side effects are
isolated
● Reduce dependencies by moving to functional. Isolate dependencies to OO
shell.
● Number of control flows: Isolated inside the functional core
72. Benefits
● Testability: Trivial and very fast to unit test the core, minimal number of
integration tests needed for the coordinators
● Reliability: Validation can be easily added to value classes, ensure the
system is always in a valid state. Side effects and exceptions are isolated
within coordinator classes.
● Readability: Functional core has no dependency and self-contained → very
easy to read and understand in isolation
73. Benefits (cont.)
● Extensibility: Very easy to add new or update existing behavior without
worrying about changing the behavior since the core has no side effects and
no dependency.
● Scalability: Functional core can be put on parallel threads easily as there is
no shared data. Coordinator objects can also be moved to
distributed/microservice model easily as messaging is done through
serializable values.
74. ● OOP is not about class and inheritance
● Software design is complex
● 2 ways to manage complexity: reduce and isolate
● OOP helps isolate complexity
● FP helps reduce complexity
● Best of both world: Functional Core Imperative Shell architecture
Takeaways
75. References
● The forgotten history of OOP (https://medium.com/javascript-scene/the-forgotten-history-of-oop-
88d71b9b2d9f)
● OO programming, a personal disaster (https://medium.com/@brianwill/object-
oriented-programming-a-personal-disaster-1b044c2383ab)
● What functional programming is all about
(http://www.lihaoyi.com/post/WhatsFunctionalProgrammingAllAbout.html)
● Boundaries (https://www.destroyallsoftware.com/talks/boundaries)
● Oh composable worlds (https://www.youtube.com/watch?v=SfWR3dKnFIo)
At first, it was nice. It was really fast to build any new feature. It is easy to debug and modify. However, 4 years passed.
Holistics’ code base is getting more and more complex, it is hard to change anything without breaking something. The engineers are afraid to change the code that he doesn’t understand. New features are getting slower and slower to implement.
Does this story sound familiar to you?
Do you think that OOP design is easy? Let’s start with an OOP puzzle...
Do you think that OOP design is easy?
Traditional way of thinking about OOP makes
How many of you agree with this? Disagree?
Cut into 2 parts
In Ruby, almost everything is an object. An integer is an object. A boolean is an object.
This is inspired from Smalltalk, a language created by Alan Kay. In Smalltak, object does not execute “method” but send messages to each other.
Even if/else is about sending message to boolean object.
Even a class is an object.
OOP is just one way to think about software design. Let’s actually go deeper into software design and understand it.
It is already hard with the existing solution. Each new feature adds exponentially more complexity as the new complexity multiplies existing complexity.
This is why OOP makes sense. Depending on interface that generally changes less often than implementation.
In Holistics, a better design is when one reduces some aspects of complexity.
Also, the order is from more important to less important.
There is another way that can actually reduce complexity. That’s functional programming.
You can see that in each step, the data is mutated.
At each step, the code implicitly produces a new program state.
At the end of the function, almost all inputs are no longer the same.
It is hard to know in terms of the orders between each step, which one depends on each other. In other words, is it ok to switch the order of “whip(cream)” and “beat(cheese)”?
This is slightly better as this isolate the mutations inside “mixture” object.
All the functions in here are pure functions, which don’t mutate the input arguments. The structure of the code makes it obvious which is needed before which.
We can break this code into multiple parts, and each part will have clear inputs and outputs. Pure functions easily compose.
OOP is just one way to think about software design. Let’s actually go deeper into software design and understand it.
The architecture was introduced in 2012 by Gary Bernhard in his “Boundaries” talk.
The code looks simple at first glance, but there are many problems with it.