Introductionary talk and tutorial on Groovy, discussion of its success and popularity, issues with the future of Java with examples

1. Groovy On The Trading Desk
Jonathan Felch
Quantitative Strategies
Volant Trading LLC

2. Overview
● Motivations for Polyglot Software with Groovy
● Introducing Groovy: Scripting on the VM
● Skills To Take With You: Learning Groovy
● Groovy DSLs: Creating the syntax you need
● Unlearning Java, C++, C#
● Groovy Performance and Concurrency
● Relearning Java: The Future of the VM
● Questions

3. Motivations
● High Speed OO Tools + Dynamic / Functional Tools
● Re-use existing assets
● Lazy Evaluation
● Externalize Logic and Business Rules
● Dynamic Evaluation

4. Why Groovy ?
● Meet Requirements
– Lazy & Dynamic Evaluation,
– Concise Synatx, Rapid Development, Iterative Prototyping
– Supported But Did Not Require Functional Style, Support for
Closures / Lamda Expressions
– Tool Support, Attractive Adoption Pattern
– Human Readable / Machine Executable Data Interchange
– Natural Data Integration Platform
● Strong and Focused Community
● Familiar Syntax / Ease of Adoption
● Expensible Syntax : DSLs

5. Scripting on the VM
Evolution of the VM: Towards Dynamic Programming
– Features Popular in Most Used 'Frameworks' Exploited
Specific Benefits and Uses of Dynamic Programming for a
general-purpose or 'cross-cutting' concern
● Hibernate, Spring Framework, Jboss AOP, AspectJ
– Introduction of Dynamic Programming for Testing,
Configuration and Business Rules
● BeanShell, Jython, JRuby
– Next Generation Scripting: Groovy, Scala, Clojure
● Specific to VM, not a straight port of another language
● Emphasis on Functional Programming
● Tight Integration of FP / DP with a high performance OO Language

6. Introducing Groovy
● The Blue Pill: Syntactically compatible with Java
– Easy Adoption for Java Programmers
– Virtually seamless / virtually flawless integration with Java
– Natural Support for Multiple Styles
– Compatibility allows for remarkable performance tuning
● The Red Pill: A Language On Its Own
– Type-system: Dynamically / Optionally Strong Typing
– Closures: Lambda Expressions / Functions as Objects
– GDK: Object and Collection Enhancements
– Dynam Eval: Code is Data, Data is Code
– Dynam Syntax: Write your own syntax
– Enhancements: New operators, data literals, new basic types

7. Job Trends / Adoption Hints
for new JVM Languages

8. Groovy Use Cases:
● 'Super Glue': I/O and Integration
● 'Code as Data': Configuration, Process Scripting,
Business Rules
● 'Live Analysis': Ad-Hoc Queries & Expressions
● 'Language Building': New Syntax and DSLs
● 'Half-Baked Ideas'
● 'House Elf': Cleanup and Maintenance

9. Learning Groovy
Im prov in g o n O b je c t
def abstract class OptionPxModel {
def vol, carry, strike, expiry, pc
def abstract px (def spot)
}
OptionPxModel model = new BlackScholesModel(strike : 100, pc : 'C' )
model.vol = 0.15
model.expiry = Date.parse('MM/dd/yyyy', '11/18/2010')
println “The Strike 100 Nov 2010 Call is worth ${model.px(102.0)}”
C lo s u re s :
def r = new Random()
def normRand = { r.nextGaussian() }
def stockPrice = { px, vol, rate, time →
px * exp((rate – vol**2) * time + normRand() * vol * time)
}

10. Learning Groovy: Collections
L is ts
def list = [5, 6, 7, 8]
assert list.size == list.size()
assert list[2] == list.getAt(2) == list.get(2) == 7
assert [1,2,3,4,5][-1] == 5 && [1,2,3,4,5][-2] == 4
assert list.find { it > 6 } == 7 && list.findAll { it > 6 } == [7, 8]
list.each { println “$it ${it * 2}” }
['a', 'b', 'c'].eachWithIndex{ it, i -> println "$i: $it" }
Mu ltiple A s s ig n m e n t
def (int a, int b) = [1, 2]
(a, b) = [b, a]
Ra ng e s
assert ('a'..'d') == ['a','b','c','d']
assert [ 1, *3..5, 7, *9..<12 ] == [1,3,4,5,7,9,10,11]
def oneWeekForward = (new Date() .. (new Date() + 7))
Ma ps
[ 'id': 'FX-17', name: 'Turnip',
99: 123, (-97): 987,
"tail's": true
].each { key, value → println “$key => $value” }

11. Groovy One (or 2, 3, 4) Liners Apps
Executing A Process:
”processMyFile ./output.txt $Destination”.execute()
Adding A She-Bang To All Groovy Files:
groovy -i .bak -pe "if (count == 1) println '#!/usr/bin/groovy'" *.groovy
Print the 1st column and 2nd-to-last column from file
groovy -a -pe "split[0] split[-2]" some.filename // or set of files
Create and Shuffle a Deck of Cards
def deck = [ [ *2..10, *('J Q K A'.split()) ], 'C D H S'.split() ]
.combinations().collect { it.join('') };
Collections.shuffle(deck)
Generate 21-Character OSI Symbol for Option
def osi = { symbol, year, month, day, cp, strike ->
sprintf('%1$s%2$2d%3$2d%4$2d%5$s%6$08.3f', [
symbol.padLeft(6), year as Integer, month as Integer, day as Integer,
cp as String, strike as Double ] as Object[] )
}

12. Learning Groovy: Operators
'A s O pe ra to r'
"3.14159" as Double
def readable = { it.put("12 34".reverse()); 5 } as Readable
A s O pe ra to r O n C lo s u re s + Ma ps → O b je c ts ?
def map; map = [ i: 10,
hasNext: { map.i > 0 },
next: { map.i-- } ] as Iterator
while ( map.hasNext() ) println map.next()
E lv is , S a fe Re fe re n c e , S pre a d & S pa c e s h ip
def lang = data?.favoriteLanguage ?: 'Groovy'
assert ['cat', 'elephant']*.size() == [3, 8]
assert (3 <=> 4) == -1 && (3 <=> 3) == 0 && (5 <=> 3) == 1
Re g e x
def list = ["foo", "bar", "moo"]
def pattern = ~/foo/
assert "foo" =~ list.find { it =~ pattern }
assert ["foo", "moo"] == list.findAll { it ==~ /.*oo/ }

13. One Slide Apps: RSS Reader
def url ='http://www.groovyblogs.org/feed/rss'
def items = new XmlParser().parse(url).channel.item
def cols = 'pubDate title description'.tokenize()
groovy.swing.SwingBuilder.build {
frame(id:'f', title: 'Groovy RSS', visible :true) {
scrollPane {
table {
tableModel(list: items) {
cols.each { col →
closureColumn header: col,
read: { it[col].text() }
} } } } }
f.pack()
}.show()
Credit: Dierk König, GR8 Conference 2009, Copenhagen

14. One Slide Apps: Monte Carlo
def px = 100, rate = 0.05, vol = 0.15, time = 1.0
def strikes = [ 8..12 ].collect { it * 10 }
def w = RandomNumbers.getNormDist(1000,1000)
def S = px * exp((rate-vol ** 2)* time + sqrt(time)* vol * w)
strikes.each { K →
def optionValue = max(0, S – K)
def df = exp(-rate * time)
println “${K} : ${df * optionValue as Number}”
})

15. One Slide Apps: RESTful Web Server
import com.sun.net.httpserver.*;
def isPrime = { Long N ->
if (N < 2) return false;
for (long i = 2; i*i < N; i++) if (N % i == 0) return false
return true
}
def pws = { HttpExchange exch ->
exch.sendResponseHeaders(200,0);
def pe = exch.requestURI.path.split('/')
if (pe.size() == 3 && pe[1] == 'prime') {
exch.responseBody.write("${isPrime(pe[2] as Long)}".bytes)
} else exch.responseBody.write("${exch.requestURI.path}".bytes)
exch.responseBody.close();
} as HttpHandler
def run = { handler, minutes ->
def server = HttpServer.create(new InetSocketAddress(8888),0)
server.createContext('/', handler)
server.start()
}
run(pws,1)

16. Unlearning Java: Taking the Red Pill
● Meta-Programming
● Design Patterns
● Compile Time Meta-Programming
● Power of Lazy Evaluation

17. Dynamic Language:
Evaluation or Syntax
“In Lisp, you don’t just write your program down toward
the language, you also build the language up toward your
program”
- Paul Graham
“When [Smalltalk] is used to describe an application
system, the developer extends Smalltalk, creating a
domain-specific language by adding a new vocabulary of
language elements ...”
- Adele Goldberg DSLs

18. Meta-Programming
● Understanding The Type System
– Groovy has an optionally dynamic strongly typed type system
– What does that mean ?
● Dynamic Method Invocation and GroovyObject
– invokeMethod, get/setProperty
– What is a MetaClass ?
● Meta Object Protocol
– def methodMissing(String name, args) { }
– def propertyMissing(String name)
– def propertyMissing(String name, value)
● Overloading Operators

19. GORM as a DSL
// These Class Definitions Generate SQL and Database Tables
class Book {
String title
Date releaseDate
Author author
}
class Author {
String name
}
// These Operation Generate SQL and Database Queries
def book = Book.findByTitle("The Stand")
book = Book.findByTitleLike("Harry Pot%")
book = Book.findByReleaseDateBetween( firstDate, secondDate )
book = Book.findByReleaseDateGreaterThan( someDate )
book = Book.findByTitleLikeOrReleaseDateLessThan(
"%Something%", someDate )

20. GORM as a DSL
class GORM {
def dynamicMethods = [ ]
def methodMissing(String name, args) {
def method = dynamicMethods.find { it.match(name) }
if(method) {
GORM.metaClass."$name" = { Object[] varArgs →
method.invoke(delegate, name, varArgs)
}
return method.invoke(delegate,name, args)
}
// GORM actually writes or creates a new method here
else throw new MissingMethodException(
name, delegate, args )
}
}

21. DSL: Tool Kit
● Meta-Methods
– String.metaClass.swapCase= {
delegate.collect{ c →
c in 'A'..'Z'? c.toLowerCase() :c.toUpperCase()
}.join()}
assert "Foo".swapCase()== "fOO"
Number.metaClass.multiply= { Grid g → g.multiple(delegate) }
● MOP
– propertyMissing, methodMissing, respondsTo,
hasProperty, listMethods, selectMethod
● Categories, ObjectBuilder, MetaBuilder

22. Function Eval / Visualization
new SwingBuilder().frame(title:'Quant Modeller', size : [1100,600], // ..
parameters here not shown {
tableLayout() {
tr {
td { widget(gui)
} td { widget(editor, border : titledBorder('Model Script',
preferredSize :[900,600]))
} td {
panel(border : titledBorder('Model Calibration'), preferredSize :
[200,600]) {
tableLayout(cellpadding :15) {
tr {
td { label(text : 'Params:') }
td { button(text : 'Add', * Edited Out * }
tr {
td(colspan : 2, align : 'center') {
button(text : 'Execute',
actionPerformed : {
shell.setVariable 'f', [:]
shell.eval1uate(editor.text)
def f = shell.getVariable('f')
def obj = [ f : f] as Model3d
gui.model = obj
gui.redraw()
} ) } } }

23. New AST Transforms
• @Singleton — okay, not really a pattern :-)
• @Immutable, @Lazy, @Delegate
• @Newify
• More DSL and Meta-Magic! @Category and @Mixin
• @PackageScope
• Swing’s @Bindable and @Vetoable
• Grape’s @Grab

24. The Evil Java Singleton
public class Evil {
public static final Evil instance = new Evil();
private Evil() {}
static Evil getInstance() { return instance; }
}
• In Groovy now:
–@Singleton class Evil {}
• A lazy version also:
–@Singleton(lazy = true) class Evil {}
Credit: Guillaume Laforge,
“To Infinity and Beyond” & GDK 1.7 Release notes

25. DSLs → They Start Out As APIs
// closure maddness
// Before refactoring
def phrase = "The quick brown fox jumps over the lazy dog"
def result = phrase.toLowerCase().toList().
findAll{ it in "aeiou".toList() }. // like WHERE …
groupBy{ it }. // like GROUP BY …
findAll{ it.value.size() > 1 }. // like HAVING …
sort{ it.key}.reverse(). // like ORDER BY …
collect{ "$it.key:${it.value.size()}" }.join(", ")
println result
Credit: Paul King, “Groovy DSL”

26. DSL: APIs That Get Refactored
// Refactored helper closures
def lowercaseLetters= phrase.toLowerCase()
def vowels = { it in "aeiou".toList() }
def occursMoreThanOnce= { it.value.size() > 1 }
def byReverseKey= { a, b -> b.key<=> a.key}
def self = { it }
def entriesAsPrettyString= { "$it.key:${it.value.size()}" }
def withCommaDelimiter= ", "
// Refactored main closure
println lowercaseLetters.
findAll(vowels).
groupBy(self).
findAll(occursMoreThanOnce).
sort(byReverseKey).
collect(entriesAsPrettyString).
join(withCommaDelimiter)
Credit: Paul King, “Groovy DSL”

27. Putting All Together I
● Drop Parenthesis
● Named Parameters
● Objects Graphs → Tree of 'Nouns' and 'Verbs'
● Human Readable / Machine Executable
● Specific Syntax

28. Putting All Together II
order to buy 200.shares of GOOG {
limitPrice 500
allOrNone false
at the value of { qty * unitPrice - 100 }
}
take 2.pills of chloroquinine after 6.hours
def "length of Spock's & his friends' names"() {
expect: name.size() == length
where:
name | length
"Spock" | 5
"Kirk" | 4
"Scotty" | 6
}
Credit: Paul King, “Groovy DSL”

29. Config Slurper
Log4j {
appender.stdout = "org.apache.log4j.ConsoleAppender"
appender."stdout.layout"="org.apache.log4j.PatternLayout"
rootLogger="error,stdout"
logger {
org.springframework="info,stdout"
}
additivity {
org.springframework=false
}
}
databaseURL : //
development {
databaseURL : //
}
testing {
databaseURL : //
}

30. Performance
● Dynamic Typing is Expensive (But Worth It)
– Hot Spot Opportunity
● Dynamic Method Invocation is expensive
(But Getting Better)
– Da Vinci Project and invokeDynamic bytecode
– Hot Spot Opportunity
● BigDecimal & Collections → Insanely Expensive
For Serious Math
– People need to better learn to use IEEE floating point types
–

31. Performance II
● Getting Too Groovy ???
– Abusing .each { tuple →} // for loops versus anon closures
● Simplicity --> faster
– Groovy reduces your dependency on Java Enterprise
– Gpath vs Xpath (Gpath almost always wins)
– Use Visual VM: Groovy Memory Use Can Be Mysterious
● Too Many Collections ?
– Is The Idea to 'Cache' The World As A Map
● Use Groovy-Aware Java for Faster Code
– People Need to learn IEEE Floating Point Types

32. Actors
Scaling the VM: Immutable Data Message + Services
class Player extends AbstractPooledActor {
String name
def random = new Random()
void act() {
loop {
react {
// player replies with a random move
reply Move.values()[
random.nextInt(Move.values().length)
]
}
}
}
}

33. Are Java & C# Becoming Groovy ?
● Google Guava (formerly Collections)
● Lots of new Libraries for FP, Enhanced
Collections, Tuples, Monads, etc...
● Doug Lea's Fork-Join + Predicate Libraries
● Project Lambda : Closures for Java (JDK 8 ?)
● Da Vinci Project : invokeDynamic (JDK 7 ?)
● Is Gpath LINQ for Java ? (Now ?)
● Universal Low Level VM
● Will Groovy or Groovy++ Continue ?

34. A few bad months for Java
●
JavaTM By Oracle ,
● The Java Community Process, Doug Lea
● The Java Community Process, Apache
● JavaOne, The Conference
● Oracle vs Google
● Oracle vs Apache
● IBM and Apache
● OpenJDK

35. Where is Goovy Going ?
● Some Important JDK Improvements will be in
JDK 7 (as planned): invokeDynamic
● More explicitly FP approach to concurrency and
state
– AST MP / GoF Patterns
– @Immutable / @Singleton ( lazy : true)
– GPars Actor-like syntax
● Groovy Server for Fast Scripting
– Difficult on a cluster

36. Where is Goovy Going ?
● Closures with Memory
def cl = {a, b →
sleep(3000) // create a noticeable pause
a + b
}
def mem = cl.memoize()
// memorize() variants are
// memorizeAtMost(n), memorizeAtLeast(n),
// memorizeBetween(m, n)
● Groovy++
– Not a fully compatible fork
– An interesting argument
– But no.

37. Groovy Gotchas!
● Subclassing Map, Unexpected Results
● Map.class
● the dot operator,
getAt(...), getXXX(), [ name ] literals
● Private isn't Private
● Inconvenient when you really want to hide implementation data
● Unexpected Weird Effects
– Combining Static and Dynamic Numeric Types Across Java Code
and Groovy Code
● Be careful and deliberate with typing
– Method selection surprises
● identical static closures and static functions do not respond to
the same way Invoked dynamically

38. Best Practices
● Explicitly Signatures and Types Where You Can
● Better GroovyDoc documentation
● Faster method selection
● More shareable (with Java / Scala / Ioke / etc..) Libraries
● Write Groovy Aware (GroovySupport) Java
● This is where to build your high-speed / high-performance code
● Use ConfigSlurper to externalize environment / context data
● Use CliBuilder and Groovy Server if it fits your production model
● Add Performance Tests To Your Suite / Learn to use Visual VM
– Testing is more important with dynamic and functional programming
– Small tests – small in terms of code coverage – on big data sets
● Best place to find surprising performance enhancements
– Automating 'Just Got Lucky' Into Your Development Cycle

39. Acknowledgments & Suggestions
● Derik Kognig, Author Groovy In Action
● Peter Bell, Systems Forge
● Paul King, Asset Australia
● Guillaume Laforge, Groovy Project Manager
● PLEAC, Project Euler, Groovy Zone

40. Contact Info
● jonathan.felch@gmail.com
● jfelch@volanttrading.com
● www.twitter.com/Jonathan.Felch
– Rare but regular contributions via Blog, GitHub,
SlideShare
● 3 – 5 contributions a year, typically unfinished half-baked ideas
● Really targeted at a dozen or so people that I have worked with
in the past or occasionally collaborate