Modules appear in Java 9 at last, but why? After all, OSGi has been around for more than 15 years and serves as the foundation of many desktop, server, and IoT applications. Its authors state that OSGi solves various dependency management problems in the most powerful way - at run time.
In this session, we will explore the problems that OSGi tries to solve, understand its approach, and identify its failures. Then we will figure out which of those problems the authors of Jigsaw ignored (and why), and which ones they solved (and how). Finally, we will talk about the challenges that Jigsaw brings with itself to the Java ecosystem and what we the developers have to do about them.
https://2017.javazone.no/program/83eb9ed50ba74946ad418a3abea0467a
7. Why Jigsaw is not OSGi?
Mark Reinhold (the Chief Architect of the Java Platform Group):
āā¦As it (OSGi) stands, moreover, itās useful for
library and application modules but, since itās
built strictly on top of the Java SE Platform, it
canāt be used to modularize the Platform itselfā
7
8. Why Jigsaw is not OSGi?
Question: why the presence of j.l.Object in Java
Language Specification, implemented in Java in
turn, does not lead to a bootstrap problem?
Š¤Š°ŠŗŃ: Š”ŃŃŠµŃŃŠ²ŃŠµŃ ŃŠµŠ°Š»ŠøŠ·Š°ŃŠøŃ Java SE, Š³Š“Šµ
OSGi ŠæŠ¾Š“Š“ŠµŃŠ¶ŠøŠ²Š°ŠµŃŃŃ Š½Š° ŃŃŠ¾Š²Š½Šµ JVM (Š½Š°
ŃŃŠ¾Š²Š½Šµ ŠæŠ»Š°ŃŃŠ¾ŃŠ¼Ń).
8
9. Why Jigsaw is not OSGi?
Question: why the presence of j.l.Object in Java
Language Specification, implemented in Java in
turn, does not lead to a bootstrap problem?
Fact: There is a Java SE implementation that
supports OSGi at the JVM level (at the platform
level).
9
10. Nikita Lipsky
ā¢ 20+ years in software development
ā¢ Excelsior JET project initiator
ā 16+ years of contributions
ā compiler engineer
ā team lead
ā product lead
ā etc.
ā¢ Twitter: @pjBooms
10
11. Excelsior JET?
ā¢ AOT-centric Java SE implementation
ā certified as Java Compatible since 2005
ā¢ AOT compiler + Java Runtime
ā mixed compilation: AOT + JIT
ā AOT support for custom classloaders (Eclipse RCP, Tomcat)
ā¢ Toolkit
ā Startup Optimizer
ā Deployment
11
12. Agenda
ā¢ Why OSGi
ā¢ How OSGi does it
ā¢ Why NOT OSGi
ā¢ Why Jigsaw is not OSGi
ā¢ Jigsaw mantra
ā¢ Jigsaw problems
12
13. Where OSGi?
Standardized by OSGi Alliance (IBM, Adobe, Bosch,
Huawei, NTT, OraŃle)
Implementations:
ā¢ Equinox
ā Eclipse IDE, Eclipse RCP, IBM Websphere
ā¢ Apache Felix
ā Oracle WebLogic, Glassfish, Netbeans
13
17. OSGi Module System
OSGi module ā Bundle:
ā¢ Jar or directory
ā¢ Import/export is defined in META-INF/MANIFEST.MF:
Manifest-Version: 1.0 Bundle-ManifestVersion: 2
Bundle-SymbolicName: B
Bundle-Version: 1
Bundle-Name: B Bundle
Export-Package: org.bar
Import-Package: org.foo;version=ā[1,2)ā
17
18. OSGi Module System
ā¢ OSGi bundle imports/exports
ā packages (Import-Package/Export-Package)
ā services (Import-Service/Export-Service).
ā¢ May import other bundles directly
ā Require-Bundle directive
ā However it is not the best practice (less flexible)
18
19. OSGi Module System
19
Bundle A
Class A
Bundle B
Class B
Bundle C
Class C
Export-Package: packageC
Import-Package: packageA
Export-Package: packageA
I
Import-Package: packageA,
packageC
20. OSGi Runtime
ā¢ Resolves Import/Export of OSGi bundles
(wiring)
ā¢ Defines bundle life cycle
ā May start (activate) bundles lazily
ā Enables on-the-fly bundle updates without system
restart (aka hot redeploy).
20
22. Versioning in OSGi
ā¢ OSGI resolves JAR hell:
ā import/export is qualified by version
ā If two bundles require a library bundle of two
different versions, both versions will be loaded
22
23. Why OSGi?
So, OSGi promises:
ā¢ Modularity
ā explicit dependencies
ā encapsulation
ā¢ JAR Hell problem resolution
ā via versioning
ā¢ Hot ReDeploy
ā via ability to update a separate bundle dynamically
23
24. Why OSGi?
So, OSGi promises:
ā¢ Modularity
ā explicit dependencies
ā encapsulation
ā¢ JAR Hell problem resolution
ā via versioning
ā¢ Hot ReDeploy
ā via ability to update a separate bundle dynamically
24
25. Why OSGi?
So, OSGi promises:
ā¢ Modularity
ā explicit dependencies
ā encapsulation
ā¢ JAR Hell problem resolution
ā via versioning
ā¢ Hot ReDeploy
ā via ability to update a separate bundle dynamically
25
26. Why OSGi?
So, OSGi promises:
ā¢ Modularity
ā explicit dependencies
ā encapsulation
ā¢ JAR Hell problem resolution
ā via versioning
ā¢ Hot ReDeploy
ā via ability to update a separate bundle dynamically
26
29. Versioning in OSGi
Question: How to implement versioning?
Task: For given A module importing Lib (v1)
library, and B module importing Lib (v2), it is
required that both versions of Lib working
without conflicts.
Solution: load the versions of the library by
different classloaders.
29
30. Versioning in OSGi
Question: How to implement versioning?
Task: For a given module A importing library Lib
(v1), and module B importing Lib (v2), it is
required that both versions of Lib work without
conflicts.
Solution: load the versions of the library by
different classloaders.
30
31. Versioning in OSGi
Question: How to implement versioning?
Task: For a given module A importing library Lib
(v1), and module B importing Lib (v2), it is
required that both versions of Lib work without
conflicts.
Solution: load versions of the library by different
classloaders.
31
32. Versioning in OSGi
Thus each OSGi bundle is loaded by its own
classloader:
ā¢ Subtype of java.lang.ClassLoader
ā¢ Unique class namespace
ā¢ No conflicts with classes of other bundles
32
35. Encapsulation in OSGi
src/com/foo/exported/A.java:
package com.foo.exported;
import com.foo.internal.B;
public class A {
B useB;
}
src/com/foo/internal/B.java:
package com.foo.internal;
public class B {
}
How to make the class B inaccessible from outside?
35
37. Encapsulation in OSGi
Question: how to make an internal class of a
module declared public inaccessible outside?
Answer: classloaders (!) may hide internal
classes.
37
39. Dynamic Updates
Task: update one (!) changed bundle in a
running program without stopping it
Solution: CLASSLOADERS!
(unload old bundle, load new bundle by a new
classloader)
39
41. Lazy start
Start of bundles in OSGi is implemented via the
bundle activators feature:
ā¢ Each bundle may have an activator
ā Bundle-Activator manifest directive
ā Implementation of an interface with methods
start(), stop()
ā Static class initializer analogue for bundles
41
42. Lazy start
Start of bundles in OSGi is implemented via the
bundle activators feature:
public interface BundleActivator {
void start(BundleContext context) throws Exception;
void stop(BundleContext context) throws Exception;
}
42
43. Lazy start
ā¢ Bundle start can be defined by OSGi
configuration
ā¢ Otherwise the bundle starts when it is
required by other started bundles
43
44. Lazy start
Task: start (load) bundles only when they
become needed for program execution
44
45. Lazy start
Solution: CLASSLOADERS (again!) in OSGi invoke
the start()method of the bundle activator
right before loading of the first class of a bundle
Since classloading in the JVM is lazy, bundle
activation becomes lazy AUTOMATICALLY
45
46. Lazy start
Solution: CLASSLOADERS (again!) in OSGi invoke
the start()method of the bundle activator
right before loading of the first class of a bundle
Since classloading in the JVM is lazy, bundle
activation becomes lazy AUTOMATICALLY
46
47. Lazy start
Solution: CLASSLOADERS (again!) in OSGi invoke
the start()method of the bundle activator
right before loading of the first class of a bundle
Since classloading in the JVM is lazy, bundle
activation becomes lazy AUTOMATICALLY
47
65. On the fly updates?
Bundle ABundle B
Does it work for bundle A?
65
66. On the fly updates?
Bundle B
Class B
Bundle A
Class Š
66
67. On the fly updates?
Bundle B
Class B
Bundle A
Class Š
If bundle B imports bundle Š then there is a
class from B that references a class from A
67
71. Symbolic references resolution
ā¢ A class references other classes and their
fields/methods symbolically
ā¢ JVM resolves those symbolic references with
real values at runtime
71
72. Symbolic references resolution
ā¢ A class references other classes and their
fields/methods symbolically
ā¢ JVM resolves those symbolic references with
real values at runtime
ā¢ Once a reference is resolved, the value of the
reference is never changed!
72
73. On the fly updates?
ā¢ If a reference from a class B to a class A is
resolved then the class Š cannot be unloaded
from the JVM without unloading the class B
73
74. On the fly updates?
ā¢ If a reference from a class B to a class A is
resolved then the class Š cannot be unloaded
from the JVM without unloading the class B
ā¢ Hence if a bundle Š imports a bundle Š then
the bundle Š cannot be unloaded without
also unloading the bundle B
74
75. On the fly updates?
Bundle ABundle B
Letās update bundle A
75
81. On the fly updates?
ā¢ On the fly updates in OSGi work, more or less,
only for leaf bundles that are not imported by
other bundles ā plugins
81
82. On the fly updates?
ā¢ On the fly updates in OSGi work, more or less,
only for leaf bundles that are not imported by
other bundles ā plugins
ā¢ There are much simpler ways than OSGi to
implement plugins
82
83. On the fly updates?
Even leaf bundles are not so easy to unload from
the JVM:
ā¢ The classes from leaf bundles can live in the
JVM after their unloading
ā Known problem: Classloaders Memory Leak
83
84. One does not simply
unload a class from the JVM 84
97. Loading constraints
ā¢ Loading constraints prohibit two classes with
the same fully qualified name to appear in the
namespace of another class
97
98. Loading constraints
B.java:
class B {
T1 f1 = A.f;
int f2 = A.m(t2);
}
A.java:
class A {
static T1 f;
static int m(T2 t)
If B is loaded by L1 classloader and A is loaded by L2 then the
JVM will check that (T1, L1) = (T1, L2) and (T2, L1) = (T2, L2)
==
98
100. Versioning?
ā¢ JVM throws java.lang.LinkageError on loading
constraints violation
ā¢ OSGi DOES NOT help developers to avoid
loading constraints violation
ā Just google for āOSGIā and āLinkageErrorā to
estimate the scale of the problem
100
101. Versioning?
There are bundles in the latest Eclipse versions
with potential loading constraints violations and
nobody cares!
101
105. Reflection ā universal countermeasure
against encapsulation in Java
Encapsulation?
setAccessible(true)
105
106. Encapsulation?
Question: well, at least the encapsulation
problem is solved by OSGi, right?
Answer: OSGi does not protect from
unauthorized access via Reflection
106
110. Lazy start?
Bundle activation order in OSGi depends on
classloading order directly
ā A bundle is started from loadClass() of the
bundle classloader
110
111. Lazy start?
Bundle activation order in OSGi depends on
classloading order directly
ā A bundle is started from loadClass() of the
bundle classloader
However, class loading order is not defined by
the JVM specification!
111
112. Symbolic references resolution
A class may reference other classes and their
fields/methods symbolically. A JVM may resolve
references:
ā¢ Lazily
ā Each reference is resolved on first access
ā¢ Eagerly
ā All references are resolved at the time of class loading
112
113. Lazy start?
The classloading order depends on the class
reference resolution scheme in the given JVM
implementation: lazy, less lazy, eager.
113
114. Lazy start?
Bundle B activator:
class B implements BundleActivator {
public void start() {
assert A.f!= null;
}
Bundle A activator:
class A implements BundleActivator {
static T f;
public void start() {
f = new T();
}
Typical case: Š thinks that Š is already activated, but in fact
Š can be activated after B, so assertion fails
114
115. Lazy start?
ā¢ Bundle activation scheme in OSGi is a time
bomb:
ā If the JVM starts to resolve class references less
lazily, practically all OSGi applications will stop
working
115
116. Why NOT OSGi?
ā¢ Modularity with cycles
ā¢ Hot Redeploy works for leaf bundles only
ā¢ No protection from loading constraints violation
ā¢ No protection of implementation details from
reflective access
ā¢ Bundle activation order depends substantially
on class reference resolution scheme
116
119. Jigsaw vs. OSGi
OSGi is dynamic essentially
ā modules appear at run time only
119
120. Jigsaw vs. OSGi
Jigsaw was thought up as static from the beginning.
Practically all JDK tools know about modules:
ā javac: respects modules visibility rules
ā jdeps: analyses dependencies
ā jar, jmod: pack modules
ā jlink: prepares final image for deployment
ā java: there are modules in runtime as well
120
129. Jigsaw and classloaders
Backward compatibility problem:
According to the specification
getClassloader() == null
for core platform classes.ŃŠ¾ ŠæŃŠ¾ŃŠøŠ²Š¾ŃŠµŃŠøŃ
That precludes splitting of the platform into
modules, with each module loaded by its loader
129
130. Jigsaw and classloaders
Backward compatibility problem:
According to the specification
getClassloader() == null
for core platform classes.ŃŠ¾ ŠæŃŠ¾ŃŠøŠ²Š¾ŃŠµŃŠøŃ
That precludes splitting of the platform into
modules, with each module loaded by its loader
130
132. Versioning
Another detail: import in early Jigsaw versions
(as in OSGi) was qualified by not a single version
but by Š° version range:
ā A module may declare that it can work with a
dependency of āfromā version to ātoā version
132
133. Versioning
Problem 3: Resolving dependencies (wiring
modules) from version ranges is an NP-complete
problem!
ā Reduced to 3-SAT
133
142. Reliable Configuration
Reliable configuration:
ā¢ All module dependencies are resolved
ā¢ No cyclic dependencies
ā¢ No modules containing the same packages
(split packages)
These properties are checked at startup (wiring)
142
144. Strong Encapsulation
Java 9 modules are first class citizens
ā¢ Define visibility access rules
ā via declared export
ā¢ There is no access to non-exported functionality
outside of the module even via reflection
ā Even setAccessible(true) does not work
144
145. Strong Encapsulation
Java 9 modules are first class citizens
ā¢ Define visibility access rules
ā via declared export
ā¢ There is no access to non-exported functionality
outside of the module even via reflection
ā Even setAccessible(true) does not work
145
146. Strong Encapsulation
Java 9 modules are first class citizens
ā¢ Define visibility access rules
ā via declared export
ā¢ There is no access to non-exported functionality
outside of the module even via reflection
ā Even setAccessible(true) does not work
146
149. Reliable Configuration?
ā¢ Reflective access was prohibited between
modules without explicit dependencies in
early Jigsaw drafts
ā¢ However it had to be relaxed when
classloaders had gone from JPMS
ā because Class.forName() has to work
backward compatible
149
150. Reliable Configuration?
ā¢ Reflective access was prohibited between
modules without explicit dependencies in
early Jigsaw drafts
ā¢ Had to be relaxed when classloaders had gone
from JPMS
ā because Class.forName() had to remain
backward compatible
150
151. Reliable Configuration?
However if you may have reflective
dependencies that are not explicitly declared,
where is the guarantee that the resulting
configuration is reliable?
151
153. Jigsaw Layers *
To solve the application containers problem
Layers feature was introduced in JPMS:
ā¢ Local module system for each application in a
container
ā¢ Two modules containing the same package
have to belong to different layers
153
154. Jigsaw Layers *
* The picture is from Alex Buckleyās presentation: Project Jigsaw Under the hood
154
157. Strong Encapsulation?
The platform is split into modules:
ā¢ Which means that private APIs become really
private
ā¢ But what about
sun.misc.Unsafe ?
157
160. Strong Encapsulation?
DI frameworks essentially depend on:
ā¢ reflective access to the code in
which they inject dependencies
ā¢ including non-exported code
160
165. Strong Encapsulation?
Open modules were introduced to solve the DI
frameworks problem:
ā¢ An open module allows reflective access to its
non-exported functionality
165
168. Jigsaw benefits
If all your dependencies are on the classpath now,
migrating to the modulepath would improve the
architecture of your application by eliminating:
ā¢ cycles in the dependencies
ā¢ split packages (jar hell)
ā¢ unsound access into implementation details of
other modules
ā¢ dependencies to JDK private API
168
169. Jigsaw benefits
If all your dependencies are on the classpath now,
migrating to the modulepath would improve the
architecture of your application by eliminating:
ā¢ cycles in the dependency graph
ā¢ split packages (jar hell)
ā¢ unsound access into implementation details of
other modules
ā¢ dependencies to JDK private API
169
170. Jigsaw benefits
If all your dependencies are on the classpath now,
migrating to the modulepath would improve the
architecture of your application by eliminating:
ā¢ cycles in the dependency graph
ā¢ split packages (jar hell)
ā¢ unsound access into implementation details of
other modules
ā¢ dependencies to JDK private API
170
171. Jigsaw benefits
If all your dependencies are on the classpath now,
migrating to the modulepath would improve the
architecture of your application by eliminating:
ā¢ cycles in the dependency graph
ā¢ split packages (jar hell)
ā¢ unsound reliance upon implementation details of
other modules
ā¢ dependencies to JDK private API
171
172. Jigsaw benefits
If all your dependencies are on the classpath now,
migrating to the modulepath would improve the
architecture of your application by eliminating:
ā¢ cycles in the dependency graph
ā¢ split packages (jar hell)
ā¢ unsound reliance upon implementation details of
other modules
ā¢ dependencies on JDK private APIs
172
173. Jigsaw benefits
Jigsaw introduces a migration path to the
modulepath:
ā¢ Old classpath forms Unnamed ModulŠµ
ā¢ Jars from classpath may be temporally moved as
is to modulepath as Auto Modules
ā¢ Module declaration can be added to auto
modules later
173
174. Jigsaw benefits
Jigsaw introduces a migration path to the
modulepath:
ā¢ Old classpath forms Unnamed Module
ā¢ Jars from classpath may be temporally moved as
is to modulepath as Auto Modules
ā¢ Module declaration can be added to auto
modules later
174
175. Jigsaw benefits
Jigsaw introduces a migration path to the
modulepath:
ā¢ Old classpath forms Unnamed ModulŠµ
ā¢ Jars from classpath may be temporarily moved
āas-isā to modulepath as Auto Modules
ā¢ Module declaration can be added to auto
modules later
175
176. Jigsaw benefits
Jigsaw introduces a migration path to the
modulepath:
ā¢ Old classpath forms Unnamed ModulŠµ
ā¢ Jars from classpath may be temporarily moved
āas-isā to modulepath as Auto Modules
ā¢ Module declaration can be added to auto
modules later
176
177. Jigsaw benefits
Unfortunately, most Java developers wonāt benefit
from JPMS immediately:
ā¢ Java EE standards do not define how they will
interoperate with JPMS yet
ā¢ Even servlet containers standard knows nothing
about modules so far
ā Dependencies in war ŃŠ°Š¹Š»Š°Ń are essentially old plain
classpath!
177
178. Jigsaw benefits
Unfortunately, most Java developers wonāt benefit
from JPMS immediately:
ā¢ Java EE standards do not define how they will
interoperate with JPMS yet
ā¢ Even the servlet container standard knows
nothing about modules so far
ā Dependencies in war files are old plain classpath in
fact!
178
180. Conclusion
ā¢ OSGi is a nice attempt to give modules to Java
developers
ā but OSGi has many problems unfortunately
ā including fundamental
ā¢ Jigsaw is a carefully designed system without
visible fundamental problems
ā but with a system of checks and balances
ā there are community acceptance problems
180