Wednesday, May 25, 2011

Java SE 8 Modularity Requirements

Mark Reinhold posted the requirements for the Module System in JavaSE 8 here today: . And, I won't deny it, I was part of the group that worked on getting these requirements formulated, although I don't fully agree with all of them, but more on that below.

There are many, many good requirements in this document and Java will certainly gain from having these realized - if this is done in an open and inclusive manner.

So what does this mean for OSGi? OSGi is a widely used, mature, modularity standard today, existing for 10+ years.
Well, the good news is that the new requirements are not incompatible with OSGi. There is actually a specific requirement around the necessity for OSGi to be able to interact with the JavaSE modules.

Many of the requirements mentioned in the document are already supported by OSGi today. The file format that defines the modules will most likely not be the MANIFEST.MF file that is used by OSGi today, but another file of some sort. OSGi will support this new mechanism to declare the module as well in the future; the semantics are the same, the format will be slightly different.

Another good thing is that once modularization is part of Java SE 8, moving to OSGi will be easier than it is today. The biggest stumbling block for migrating an existing system to OSGi today is often the modularization of a system that wasn't created in a modular way in the first place. It's not an issue with OSGi per se, but with modularity as a concept. If you have existing code that is non-modular, it can be difficult to modularize it. With JavaSE 8 this problem will appear earlier in the development cycle which should help people design their systems in a modular way from early on. If you need more than the basic JVM module system then you can move to OSGi taking your existing JavaSE 8 modules and they will just work as-is - so migration will be a breeze. Then you can enhance the modules to take advantage of  the OSGi benefits (see below).

Why not take OSGi as-is?
Well, this would be nice. And it should be quite possible to take a (possibly reduced) OSGi framework and make that the module system for Java SE 8. But to a certain degree it's an implementation detail. The important thing is that the Java SE 8 Module System is a well-defined standard. Whether JRE implementations take OSGi as the basis for their module system or write their own from scratch is up to them.

Does OSGi still add value? Absolutely! Here are some points:

  • dynamicity and lifecycle: OSGi modules are dynamic; you can add, remove and update modules from a running framework. It allows patching applications without taking them down! (I have demo'ed this many times) This will probably not be possible in the JavaSE 8 modules.
  • advanced service registry: the OSGi Service Registry is much richer than the Java SE java.util.ServiceLoader class. It allows filtering for services on additional metadata (beyond the interface name), supports multiple instantiation modes and, again, is dynamic: services can come and go.
  • richer dependency metadata: OSGi has an extremely rich metadata model that supports specifying dependencies in many ways. It does support module-level dependencies (as is proposed in the Mark's requirements doc) through the Require-Bundle header. However, most people who have been working with OSGi for the last 10+ years have realized that pure module-level dependencies are inferior to declarative dependencies such as the ones provided by OSGi's Export-Package and Import-Package. Since OSGi 4.3 generic capabilities and requirements are also supported, which allows you to create your own dependency system.
  • many other OSGi specifications: there are many OSGi specifications that describe useful things which can be used on top of any OSGi framework. See the compendium and enterprise specifications for the details.

Where are the disagreements?
They're actually spelled out in the requirements document.

  1. The form of the module definition. Having module metadata in a file different from MANIFEST.MF is fine, in my opinion. MANIFEST.MF is a near text-file format with some strange limitations. A proper DSL would be nice for module definitions. I don't see the value, however in compiling that information into some binary format. It has been invaluable to me many times in the past to be able to open an OSGi bundle, look at the manifest and see exactly from that information what the module definition is. Compiling the information into a binary format will make it harder to see the module definition and it might also make life harder for tools that work with the module system.
  2. The file extension of the module definition. Some argued that the module source definition should be in a .java file and because it's in a .java file there should be a compiled .class file in the resulting Jar file. However, I think that the structure of a module definition screams for a specific DSL that might be quite different from ordinary Java. Especially the extensibility requirement makes a DSL much more natural than a traditional .java source file. I think the module definition should be in  META-INF/module.jmod or something like that. Then it becomes natural to migrate that file as-is to the resulting jar. In a sense the existing MANIFEST.MF does cover most of the requirements. Sure, it has some issues (esp. the 76 chars-per-line limit) but it's clear, simple and extensible - and - it is readable by humans in the Jar file.

I congratulate Mark Reinhold and all others involved in this for providing the basic requirements and design proposal that will make OSGi work nicely with the JavaSE 8 module system. OSGi will extend the basic JavaSE 8 modules and both systems will work with each other to form a layered design.


  1. One more thing to note, the requirements mention this (in the Extensible module declarations section):

    "The syntax must place all extended metadata after all standard metadata, with a clear delineation between them."

    I don't see why this requirement is needed and I actually think it's bad because it doesn't encourage extending the module definitions. Again, the MANIFEST.MF is a good example where this was not done (standard headers are mixed with custom headers) and it has proven to be extremely useful.

    Let's come up with a module definition syntax that encourages extensions and where those extensions happily co-exist with the JavaSE definitions.

  2. Just to clarify one additional point, this blog is obviously my own opinion and the following statement, is how I would like OSGi to react to the new JavaSE 8 module definitions file:

    "The file format that defines the modules will most likely not be the MANIFEST.MF file that is used by OSGi today, but another file of some sort. OSGi will support this new mechanism..."

    This decision has not yet been made in the OSGi Alliance, but I will do my best to get there :)