Current themes in the OSGi Enterprise Expert Group

After the OSGi Enterprise R5 release this summer, the OSGi Expert groups have started working on new items for the OSGi R6 releases.

At this point in time, the main themes in the Enterprise Expert Group are Cloud and JavaEE integration. In the EEG the following documents are currently being discussed:

• OSGi/CDI integration (RFC 193) - jointly led by Red Hat/JBoss, SERLI and Luminis. This RFC aims to define a standard for OSGi-CDI implementations. A number of different implementations currently exist such as weld-osgi, Pax CDI and in Glassfish and others. The aim is to bring the CDI programming model to OSGi with portability across compliant implementations.
• OSGi Cloud Ecosystems (RFC 183) - jointly led by Red Hat/JBoss and Paremus. A standard OSGi-based PaaS definition with a focus on creating a highly portable dynamic cloud environment. This work is still in the early stages of discussion but you can find some of the thinking in my EclipseCon Europe presentation: http://www.slideshare.net/bosschaert/osgi-cloud-ecosystems
• Complex Repository Requirements (RFC 187) - lead by Red Hat/JBoss. This is a small enhancement to the OSGi Repository specification which was released as part of the Enterprise R5 release and for which Thomas Diesler did the Reference Implementation at JBoss. The enhancements discussed in this RFC focus on more complex requirements for the Repository back end.
• Portable Java Contracts (RFC 180) - lead by IBM. Read more about this RFC in my previous blog post.
• Enhancements to the OSGi HTTP Service (RFC 189)* - lead by Adobe. The OSGi HTTP Service specification is highly popular because its such an easy way to get Servlets up and running. However the specification was long overdue an update. This is what this RFC is about. It is about modernising the Servlet API and also brings the white board pattern to this specification.
• REST management (RFC 182) - lead by IBM. This RFC focuses on being able to remotely manage an OSGi Framework through REST, for example for use in a Cloud environment. For more information see my previous post.
• OSGi/EJB integration (RFC 194) - lead by IBM. This RFC has just started and aims at addressing the requirements from RFP 152.
• Blueprint version 1.1 (RFC 184) - lead by IBM. Discusses a number of improvements to blueprint, which include configuration around service damping and the grace period.
• Declarative Services Enhancements (RFC 190)* - lead by Adobe. This document discusses a number of improvements to DS among which are an Administrative API and new annotations.

Note that the OSGi/JCA integration (RFC 146), lead by Jesper Pedersen from Red Hat/JBoss is likely to become active again soon.

Additionally, a group of people is proposing an OSGi standard for C/C++. They are working on a new RFP which should be available in the near future.

At this stage many documents are still at the early stages of the technical design. The target date for the OSGi Core and Enterprise R6 releases is currently Q1 2014.

Note - if you're interested in getting involved in the RFC discussions and you are working for an OSGi member company, contact me (or any of the other EG co-chairs). If you're interested but don't work for a member company, have a look at the OSGi membership page: http://www.osgi.org/Join

* These RFCs are crossover topics with the OSGi Core Expert Group CPEG.

New OSGi EEG Early Access Draft available

A new Early Access Draft has been published at OSGi that contains 3 RFCs that are currently being discussed in the Enterprise Expert Group.

Blueprint Transactions (RFC 164)
This RFC focuses on how to add declarative transaction support to the OSGi Blueprint standard.

Portable Java Contracts (RFC 180)
OSGi has always been very outspoken about versioning Java packages (for more info see the Semantic Versioning white paper: http://www.osgi.org/wiki/uploads/Links/SemanticVersioning.pdf) however, what version to use for APIs defined by JavaEE has been unclear. While many people have been talking about these APIs in terms of the JavaEE versions (e.g. Servlet 2.5, Servlet 3.0) these version numbers don't always match with the Semantic Versioning ideas that OSGi promotes. For example, with Semantic Versioning you would only increase the major version if you are breaking backward compatibility. When Servlet 3.0 came out of the JCP it did actually not break compatibility with 2.5, but still the major version was increased. In the end the various OSGi-enabled application server implementors used different versioning schemes for JavaEE packages, which made it unclear how to use these in a portable way from OSGi. This RFC aims at addressing this issue in a way that is both natural to people familiar with the JavaEE version numbers as well as being able to cope with the various package version numbers used in the OSGi capable Application Servers.

REST Management Interface (RFC 182)
In the context of Cloud Computing, REST is becoming the dominant way of accessing and controlling remote applications running in the cloud. This RFC defines a remote management interface for OSGi bundles over REST and defines both JSON as well as XML data bindings.

You can find the Early Access Draft here: http://www.osgi.org/News/20121011
Remember that these drafts are not final - if you have any feedback, let us know either via bugzilla (http://www.osgi.org/bugzilla) or via the osgi-dev mailing list: http://www.osgi.org/MailLists

Disappointed by Jigsaw delay? Java Modularity is here today!

I recently heard that some people are expressing disappointment over the lack of a built-in module system in Java 8, now that Jigsaw is delayed. Some are apparently even looking at technologies other Java because of this. Well, I'd like to point out that There is No Need To Wait! All the modularity that your application needs is already available with OSGi! OSGi pretty much supports everything that Jigsaw was planning to support to allow modularization of applications. And OSGi has more features such as a dynamic lifecycle and a very useful services model. Still OSGi frameworks can be remarkably small. Of the popular implementations Apache Felix is probably the smallest with its implementation in under 500kilobytes. But Eclipse Equinox and our own JBoss OSGi Framework inside AS7 are also very good opensource implementations.

Want to learn how to use OSGi? Like many people I like learning stuff by reading a book. Recently a number of books have been published that provide really good explanations of how to use OSGi, notably OSGi in Action, OSGi in Depth and Kirk Knoernschilds Modular Architecture book, but probably my favorite book is Enterprise OSGi in Action. Besides containing a good description of a number of Enterprise OSGi specs, it also describes really well why certain things are done in OSGi they way they are, which give you a nice and deep understanding of the technology. Additionally it contains some great chapters on how to get your build, test and development environments to work smoothly with OSGi. Enterprise OSGi in Action will be in print later this year, but you can already get it in electronic form from here.

Besides the books, the OSGi Javadoc can be very useful and if you really want to know the details of it all, the OSGi specs are a great resource. Oh, and don't forget to check out the Wikipedia page that shows where you can get implementations of the various OSGi technology.

So in short - no need to wait or turn away from Java if you need Modularity. OSGi is here today and has been used in production by numerous users for many years, just get started by downloading one of the opensource implementations and try it out!

Presenting about Cloud Ecosystems at OSGi Community Event 2012

Over the past few months I've been piloting what I started to call an OSGi Cloud Ecosystem. You can see the details on my technical blog <coderthoughts/>. I am coming to the conclusion that OSGi can provide a really interesting basis for a Platform as a Service that can support the fluidity of the cloud, where services migrate from one VM to another and systems can scale up and down as needed, without having to worry about getting swamped by configuring what runs where etc. Your OSGi system is based on a Service Oriented Architecture where your services are implemented and consumed using the widely supported OSGi services programming model. Your system is comprised for multiple cloud VMs each of which may take different roles over time. As the need arises you add or remove cloud VMs that contain some of your OSGi services, migrating services across VMs as needed. However as a consumer you simply use the OSGi Service Registry to find and use your services.

The postings on <coderthoughts/> show a pilot that implements these ideas and I'll be talking more about them (and showing a demo) at the OSGi Community Event 2012 (which is part of EclipseCon Europe 2012) on October 23-25 in Ludwigsburg, Germany. See here for the programme: http://www.osgi.org/CommunityEvent2012/Agenda for general conference information go here: http://www.osgi.org/CommunityEvent2012

Hope to see you there!

Feedback requested for OSGi RFPs

Now that the summer holidays are drawing to a close for most, it would be a good time to look at some of the new work that is happening at OSGi.
In OSGi new specification efforts are kicked off by creating an RFP, a document that lists the uses-cases and requirements that the ultimate specification must implement. Once an RFP is agreed on the actual technical design and spec-work starts through the creation of a follow-up RFC.

Recently 4 new RFPs have been made public via OSGi bugzilla to see if people have further input or requirements, a process that has worked really well for the OSGi/CDI RFP and a little longer ago for the OSGi Cloud RFP.

RFP 143 OSGi Connect

The people at PojoSR have done some great work in showing how you can use OSGi Services in cases where you may not want to take on OSGi modularity (yet). RFP 143 captures requirements for creating a specification around this. The RFP can be found here: RFP 143.

RFP 150 HTTP Service Updates

The OSGi HTTP Service is widely used and it provides a very nice programming model for sevlets in an OSGi environment. However the specification is in need for an update to modernize it in relation to the Servlet spec itself. Additionally, the whiteboard pattern is introduced for servlet. You can find the RFP here: RFP 150.

RFP 151 Declarative Service Updates

A variety of updates are proposed to the Declarative Services specification. They can be found in RFP 151.

RFP 152 EJB Integration

EJB integration with OSGi has been done at Apache Aries, Glassfish and ObjectWeb (EasyBeans / JOnAS). Other appservers such as JBoss and others also provide some form of interaction between EJBs and OSGi. This RFP aims a writing a specification for this integration to make OSGi-enabled EJBs portable. You can find the RFP here: RFP 152.

So if you have any thoughts about these topics, do chime in and leave your comments or ideas on the associated bugs!

No Jigsaw for JavaSE 8

Yesterday Mark Reinhold announced that Oracle will not pursue modularity for JavaSE 8: http://mreinhold.org/blog/late-for-the-train. Following long delays for the Modularity JSR the decision was made not to postpone JavaSE 8 or rush through the JSR but rather postpone the Modularity part of Java 8 and push ahead with the rest of it.

So what does this mean for Modularity in Java? Well, I think its an appropriate decision to not rush things through. As modularity is such a fundamental piece of functionality you don't want to cut corners with the architecture. OSGi has been providing modularity to Java for over 12 years now and there are a lot of lessons to be learnt from it.

Project Jigsaw is a prototype of a modularity system that's part of the Java runtime, but at this stage it's just one approach to this. In Project Penrose work is ongoing around improving the extensibility of Jigsaw so that it can work better with other module systems such as OSGi.

While some work is definitely needed to modularize Java itself, as an application developer you don't need to wait for this. If you need modularity in your applications, then OSGi provides all that you need today and it has been proven for years by many in the field.

It's good that the Modularity JSR will get some proper time now. I hope the process will be democratic and that the result will provide the best technology possible that keeps extensibility in mind.

OSGi Enterprise R5 specifications released

While the Early Access draft was already available for a couple of months, the OSGi R5 Core and Enterprise specifications are now officially released. The Enterprise release contains a couple of significant new specifications. While the blog post on the OSGi website gives general information about this release, I'm spending a few more words on the new specs that are part of the Enterprise release here.

The Repository specification
Repository is a very interesting new spec that provides an API based on generic capabilities and requirements to access (possible remote) repositories. With this API you can do things like: 'I need a bundle that exports package javax.servlet version 2.5' or you can say 'I need a bundle that provides a Declarative Services implementation version 1.2' without having to know what the name of the bundle is. The Repository can be used with any kind of resource (it doesn't have to be an OSGi bundle) and it works with any type of capability. So you can also make your own capabilities and express requirements in terms of those to find your resources.
Besides the API it also defines an XML interchange format that can be used to export and import repository information across repositories. Note that the resources known to the repository don't actually need to reside inside it, they can live anywhere as long as its accessible via a URL.
The Reference Implementation of the Repository is currently being worked on in the JBoss OSGi project.

The Resolver Specification
OSGi Frameworks have contained resolvers for a long time. This specification allows other entities to use this resolver as a standalone entity as well. The Resolver can obviously be used to resolve OSGi type of requirements, which can be used to evaluate what-if scenarios. The Resolver is stateless, and doesn't need to perform its operations in relation to the local framework, so it can be used to see whether a bundle or set of bundles would resolve in another framework. Additionally, the Resolver can be used on any type of constraint that can be expressed as generic capabilities and requirements, and, as with the Repository, it doesn't need to operate on bundles per se. It can resolve requirements over a set of capabilities provided by any type of resource. While I haven't tried it (yet) you should be able to use it in completely different domains. The RI of the Resolver is in a well-advanced stage at the Apache Felix project.

Subsystems
The Subsystems specification can be described in many different ways. One of the things it allows you to do is to create an archive (an .esa file) which contains a number of bundles which together provide a piece of functionality. For example, take the CXF-DOSGi project which implements the Remote Services specification. To deploy CXF-DOSGi you previously had to deploy a large number of bundles. Composite systems like this can now be bundled in a subsystem, where you only need to deploy a single subsystem to get all these bundles in your framework. There are many other aspects to subsystems too. Instead of bundeling up all the bundles in the .esa file you can also describe the subsystem purely in a declarative sense, by only having a descriptor that says 'my subsystem is defined by bundle1, bundle2 and bundle3'. The subsystem implementation can then use the Repository and Resolver to locate these bundles, pull in their transitive dependencies and deploy the whole lot in your OSGi framework.
Additionally, Subsystems provides a feature for QA teams. It allows you to add an additional Deployment manifest which can lock down the actual versions of the bundles that the subsystem is to be used with. QA teams can use this to lock down the versions to the ones they tested.
Besides all this, Subsystems provides isolation features which are aimed at allowing multiple subsystems to run within the same framework without interference. The Subsystems RI is being implemented in Apache Aries.

ServiceLoader Mediator Specification
This specification deals with getting java.lang.ServiceLoader to work in an OSGi framework. ServiceLoader has been in JavaSE since version 6 and there are lots of problems with it, the main ones being that its non-modular and non-dynamic by design. However there is code out there that uses ServiceLoader and this code needs to work in an OSGi framework as well. That's what this specification enables. The reference implementation is being done in the Apache Aries SPI-Fly component.

Common Namespaces Specification
Namespaces in this context describe various types of information that can be expressed as generic capabilities and requirements. This specification describes a number of these namespaces. They can be used during the resolution process but also as part of the metadata in a Repository so that they can be found using Repository queries. The following namespaces are defined in this spec (note that the Core spec also defines a number of different namespaces):

• osgi.extender: this allows a bundle to express that it needs an particular extender (e.g. I need Blueprint in order to function). The extenders themselves provide this capability.
• osgi.contract: can be used as a convenient way to import a number of packages that together provide a certain contract. For example you could say: give me all the packages that are part of the Servlet 3.0 spec. This namespace is intended for tools to allow developers to specify a contract rather than the individual imports. Under the covers this contract can automatically be expanded into the appropriate import-package statements.
• osgi.service: allows a bundle to express that it needs a given service and similarly allows a bundle to express that it can provide a certain service.

Also included is a new Coordinator specification (which was first released in the Compendium 4.3 release) and significant updates to the JMX specification. The Configuration Admin specification got some new features (among which are 'targeted pids') and some minor changes were made to a number of other specs.

You can download the OSGi Enterprise R5 specification from here: http://www.osgi.org/Release5

Java 8: extensible module definitions with JSON

While the Java Modularity JSR [1] is still AWOL (come on, Oracle, I thought that this JSR was planned to be launched by September 2011!), I did some work in OpenJDK Penrose around changing the Java SE 8 module metadata from the .java/.class format into JSON.

JSON is quite nice because

• it's a simple text format
• easy to read by humans
• easy to parse for computers
• its highly extensible

with JSON, the SE 8 Quick Start module org.astro module definition looks like this:

{
  "module" :
  {
    "name" : "org.astro",
    "version" : "1.2",
    "exports" : [ {"name" : "org.astro"} ]
  }
}

and the same module info file can be expanded to contain OSGi metadata, for example to add a version to the exported package like this:

{
  "module" :
  {
    "name" : "org.astro",
    "version" : "1.2",
    "exports" : 
    [
      { 
        "name" : "org.astro", 
        "org.osgi.version" : "1.2.3"
      }
    ]
  }
}

Any OSGi metadata can be added to the JSON module descriptor. I've added a simple test to the Penrose codebase to give an idea what this looks like: hello-json-custom.sh.

This works today in OpenJDK project Penrose. The codebase still supports module-info.java, to allow a gradual migration to JSON-based metadata, however I'd like to see the module-info.java/.class support go away ultimately.

While Jigsaw/Penrose are two variants on a module system for Java SE, other ones are clearly possible. Once the modularity JSR is up in action, it should seriously consider OSGi as a module system for Java SE 8. Apache Harmony has clearly demonstrated in the past that modularizing the Java platform using OSGi metadata works quite well. OSGi has proven to be an excellent module system for Java for the last 12 or so years, so it would be foolish not to take that experience on board...

On the lack of Modularity JSR, I think at that at this stage we can expect that it will delay the release of Java 8, given that modularity is such an important building block for Java, and is certainly not something that can be rushed through.

[1] 'Java Platform Module System' JSR as mentioned in the Java SE 8 umbrella JSR.

Presentation on new specs in OSGi 5.0 Enterprise release (EclipseCon/OSGi DevCon 2012)

Last week Tim Diekmann and I presented an overview of the new specifications that are part of the OSGi 5.0 Enterprise Release. You can find this presentation online at this link.

Draft releases of the OSGi R5 specs can be downloaded from here:

• Core http://www.osgi.org/download/osgi.core-5.0.0-pfd.pdf
• Enterprise http://www.osgi.org/download/osgi.enterprise-5.0.0-pfd.pdf

The actual final specifications will be made available not too long from now. They are currently in the voting process at the OSGi Alliance. Except for some typos and minor formatting issues the final specs are identical to the above EA drafts.

OSGi Cloud Workshop at EclipseCon/OSGiDevCon 2012

Many people are seeing how well OSGi fits conceptually with Cloud Computing. Its modularity is frugal on memory, and provides isolation. Additionally it focuses components on a particular task, which makes them better maintainable too. Furthermore, OSGi dynamicity makes it possible to create Cloud components that react to changes in the cloud environment in a natural way. They can be dynamically configured, reconfigured and an OSGi Framework can be dynamically provisioned and re-provisioned as well!

Over the past while the OSGi Cloud Working Group has been looking at how OSGi can be enhanced to provide additional support for Cloud Computing. This has resulted in a requirements document called RFP 133.

At this point in time we need to discuss which of these requirements should be tackled first? Is it Discovery in the Cloud? Or Metadata? Or Management or maybe something else?
At the same time we should embrace any existing standards in this area and certainly not attempt to reinvent the wheel!

This discussion is the topic of the Second OSGi Cloud Workshop which is held this year at the EclipsCon/OSGiDevCon venue on Thursday March 29 at 9am.

It is set to be a morning with some presentations (the lineup currently is: Paremus, JClouds, eBay and RedHat/JBoss), a demo, and a lot of time for discussion. Entrance is free - you don't need an EclipseCon pass, but you do need to register as numbers are limited (see here also for details on the location of the venue): http://osgi-cloudworkshop-2012.eventbrite.co.uk

Oh, and just to be sure: EclipseCon/OSGiDevCon 2012 has moved and is in Washington/Reston this year!

JavaSE 8 Modularity, Project Jigsaw and OSGi

One of the requirements for the JavaSE 8 module system is to provide support for and be able to integrate with OSGi. I am therefore very happy to see that a new OpenJDK project is being proposed to focus on precisely this: Project Penrose. Tim Ellison posted the proposal here: http://mail.openjdk.java.net/pipermail/discuss/2012-January/002320.html

They're still looking for initial supporters, authors and committers, so if you're interested in helping out just sign up!

BTW I posted before on this topic, see here: http://osgithoughts.blogspot.com/2011/05/java-se-8-modularity-requirements.html