InfoQ interview about work in the EEG

A little while ago I did an interview with Alex Blewitt from InfoQ about current activities in the OSGi Enterprise Expert Group. The interview is now available from here: http://www.infoq.com/interviews/Bosschaert-changes-OSGi-Enterprise-Specification

OSGi Cloud Ecosystems RFC 183 in public draft

Recently the OSGi Alliance has made an Early Access Draft available of many of the RFCs that are currently being worked on in the Enterprise Expert Group (EEG) and the Core Platform Expert Group (CPEG).

One of these is RFC 183: OSGi Cloud Ecosystems.
OSGi Cloud Ecosystems is about having a many OSGi frameworks co-operating in a Cloud or Cloud-like environment. Its about having multiple cloud nodes work together to provide a coherent logical function. This includes the ability to reconfigure your deployments to scale up when needed or scale down when possible. It also includes providing the ability to react to volatility in your Cloud environment, as nodes might die unexpectedly or may become unresponsive for some reason.

A large portion of RFC 183 is about how you discover things. How do I discover what OSGi frameworks are available in my environment? And what are the characteristics of those frameworks? What cloud or other environment are they running on and where? For certain applications knowing the actual country or region of your deployment is vital, so this is one of the pieces of information available.

When a cloud node appears it often appears on a dynamically assigned node. RFC 183 is about letting the other OSGi frameworks in the environment know that there is a new Framework available so that they can communicate with it. Information passed to other frameworks includes things as the IP address of the node, which is obviously vital in order to be able to communicate. Additionally information about the type of infrastructure the node is running on is made available. Typically the process that reacts to changes in the topology involves some of the following:

• If a new framework appears, it needs to be provisioned, meaning that an entity in the system needs to decide what artifacts need to be deployed on it.
• Once provisioned it needs to become part of the overall function of the system. This means that other entities on the various participating nodes must be wired up to use the new node.
• Similarly, when a VM is taken out of the environment, the interested other nodes are informed so that they can adjust to the situation.
• Other conditions may also require a change in the way the system is provisioned can be:
• when the load on a particular node gets too high - we need to find some quiet nodes that can take over some of the load.
• when the functionality offered by the application changes, maybe the user has changed the application configuration.
• when the system is quiet - we can give some resources back for consumption by other components.

RFC 183 aims at making it really easy to use OSGi services across Cloud nodes. This means that as a developer you're just writing OSGi services (which are really just Java POJOs) and that you can make these available across your Cloud Ecosystem to all the frameworks available. The mechanics are based on the OSGi Remote Services specs. The nice thing is that the Discovery piece takes care of wiring these services up, so you don't need to know where the service is running, you can simply find it in your local Service Registry and use it. If the service migrates to another VM, or another copy of it becomes available, Discovery will update the Service Registry in your framework so that it's up to date with what services are available in the ecosystem. The OSGi service model is already very much geared towards dynamicity of services, so the potential volatility of a cloud deployment fits here very well. Existing service component such as Declarative Services, Blueprint and others can work well with these Distributed Services while providing a very simple programming model at the same time.

RFC 183 is really about providing a base layer for dynamic applications that run in the cloud. It's minimal in the sense that it provides some primitives that allow you to create a dynamic, service-based cloud system in a way that is very familiar if you are used to OSGi services. It gives you all you need to write a provisioner that is part of the system, but it does not define a cloud deployment descriptor format or anything like that. At this stage you need to write your own provisioner that knows how to provision your application, although a more generic provisioning system could be written using the primitives provided.
Or maybe someone wants to create a PaaS that can take direct deployment of Blueprint beans or Declarative Service components. These too could also be modeled atop of what RFC 183 attempts to provide.

At the moment RFC 183 does not attempt to provide a standard mechanism for creating or shutting down compute nodes. There are already a number of projects that abstract over this and I think it would be best to utilize those right now, although it might make sense to describe at some point how these are integrated with the OSGi Service Registry.

It's still early days, this is the first EA draft that contains RFC 183, but from speaking with people who are using OSGi in a cloud environment, many solutions are moving into a similar direction, so it looks like things are converging.

You can find RFC 183 in the draft here: http://www.osgi.org/Download/File?url=/download/osgi-early-draft-2013-03.pdf

If you have any thoughts about it, let us know what you think at the osgi-dev mailing list or to create a bug around the topic in OSGi buzilla.

Additionally, I'll be talking about this topic at EclipseCon: http://www.eclipsecon.org/2013/sessions/osgi-cloud-ecosystems

Current themes in the OSGi Enterprise Expert Group

After the OSGi Enterprise R5 release this summer, the OSGi Expert groups have started working on new items for the OSGi R6 releases.

At this point in time, the main themes in the Enterprise Expert Group are Cloud and JavaEE integration. In the EEG the following documents are currently being discussed:

• OSGi/CDI integration (RFC 193) - jointly led by Red Hat/JBoss, SERLI and Luminis. This RFC aims to define a standard for OSGi-CDI implementations. A number of different implementations currently exist such as weld-osgi, Pax CDI and in Glassfish and others. The aim is to bring the CDI programming model to OSGi with portability across compliant implementations.
• OSGi Cloud Ecosystems (RFC 183) - jointly led by Red Hat/JBoss and Paremus. A standard OSGi-based PaaS definition with a focus on creating a highly portable dynamic cloud environment. This work is still in the early stages of discussion but you can find some of the thinking in my EclipseCon Europe presentation: http://www.slideshare.net/bosschaert/osgi-cloud-ecosystems
• Complex Repository Requirements (RFC 187) - lead by Red Hat/JBoss. This is a small enhancement to the OSGi Repository specification which was released as part of the Enterprise R5 release and for which Thomas Diesler did the Reference Implementation at JBoss. The enhancements discussed in this RFC focus on more complex requirements for the Repository back end.
• Portable Java Contracts (RFC 180) - lead by IBM. Read more about this RFC in my previous blog post.
• Enhancements to the OSGi HTTP Service (RFC 189)* - lead by Adobe. The OSGi HTTP Service specification is highly popular because its such an easy way to get Servlets up and running. However the specification was long overdue an update. This is what this RFC is about. It is about modernising the Servlet API and also brings the white board pattern to this specification.
• REST management (RFC 182) - lead by IBM. This RFC focuses on being able to remotely manage an OSGi Framework through REST, for example for use in a Cloud environment. For more information see my previous post.
• OSGi/EJB integration (RFC 194) - lead by IBM. This RFC has just started and aims at addressing the requirements from RFP 152.
• Blueprint version 1.1 (RFC 184) - lead by IBM. Discusses a number of improvements to blueprint, which include configuration around service damping and the grace period.
• Declarative Services Enhancements (RFC 190)* - lead by Adobe. This document discusses a number of improvements to DS among which are an Administrative API and new annotations.

Note that the OSGi/JCA integration (RFC 146), lead by Jesper Pedersen from Red Hat/JBoss is likely to become active again soon.

Additionally, a group of people is proposing an OSGi standard for C/C++. They are working on a new RFP which should be available in the near future.

At this stage many documents are still at the early stages of the technical design. The target date for the OSGi Core and Enterprise R6 releases is currently Q1 2014.

Note - if you're interested in getting involved in the RFC discussions and you are working for an OSGi member company, contact me (or any of the other EG co-chairs). If you're interested but don't work for a member company, have a look at the OSGi membership page: http://www.osgi.org/Join

* These RFCs are crossover topics with the OSGi Core Expert Group CPEG.

New OSGi EEG Early Access Draft available

A new Early Access Draft has been published at OSGi that contains 3 RFCs that are currently being discussed in the Enterprise Expert Group.

Blueprint Transactions (RFC 164)
This RFC focuses on how to add declarative transaction support to the OSGi Blueprint standard.

Portable Java Contracts (RFC 180)
OSGi has always been very outspoken about versioning Java packages (for more info see the Semantic Versioning white paper: http://www.osgi.org/wiki/uploads/Links/SemanticVersioning.pdf) however, what version to use for APIs defined by JavaEE has been unclear. While many people have been talking about these APIs in terms of the JavaEE versions (e.g. Servlet 2.5, Servlet 3.0) these version numbers don't always match with the Semantic Versioning ideas that OSGi promotes. For example, with Semantic Versioning you would only increase the major version if you are breaking backward compatibility. When Servlet 3.0 came out of the JCP it did actually not break compatibility with 2.5, but still the major version was increased. In the end the various OSGi-enabled application server implementors used different versioning schemes for JavaEE packages, which made it unclear how to use these in a portable way from OSGi. This RFC aims at addressing this issue in a way that is both natural to people familiar with the JavaEE version numbers as well as being able to cope with the various package version numbers used in the OSGi capable Application Servers.

REST Management Interface (RFC 182)
In the context of Cloud Computing, REST is becoming the dominant way of accessing and controlling remote applications running in the cloud. This RFC defines a remote management interface for OSGi bundles over REST and defines both JSON as well as XML data bindings.

You can find the Early Access Draft here: http://www.osgi.org/News/20121011
Remember that these drafts are not final - if you have any feedback, let us know either via bugzilla (http://www.osgi.org/bugzilla) or via the osgi-dev mailing list: http://www.osgi.org/MailLists

Presenting about Cloud Ecosystems at OSGi Community Event 2012

Over the past few months I've been piloting what I started to call an OSGi Cloud Ecosystem. You can see the details on my technical blog <coderthoughts/>. I am coming to the conclusion that OSGi can provide a really interesting basis for a Platform as a Service that can support the fluidity of the cloud, where services migrate from one VM to another and systems can scale up and down as needed, without having to worry about getting swamped by configuring what runs where etc. Your OSGi system is based on a Service Oriented Architecture where your services are implemented and consumed using the widely supported OSGi services programming model. Your system is comprised for multiple cloud VMs each of which may take different roles over time. As the need arises you add or remove cloud VMs that contain some of your OSGi services, migrating services across VMs as needed. However as a consumer you simply use the OSGi Service Registry to find and use your services.

The postings on <coderthoughts/> show a pilot that implements these ideas and I'll be talking more about them (and showing a demo) at the OSGi Community Event 2012 (which is part of EclipseCon Europe 2012) on October 23-25 in Ludwigsburg, Germany. See here for the programme: http://www.osgi.org/CommunityEvent2012/Agenda for general conference information go here: http://www.osgi.org/CommunityEvent2012

Hope to see you there!

OSGi Enterprise R5 specifications released

While the Early Access draft was already available for a couple of months, the OSGi R5 Core and Enterprise specifications are now officially released. The Enterprise release contains a couple of significant new specifications. While the blog post on the OSGi website gives general information about this release, I'm spending a few more words on the new specs that are part of the Enterprise release here.

The Repository specification
Repository is a very interesting new spec that provides an API based on generic capabilities and requirements to access (possible remote) repositories. With this API you can do things like: 'I need a bundle that exports package javax.servlet version 2.5' or you can say 'I need a bundle that provides a Declarative Services implementation version 1.2' without having to know what the name of the bundle is. The Repository can be used with any kind of resource (it doesn't have to be an OSGi bundle) and it works with any type of capability. So you can also make your own capabilities and express requirements in terms of those to find your resources.
Besides the API it also defines an XML interchange format that can be used to export and import repository information across repositories. Note that the resources known to the repository don't actually need to reside inside it, they can live anywhere as long as its accessible via a URL.
The Reference Implementation of the Repository is currently being worked on in the JBoss OSGi project.

The Resolver Specification
OSGi Frameworks have contained resolvers for a long time. This specification allows other entities to use this resolver as a standalone entity as well. The Resolver can obviously be used to resolve OSGi type of requirements, which can be used to evaluate what-if scenarios. The Resolver is stateless, and doesn't need to perform its operations in relation to the local framework, so it can be used to see whether a bundle or set of bundles would resolve in another framework. Additionally, the Resolver can be used on any type of constraint that can be expressed as generic capabilities and requirements, and, as with the Repository, it doesn't need to operate on bundles per se. It can resolve requirements over a set of capabilities provided by any type of resource. While I haven't tried it (yet) you should be able to use it in completely different domains. The RI of the Resolver is in a well-advanced stage at the Apache Felix project.

Subsystems
The Subsystems specification can be described in many different ways. One of the things it allows you to do is to create an archive (an .esa file) which contains a number of bundles which together provide a piece of functionality. For example, take the CXF-DOSGi project which implements the Remote Services specification. To deploy CXF-DOSGi you previously had to deploy a large number of bundles. Composite systems like this can now be bundled in a subsystem, where you only need to deploy a single subsystem to get all these bundles in your framework. There are many other aspects to subsystems too. Instead of bundeling up all the bundles in the .esa file you can also describe the subsystem purely in a declarative sense, by only having a descriptor that says 'my subsystem is defined by bundle1, bundle2 and bundle3'. The subsystem implementation can then use the Repository and Resolver to locate these bundles, pull in their transitive dependencies and deploy the whole lot in your OSGi framework.
Additionally, Subsystems provides a feature for QA teams. It allows you to add an additional Deployment manifest which can lock down the actual versions of the bundles that the subsystem is to be used with. QA teams can use this to lock down the versions to the ones they tested.
Besides all this, Subsystems provides isolation features which are aimed at allowing multiple subsystems to run within the same framework without interference. The Subsystems RI is being implemented in Apache Aries.

ServiceLoader Mediator Specification
This specification deals with getting java.lang.ServiceLoader to work in an OSGi framework. ServiceLoader has been in JavaSE since version 6 and there are lots of problems with it, the main ones being that its non-modular and non-dynamic by design. However there is code out there that uses ServiceLoader and this code needs to work in an OSGi framework as well. That's what this specification enables. The reference implementation is being done in the Apache Aries SPI-Fly component.

Common Namespaces Specification
Namespaces in this context describe various types of information that can be expressed as generic capabilities and requirements. This specification describes a number of these namespaces. They can be used during the resolution process but also as part of the metadata in a Repository so that they can be found using Repository queries. The following namespaces are defined in this spec (note that the Core spec also defines a number of different namespaces):

• osgi.extender: this allows a bundle to express that it needs an particular extender (e.g. I need Blueprint in order to function). The extenders themselves provide this capability.
• osgi.contract: can be used as a convenient way to import a number of packages that together provide a certain contract. For example you could say: give me all the packages that are part of the Servlet 3.0 spec. This namespace is intended for tools to allow developers to specify a contract rather than the individual imports. Under the covers this contract can automatically be expanded into the appropriate import-package statements.
• osgi.service: allows a bundle to express that it needs a given service and similarly allows a bundle to express that it can provide a certain service.

Also included is a new Coordinator specification (which was first released in the Compendium 4.3 release) and significant updates to the JMX specification. The Configuration Admin specification got some new features (among which are 'targeted pids') and some minor changes were made to a number of other specs.

You can download the OSGi Enterprise R5 specification from here: http://www.osgi.org/Release5

Presentation on new specs in OSGi 5.0 Enterprise release (EclipseCon/OSGi DevCon 2012)

Last week Tim Diekmann and I presented an overview of the new specifications that are part of the OSGi 5.0 Enterprise Release. You can find this presentation online at this link.

Draft releases of the OSGi R5 specs can be downloaded from here:

• Core http://www.osgi.org/download/osgi.core-5.0.0-pfd.pdf
• Enterprise http://www.osgi.org/download/osgi.enterprise-5.0.0-pfd.pdf

The actual final specifications will be made available not too long from now. They are currently in the voting process at the OSGi Alliance. Except for some typos and minor formatting issues the final specs are identical to the above EA drafts.

OSGi Enterprise Specification Early Access Draft

A new Early Access Draft containing RFCs aimed at the upcoming OSGi Enterprise Specifications has been made public. The following RFCs are expected to be part of the OSGi Enterprise Release next year:

• RFC 112 OBR
• RFC 152 Subsystems
• RFC 167 java.util.ServiceLoader support
• RFC 169 JMX updates

In addition, the EA draft contains a number of RFCs related to the Core and Compendium specifications.
You can find this draft on the OSGi website here: http://www.osgi.org/download/osgi-early-draft-2011-09.pdf

OSGi Enterprise Specification Early Access Draft available!

An early access draft for the upcoming OSGi Enterprise Specifications has just been made available. This is really interesting stuff and it shows what we've been working on in the Enterprise Expert Group over the last year or so. Although still work in progress, some of these specifications will be included in the next OSGi Enterprise Release which is currently planned to be released during the first half of 2012.

The main theme for the Enterprise Release is around Subsystem/Application support and OBR (the OSGi Bundle Repository). These are covered in RFC's 152 and 112. Together they greatly improve the way you can define and distribute, assemble and install OSGi applications and I'm really looking forward to the moment that these specs are finished and broadly implemented.

Additionally you can find in the draft: RFC 146 JCA support, two new Blueprint specifications and an update to the JMX specification (chapter 124 in the Enterprise Release 4.2).

Finally there is RFC 167. This is a new RFC that I've been working on. It intends to fix a small but quite annoying issue: the fact that java.util.ServiceLoader() and META-INF/services use generally don't work in OSGi. You can work around it, as I've blogged about in 2008, but it's ugly. Using RFC 167 this issue can be addressed so that any non-OSGi code that uses this pattern can work in OSGi without too much hassle. I've been working on an implementation of this in Apache Aries, if you're interested in trying it out.

Note that the specifications are not yet final so significant changes might still occur. Also, feedback is welcomed! You can leave comments here on my blog or write an email to the osgi-dev mailing list.

You can find the EA draft here: http://www.osgi.org/download/osgi-early-draft-2011-05.pdf

Nice magazine about OSGi

The people behind the Jax conferences have created a PDF magazine with a focus on OSGi. There's some good articles in there, so definitely worth a read!

Table of Contents:

• Introduction to OSGi by Jerome Moliere
• What's new in Core 4.3 by BJ Hargrave and Peter Kriens
• Developing with BndTools by Marcel Offermans and Alexander Broekhuis
• OSGi a la Carte by Valentin Mahrwald and Holly Cummins
• OSGi and JPA by Dmytro Pishchukhin
• Cure for Complexity (Business focus) by Richard Nicholson
• Enterprise OSGi by Tim Diekmann and myself

Lots of topics, from Core to Enterprise to Business and also information about popular OSGi projects and developing for OSGi! The download process is a little involved, but it took me only a few seconds to get it. You can get the PDF here: http://jaxenter.com/java-tech-journal/JTJ-2011-04

Enterprise OSGi in JBoss AS7

As many of you know, I have been working on OSGi support in the JBoss Application Server for the past year now. My colleagues at JBoss/Red Hat and I have built an OSGi Framework from the ground up with as focus the best possible integration with the Application Server.

AS7 has been OSGi-enabled since its Alpha1 release and over the past few weeks Beta's have been pushed out on a bi-weekly basis. It's starting to look really good!

Next week at Red Hat Summit/JBoss World in Boston I'll be presenting and demoing the current state of things w.r.t. to Core OSGi, Enterprise OSGi and OSGi/AS integration in AS7. My session is on Friday at 11am: http://www.redhat.com/summit/sessions/jboss.html#38 - hope to see you there!

OSGi Cloud Computing RFP 133

I'm quite excited by the work that is being done in the OSGi Cloud Working Group recently. In this working group we are looking at what role OSGi standards can play in a Cloud Computing environment. Currently we're focusing on the possible requirements in RFP 133. Both OSGi Alliance members as well as non-members are involved in defining this document.

If you're interested, have a look at the RFP. It's available as an attachment to this bug in the public OSGi Bugzilla system: https://www.osgi.org/bugzilla/show_bug.cgi?id=114. Feel free to comment!

If you're interested in becoming more involved, join the OSGi Cloud Working Group mailing list: https://mail.osgi.org/mailman/listinfo/cloud-workshop.

OSGi 4.2 Enterprise Interfaces Jar now available in Maven Central

I can only say: apologies that this took so long, but the OSGi Enterprise Jar is now available from maven central:

group ID: org.osgi
artifact ID: org.osgi.enterprise
version: 4.2.0

or directly here:  http://repo1.maven.org/maven2/org/osgi/org.osgi.enterprise/4.2.0/

This should make using Enterprise OSGi from Maven builds a lot easier.

CXF Distributed OSGi 1.2 is out!

I'm really excited to be able to say that the CXF Distributed OSGi subproject has done its 1.2 release.

With this release it now also provides the Reference Implementation of the Remote Service Admin specification (chapter 122 of the OSGi Enterprise Specification) in addition to being the Reference Implementation of the Remote Services specification (chapter 13 in the OSGi Enterprise Spec). Together with the Apache ZooKeeper-based discovery implementation it provides all you need to do Distributed OSGi.

A lot of work as gone into this release, which is a major refactoring from the previous code base. This was necessary to support the RSA specification which provides a standard API for the internal building blocks that constitute a Remote Services implementation. This provides a number of benefits:

• You can mix & match components from various implementations, for instance you could use the Web-Services based OSGi Service Remoting provided by CXF together with a Discovery System from another project.
• You could replace the default Topology Manager with a custom built one to control more explicitly which services should be exported or imported and how.
• You could make some interesting mash-ups. I think that the Tuscany project provides a implementation of the Topology Manager for SCA, so in theory you could replace the default one with the Tuscany one and control you CXF-based Remote Services through SCA. I haven't tried this out yet, but it should be possible :)

Besides the real spec text (which is quite readable) you can find a some more information on the components described by the RSA spec on slides 6-8 of my Enterprise OSGi presentation.

You can find the CXF-DOSGi 1.2 release here: http://cxf.apache.org/distributed-osgi.html