OSGi After 20 Years(blog.osgi.org)
blog.osgi.org
OSGi After 20 Years
https://blog.osgi.org/2019/05/osgi-after-20-years.html
52 comments
What does REST have to do with OSGi? I appreciate OSGi quite a bit, bit how does one migrate a module to a microservice without changing the API surface to reflect new error modes that come with networking and missing transactions?
Java RMI, CORBA etc. have failed. Is OSGi somehow an RPC mechanism?
Java RMI, CORBA etc. have failed. Is OSGi somehow an RPC mechanism?
I'd call the same interface but behind that is a layer that talks to REST and if there is some networking error that's a runtime exception. You are probably bringing a DB at that point so I don't know if you need explicit transaction logic. So you could have the service in the same JVM but then cut it out and make it a microservice just by adding a REST "proxy".
[deleted]
it's a local call mechanism, you can't get more nineteen ninety nine than that.
why?
to solve the hypothetical issue of needing two services that depends on different version of the same libraries within the same jvm
everything else it does you could do better with dependency injection and locator/proxy patterns
why?
to solve the hypothetical issue of needing two services that depends on different version of the same libraries within the same jvm
everything else it does you could do better with dependency injection and locator/proxy patterns
"Few people outside the OSGi even remotely understand the requirements and capability model, and that is their loss. However, organizations that reach the maturity level to use it will never let it go. "
I consider it a poor evangelism approach to make me feel stupid and useless. In all honesty I tried to understand OSGi about ten years ago and failed to understand how to get started and how my current project could benefit from this. So we kept using maven and eventually gradle.
I consider it a poor evangelism approach to make me feel stupid and useless. In all honesty I tried to understand OSGi about ten years ago and failed to understand how to get started and how my current project could benefit from this. So we kept using maven and eventually gradle.
It isn't really a competitor to maven or Gradle.
I can see how running a bunch of fat jar services each in dedicated JVM can be competitor to running loads and loads of little OSGi components in one big huge JVM.
Maven and Gradle don't run anything in a dedicated JVM. They're just tools to accomplish various goals based on your dependencies: compile, package, test, maybe deploy. Not even remotely similar to runtime module management like you get in OSGi.
About two or three years ago, I tried to learn OSGi. But I had severe difficulty actually getting started and understanding the intended application model. It seems that the OSGi website links to more resources to get started now[1], so perhaps I should give it another shot.
[1] https://www.osgi.org/developer/where-to-start/
[1] https://www.osgi.org/developer/where-to-start/
The documentation was exceedingly sparse at least when I had to do some OSGi stuff a few years ago. I basically depended on Stack Overflow and obscure blog posts to get anything done.
I had real world experience with OSGi 10 years ago and it was terrible. The JVM still had permgen leaks from loading and unloading stuff so any OSGi deployment was bound to just OOM on permgen eventually. My container had poor (or I didn't understand it well enough) ways to define boot ordering and dependencies within the services (like docker-compose's restart: always and depends_on: stuff) so you would get service-C starting before service-A, which would fail for unrelated reasons but the JVM would go ahead and start listening on port 8080! Bring on the traffic to your broken stuff.
I'm in a OSGi platform today and have to say it's still a broken mess
beyond the fact that the world moved from big irons 25 years ago making this whole exercise in in-jvm-microservices pointless, messing with the classloader breaks most of the good things of the Java platform
classloader deadlocks? check
debugger having a stroke trying to use the correct source between all the overlapping libraries at different version within the classloader tree? check
hot code road unable to figure out all the instances of services to replace causing weird class cast exceptions of classes over themselves? check
having to create library projects that include interfaces from independent services because in the OSGi tree you can't have cross dependents? check
I hate the thing with passion and move into real microservices fast enough
beyond the fact that the world moved from big irons 25 years ago making this whole exercise in in-jvm-microservices pointless, messing with the classloader breaks most of the good things of the Java platform
classloader deadlocks? check
debugger having a stroke trying to use the correct source between all the overlapping libraries at different version within the classloader tree? check
hot code road unable to figure out all the instances of services to replace causing weird class cast exceptions of classes over themselves? check
having to create library projects that include interfaces from independent services because in the OSGi tree you can't have cross dependents? check
I hate the thing with passion and move into real microservices fast enough
throw it all away and get into the real thing https://spring.io/projects/spring-boot
> Calling a nano-service has no overhead, unlike calling a micro-service.
It's statements like that which drive me crazy. It may very well have lower overhead, but even a function call has overhead, and OSGi's overhead is not small (at least last time I used it, circa 2012).
Remember Eclipse? Remember how, once you installed a couple of java plugins, it'd take up to several minutes to start up? All those plugins were OSGi, via the Equinox framework. The overhead was noticeable and severe. Contrast that to IntelliJ, where I have it set up to support a dozen or so languages, all with various plugins, and startup never takes more than 30 seconds. Still not as fast as I'd like, but I shudder to think how Eclipse would manage.
I also echo a lot of the commenters here from the development perspective. OSGi is one of the least pleasant systems I've had to write for. Some of that comes down to Java's questionable approach to dynamic class loading, but OSGi added a ton of cruft and complexity on top of that.
It's statements like that which drive me crazy. It may very well have lower overhead, but even a function call has overhead, and OSGi's overhead is not small (at least last time I used it, circa 2012).
Remember Eclipse? Remember how, once you installed a couple of java plugins, it'd take up to several minutes to start up? All those plugins were OSGi, via the Equinox framework. The overhead was noticeable and severe. Contrast that to IntelliJ, where I have it set up to support a dozen or so languages, all with various plugins, and startup never takes more than 30 seconds. Still not as fast as I'd like, but I shudder to think how Eclipse would manage.
I also echo a lot of the commenters here from the development perspective. OSGi is one of the least pleasant systems I've had to write for. Some of that comes down to Java's questionable approach to dynamic class loading, but OSGi added a ton of cruft and complexity on top of that.
Doesn’t IntelliJ use OSGi for plugin loading too? I looked into this a while ago and it looked like IntelliJ bundles all the OSGi stuff.
No, it doesn't - it just uses a standard IoC container per plugin and some simple classloader isolation between them.
I only have experience with OSGi from analysing the architecture of jailbroken Kindle Touches. It's possible Lab126 didn't use it correctly, though. They don't have a good track record (the first jailbreak for the KT was a mp3 with html tags that were rendered unescaped by the stone age old browser which was running as root).
The article claims "OSGi provides the type safety between modules that Java provides between classes" but I didn't see any of that.
Just a ton of services and modules that send message around that let to so much decoupling that you didn't have any Java language support anymore.
It looked like trying to do OO in C. You can do that but then you are completely on your own and have to ensure everything by hand.
The article claims "OSGi provides the type safety between modules that Java provides between classes" but I didn't see any of that.
Just a ton of services and modules that send message around that let to so much decoupling that you didn't have any Java language support anymore.
It looked like trying to do OO in C. You can do that but then you are completely on your own and have to ensure everything by hand.
I have heard of so many projects inside enterprise and product vendors which were unable to get this OSGi thing working. The answer from OSGi fans is always in same vein: 'you are holding it wrong'.
This article seems a bit arrogant. But I have to acknowledge that they pioneered the trend of microservices in terms of overcomplicated designs and slowness.
Honestly. Some of the most horrifying systems I have worked with were based on the idea that you should be able to swap implementations during runtime. That was the time when OSGi was the thing and (like mentioned in the article) there seem to be only a few how to use OSGi properly. I'm a burned child and certainly don't have the knowledge, either ...but the blog post doesn't give a single reason to change that.
Honestly. Some of the most horrifying systems I have worked with were based on the idea that you should be able to swap implementations during runtime. That was the time when OSGi was the thing and (like mentioned in the article) there seem to be only a few how to use OSGi properly. I'm a burned child and certainly don't have the knowledge, either ...but the blog post doesn't give a single reason to change that.
OSGi is essentially classic bureaucractic enterprise-ism meets OOP dogma cult, and like all "hyperabstraction" in general, greatly decreases systems-level understanding and macro-simplicity (critically important for things like debugging) in order to favour micro-simplicity. To sum it up, "splitting a system into easier to understand parts until each part is trivially understandable causes the actual complexity to become hidden between the interaction of the parts."
To sum it up, "splitting a system into easier to understand parts until each part is trivially understandable causes the actual complexity to become hidden between the interaction of the parts."
Is that in quotes because it's a quote? If so, from where? Or is this a form of emphasis?
Is that in quotes because it's a quote? If so, from where? Or is this a form of emphasis?
I'm not sure. Me and my coworkers have been saying similar things for a while, and it's not really that deep of an observation. Abstraction can't always magically make things easier. It just moves the complexity around, and often increases it if used unnecessarily.
Right, it's very similar to things i've said too, but clearly and succinctly put!
I see a similar problem with the common preference for unit tests over integration tests. The mistakes in a system tend to be in the interactions between parts, not in the parts themselves, and by construction, unit tests can't test those.
I see a similar problem with the common preference for unit tests over integration tests. The mistakes in a system tend to be in the interactions between parts, not in the parts themselves, and by construction, unit tests can't test those.
If this were printed on a wall somewhere at my first job it probably would have accelerated my career by a decade or so.
As the sibling posts say, that is a great quote and insight. I'm curious if you have more exploring the idea?
Yeah. Same here. I worked with OSGi long before I used REST and microservices. To claim that OSGi somehow pioneered this stuff is both arrogant and ignorant. Sure OSGi and microservices share some similar concepts but they are both implementing ideas commonly found in distributed systems. The author just comes across as bitter. I liked OSGi when I first encountered it but for some of my colleagues it was a lot of overhead to do the simplest things. Either they had to learn it all conceptually or they have to rely on a lot of magic from the tooling. I feel that to really understand OSGi you had to know a bit about the nuts and bolts of Java and its internals. This is not common especially even among Java programmers. The advantage of REST microservices is that most developers understand REST and HTTP, etc. People have a much easier time getting REST and microservices (although you really need to know distributed systems to not get yourself into trouble with some boundary cases and failure modes).
> Some of the most horrifying systems I have worked with were based on the idea that you should be able to swap implementations during runtime.
And it's usually such an unnecessary feature!
There's nothing that makes me fear for a project's future more than when they get to the 'we'll make everything a plugin' phase.
And it's usually such an unnecessary feature!
There's nothing that makes me fear for a project's future more than when they get to the 'we'll make everything a plugin' phase.
Yeah, It implies bad things about the expected performance of the JVM or the quality of the application being run. Either startup time or the time taken for the JIT to get a handle on things is apocalyptically bad.
A bit? Like there weren't similar systems years before osgi.
Frankly, the fact that osgi solved an important problem and people still don't use it indicates a failure, not a success.
Frankly, the fact that osgi solved an important problem and people still don't use it indicates a failure, not a success.
I love OSGi, used mostly Eclipse Equinox a lot in my previous job. Now I come more across Apache Felix in my work. I can imagine people not liking it, because it can be hard to learn. I used it mostly because of dependency hell and for encapsulation of reusable components. Controlling which parts of the component is exposed, is a very powerful concept.
About reliability, I can only say that it is very reliable. It has to be, since it was developed to continue running, even if one of the loaded plugins crashes. Imagine running plugins for the ABS, entertainment and navigation system and the navigation system crashes. With OSGi the navigation system can be restarted without impacting other more crucial components.
I hear sometimes people saying, Java modules are a replacement for OSGi. This sounds to me a lot like saying that Blockchain can be used as distributed storage system.
About reliability, I can only say that it is very reliable. It has to be, since it was developed to continue running, even if one of the loaded plugins crashes. Imagine running plugins for the ABS, entertainment and navigation system and the navigation system crashes. With OSGi the navigation system can be restarted without impacting other more crucial components.
I hear sometimes people saying, Java modules are a replacement for OSGi. This sounds to me a lot like saying that Blockchain can be used as distributed storage system.
> Imagine running plugins for the ABS, entertainment and navigation system and the navigation system crashes. With OSGi the navigation system can be restarted without impacting other more crucial components.
This is exactly the Erlang 'let it crash' philosophy. Supervisor 'processes' (message-passing green threads) run concurrently and monitor and restart each other as necessary.
The only difference is that this is not at the plugin level but at the process level, a core lightweight concurrency primitive provided by the Erlang runtime.
This is exactly the Erlang 'let it crash' philosophy. Supervisor 'processes' (message-passing green threads) run concurrently and monitor and restart each other as necessary.
The only difference is that this is not at the plugin level but at the process level, a core lightweight concurrency primitive provided by the Erlang runtime.
Interestingly enough, this is exactly how a car actually works. All systems have to send a heartbeat over the CAN bus, if not they are restarted.
If anyone is interested in how a real product is using OSGi have a look at Liferay [1].
It uses declarative services [2] with annotations to define components and their dependencies. I am not very familiar with the underlaying tooling but the end result is quite simple (see [3] for a simple component with 3 dependencies).
Disclaimer: I work at Liferay.
[1] https://github.com/liferay/liferay-portal
[2] https://osgi.org/specification/osgi.cmpn/7.0.0/service.compo...
[3] https://github.com/liferay/liferay-portal/blob/5c29b7173777b...
It uses declarative services [2] with annotations to define components and their dependencies. I am not very familiar with the underlaying tooling but the end result is quite simple (see [3] for a simple component with 3 dependencies).
Disclaimer: I work at Liferay.
[1] https://github.com/liferay/liferay-portal
[2] https://osgi.org/specification/osgi.cmpn/7.0.0/service.compo...
[3] https://github.com/liferay/liferay-portal/blob/5c29b7173777b...
I believe that Liferay uses another bit of mirror-universe saucer tech, the Content Repository for Java API:
https://jcp.org/en/jsr/detail?id=283
This is essentially a hierarchical database - think something like an XML document, a hierarchy of nodes of particular types with attributes according to their type - but has a few really eye-opening features, like versioning, and branching and merging of versions. I'm not aware of any other kind of database (apart from VCSes!) that does that.
https://jcp.org/en/jsr/detail?id=283
This is essentially a hierarchical database - think something like an XML document, a hierarchy of nodes of particular types with attributes according to their type - but has a few really eye-opening features, like versioning, and branching and merging of versions. I'm not aware of any other kind of database (apart from VCSes!) that does that.
There’s a hidden world of Enterprise CMS systems, all built on JCR. Liferay, Adobe, Jahia etc... as you said, it’s like stepping into a parallel universe
I used Apache Jackrabbit (a JCR) at a previous job to store PDFs of insurance rate manuals. We could tag them by province, line of business, whatever, and then query it quickly. Jackrabbit is a fantastic piece of software.
And Adobe’s AEM lives on top of Jackrabbit Oak, their next-gen JCR. For a lot of purposes, Jackrabbit Oak makes a pretty good general-purpose headless CMS in its own right (assuming you’re good with the JCR pragmatics).
The article makes OSGi look like the inventor of encapsulation and interfaces, but those concepts existed way before OSGi (from Smalltalk to frameworks like CORBA).
What OSGi produced is a model to handle dependencies, service lifecycle, and provisioning. But in the Java ecosystem, those features were replaced with a mix and match of smaller projects, that are easier to adopt and use.
Instead of adapting to the community, OSGi frameworks like Eclipse Equinox live on their own island.
Take for example provisioning: Equinox has its own framework called P2. You don't need P2 to use OSGi. But, it's the way of doing things in Equinox. So to share a library, you need to provide packages using a P2 update site. There is no easy way of using Maven Repos [1]. The end result: tons of unnecessary complexity, and more isolation of projects that relies on Equinox tooling [2].
[1]: Tycho is the solution for interacting with Maven repos; but is not easy to setup...and in the end all the trouble doesn't pay: you end with a mix of dependency definitions that is hard to maintain/troubleshoot.
[2]: I'm looking at you SWT... you promise a nice native UIs, but consuming SWTs JARs from a non-Equinox project is a pain... thank you P2 and OSGi! :)
What OSGi produced is a model to handle dependencies, service lifecycle, and provisioning. But in the Java ecosystem, those features were replaced with a mix and match of smaller projects, that are easier to adopt and use.
Instead of adapting to the community, OSGi frameworks like Eclipse Equinox live on their own island.
Take for example provisioning: Equinox has its own framework called P2. You don't need P2 to use OSGi. But, it's the way of doing things in Equinox. So to share a library, you need to provide packages using a P2 update site. There is no easy way of using Maven Repos [1]. The end result: tons of unnecessary complexity, and more isolation of projects that relies on Equinox tooling [2].
[1]: Tycho is the solution for interacting with Maven repos; but is not easy to setup...and in the end all the trouble doesn't pay: you end with a mix of dependency definitions that is hard to maintain/troubleshoot.
[2]: I'm looking at you SWT... you promise a nice native UIs, but consuming SWTs JARs from a non-Equinox project is a pain... thank you P2 and OSGi! :)
> The reason why microservices work so well is it provides a well-defined API entry point into a module. The caller ... can ignore the messy details of how that API is implemented
Honestly, everyone just re-interprets microservices as whatever they feel like! This particular quote is just describing modularity and encapsulation, something that Parnas could have told them in the '70s.[1]
Microservices are about decoupling teams from each other so they can work and deploy on their own schedule. To a lesser extent, they're about scaling different parts of the system independently (on stacks that don't natively support that with a sane concurrency model). That's really just it. Everything else is hype and buzzwords.
[1] https://blog.acolyer.org/2016/09/05/on-the-criteria-to-be-us...
Honestly, everyone just re-interprets microservices as whatever they feel like! This particular quote is just describing modularity and encapsulation, something that Parnas could have told them in the '70s.[1]
Microservices are about decoupling teams from each other so they can work and deploy on their own schedule. To a lesser extent, they're about scaling different parts of the system independently (on stacks that don't natively support that with a sane concurrency model). That's really just it. Everything else is hype and buzzwords.
[1] https://blog.acolyer.org/2016/09/05/on-the-criteria-to-be-us...
Why can't you decouple teams form each other with modularity and encapsulation? I never worked in a team, but I could imagine that teams just have some agreements about interfaces and then work independently on a single code base but in a different classes and packages.
Because modules in the same codebase have an unfortunate tendency to cross each others' boundaries. It's just too easy for them to reach into each others' implementation classes and couple together strongly. But if they're different codebases, there's a stronger wall between them.
(Micro)services and separate codebases are orthogonal.
You can have multiple separate components (libraries) implemented in different codebases and linked as a single executable (which is actually really common).
You can have (micro)services deployed from as single huge codebase - just take a look at all so-called monorepos.
You can have (micro)services deployed from as single huge codebase - just take a look at all so-called monorepos.
Frankly, their biggest accomplishment is operating in several spaces where I've come across the abbreviation without me ever having to know what it is or why it needs to exist (which seems to indicate… it doesn't?)
OSGi is what killed Eclipse. Instead of focusing on producing a fast, modern IDE, Eclipse became a design-by-committee monster that had to support just about any use case on the client.
We had to use OSGi in a student project for a distributed systems module.
It was a nightmare. We were basically forced to use Eclipse (instead of IntelliJ which everybody was accustomed to). Eclipse alone was a nightmare to use (slow, imports not founds, etc).
On the OSGi side:
There is no package manager. One team tried to set up maven but they failed. We just loaded jars.
The manifest files were a mess. Eclipse launch configurations were a mess (had to be re-imported after any change).
A small project which only required a backend was needlessly split into 5 parts. Over-engineering at its finest.
It was a nightmare. We were basically forced to use Eclipse (instead of IntelliJ which everybody was accustomed to). Eclipse alone was a nightmare to use (slow, imports not founds, etc).
On the OSGi side:
There is no package manager. One team tried to set up maven but they failed. We just loaded jars.
The manifest files were a mess. Eclipse launch configurations were a mess (had to be re-imported after any change).
A small project which only required a backend was needlessly split into 5 parts. Over-engineering at its finest.
I am looking into classloader isolation solution and it seems OSGi, especially the Classloader specification is wonderful. The whole OSGi spec is a 400+page book if you are into services, security, etc... but the Classloading isolation spec is small and very useful.
I am in need of a way to allow plugins/integration with other teams. If we use standard Java package sharing mechanism, sooner or later teams will run into dependency conflict hell. More often, microservices (IPC) is suggested, too. But, the nature of our application is sensitive to latency and require high throughput. That throws IPC out of the picture.
Then, I remember the servlet API and how the servlet spec dictates classloading isolation at the container level. I rolled my own classloader to isolate the plugins/integrations but it is not as nice as OSGi implementation.
At this point, I am very inclined into using OSGi just for the sole classloading purpose.
One question that bother me is: since we are not that hardcore OSGi users, would it make sense for us to just bundle every plugins' into a fat jar and use bundle-classpath to resolve the plugin's owned dependencies? I know that by doing so, we are not strictly adhere to OSGi classes' sharing but it would lower the friction for independent teams to integrate with us.
I am in need of a way to allow plugins/integration with other teams. If we use standard Java package sharing mechanism, sooner or later teams will run into dependency conflict hell. More often, microservices (IPC) is suggested, too. But, the nature of our application is sensitive to latency and require high throughput. That throws IPC out of the picture.
Then, I remember the servlet API and how the servlet spec dictates classloading isolation at the container level. I rolled my own classloader to isolate the plugins/integrations but it is not as nice as OSGi implementation.
At this point, I am very inclined into using OSGi just for the sole classloading purpose.
One question that bother me is: since we are not that hardcore OSGi users, would it make sense for us to just bundle every plugins' into a fat jar and use bundle-classpath to resolve the plugin's owned dependencies? I know that by doing so, we are not strictly adhere to OSGi classes' sharing but it would lower the friction for independent teams to integrate with us.
In my opinion the main issue with OSGi class loading is that the spec couples three separate concerns:
peer class loading (as opposed to hierarchical)
dependency resolution
dependency specification and module (bundle) packaging
This makes OSGi an all or nothing solution: to use OSGi class loading you are forced to package your code in a specific way - which in turn makes it difficult to reuse existing libraries.
Lack of peer class loading has been a long standing pain point in Java - and in my opinion one of the main obstacles for really interesting use cases - especially related to mobile code ( see Jini ).
peer class loading (as opposed to hierarchical)
dependency resolution
dependency specification and module (bundle) packaging
This makes OSGi an all or nothing solution: to use OSGi class loading you are forced to package your code in a specific way - which in turn makes it difficult to reuse existing libraries.
Lack of peer class loading has been a long standing pain point in Java - and in my opinion one of the main obstacles for really interesting use cases - especially related to mobile code ( see Jini ).
You can take a look at my tiny web-server for the simplest hot-deployment implementation I have made/found so far: https://github.com/tinspin/rupy/blob/master/src/se/rupy/http...
You can easily get the good parts of OSGi, without its cruft and complexity: https://github.com/7mind/slides/raw/master/02-roles/target/r... (by advanced DI with garbage collection)
Now, OSGi may have a poor image in the current Microservices/cloud vendor driven bubble. But let’s step a bit back and ask: 1. isn’t constructing software from building-blocks useful? What’s up with reuse, synergy, not throw away software?
2. Wouldn’t services be useful that don’t just die but reconfigure (adapt) when upstream dependencies change/go away?
3. Is time to market really the best incentive these days? What’s up with making sure software is constructed with security and privacy in mind? What about assuming your startup isn’t sold (sell and forget) but you need to evolve it?
So, incidentally those qualities are part of the OSGi values made up about 20 years ago. If not OSGi (a messenger of values), wouldn’t the values make more sense now more than ever?
What about „green“ software development?
/ Toni