liquibase-karaf-feature

A Karaf feature for liquibase-core

NOTE! Version 4.15.0 is the first release of the liquibase karaf feature in nearly three years See Using liquibase from OSGi bundles if you come from version 3.8.0 of this feature, for changes that need to be made.

This project contains a karaf feature for easily using Liquibase from OSGi-based applications running in apache karaf.

If you haven't yet encountered it: liquibase is a really smooth solution for handling your RDBMS schemas. Smooth initial startup, and smooth evolution of schemas (adding columns, adding tables, dropping columns and dropping tables).

Liquibase does the same job as ad-hoc delta script solutions, but liquibase does the job in a clean and robust way, tested and refined over the 11 years of its existence.

Liquibase does pretty much the same thing as flyway but in a different way that fits my programmer's mind better. And liquibase is cross-database capable, i.e. done right it's possible to write schema migrations in ways that will make them work on all databases with a JDBC driver.

Project status

file:https://maven-badges.herokuapp.com/maven-central/no.priv.bang.karaf/liquibase-core-karaf/badge.svg

Release history

Installing the liquibase feature in karaf

Date	Version	Liquibase version	Liquibase slf4j version	Comment
<2022-10-30 Sun 15:48>	4.17.1	4.15.1
<2022-08-20 Sat 19:27>	4.15.0	4.15.0		First liquibase 4.x release of the feature
<2019-11-18 Mon 21:25>	3.8.0	3.8.0	2.0.0
<2019-11-18 Mon 20:42>	3.7.0	3.7.0	2.0.0	Use snakeyaml 1.23
<2019-11-18 Mon 19:33>	3.6.3	3.6.3	2.0.0
<2019-11-17 Sun 22:58>	3.6.2	3.6.2	2.0.0
<2019-11-17 Sun 22:09>	3.6.1.1	3.6.1	2.0.0	Loads snakeyaml 1.18 instead of 1.17
<2019-11-17 Sun 17:35>	3.6.1	3.6.1	2.0.0	Broken because of wrong snakeyaml version
<2019-11-17 Sun 21:27>	3.6.0.1	3.6.0	2.0.0	Loads snakeyaml 1.18 instead of 1.17
<2019-11-17 Sun 16:01>	3.6.0	3.6.0	2.0.0	Broken because of wrong snakeyaml version
<2019-11-16 Sat 23:09>	3.5.5	3.5.5	2.0.0	Use version 3.5.1 of maven-bundle-plugin
<2019-11-16 Sat 11:28>	3.5.4	3.5.4	2.0.0	Updated pom.xml release config, update karaf to 4.2.7
<2017-08-06 Sun 18:48>	3.5.3	3.5.3	2.0.0	First release with the same version as the liquibase version
<2017-08-06 Sun 15:18>	1.0.2	3.5.3	2.0.0	First successful release
<2017-08-06 Sun 12:03>	1.0.1	3.5.3	2.0.0	Failed release
<2017-08-05 Sat 21:37>	1.0.0	3.5.3	2.0.0	Failed release

To install this feature:
1. start karaf and give the following commands to the karaf console:
#+BEGIN_EXAMPLE feature:repo-add mvn:no.priv.bang.karaf/liquibase-core-karaf/LATEST/xml/features feature:install liquibase-core #+END_EXAMPLE

After this, the liquibase Java API is available to your OSGi applications and the liquibase logging will go to the karaf log.

Using liquibase from a karaf feature

To use liquibase from your own, manually edited, karaf feature, include the feature's feature repository and depend on the liquibase-core feature:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<features xmlns="http://karaf.apache.org/xmlns/features/v1.4.0" name="ukelonn.bundle.db.liquibase">
    <repository>mvn:no.priv.bang.karaf/liquibase-core-karaf/4.17.1/xml/features</repository>
    <feature name="ukelonn-db-liquibase">
        <feature>liquibase-core</feature>
    </feature>
</features>

Using liquibase from a generated karaf feature

If you generate your karaf feature repository using the karaf-maven-plugin, you can include the liquibase-core feature into your generated feature repository, by adding this dependency to the maven project building the feature repository:

  <dependency>
      <groupId>no.priv.bang.karaf</groupId>
      <artifactId>liquibase-core-karaf</artifactId>
      <version>4.17.1</version>
      <type>xml</type>
      <classifier>features</classifier>
  </dependency>

Building the feature for a different version of Liquibase

The version number of this karaf feature is intended to be the same as the Liquibase version it is a feature for.

This makes it simple for me to roll and release a new version of the feature when a new version of Liquibase is out.

But unfortunately this means that a SNAPSHOT version of the feature won't be able to refer to a real Liquibase version... at least not without a little edit:
1. clone this project:
#+BEGIN_EXAMPLE mkdir -p ~/git cd ~/git git clone https://github.com/steinarb/liquibase-karaf-feature/ #+END_EXAMPLE
2. edit the pom, changing the liquibase.version property
#+BEGIN_SRC xml ${project.version} #+END_SRC change it into an actual version #+BEGIN_SRC xml 3.5.4 #+END_SRC
3. then build the project with maven:
#+BEGIN_EXAMPLE cd liquibase-karaf-feature mvn clean install #+END_EXAMPLE

Test a new version

I have created the project liquibase-sample to test new versions of this karaf feature.

The liquibase-sample has a minimal OSGi component that loads and creates a schema in a derby in-memory database from a liquibase changelog file.

The liquibase-sample application can also be used to verify that the liquibase logs are redirected to the karaf logs.

Using liquibase from OSGi bundles

Liquibase 4 is built internally with an inversion-of-control architecture, and uses java.util.ServiceLoader to find the implementations of its services.

The ServiceLoader doesn't work well with OSGi. The ServiceLoader expects a single, flat, classloader, and a single thread, and this is not what OSGi has.

It is possible to make the ServiceLoader work in OSGi, using the Service Loader Mediator.

A single implementation of the service loader mediator exists: Apache Aries SPI Fly.

The liquibase karaf feature created from this project will load SPI Fly at the same start-level as the liquibase-core bundle.

But to make liquibase usage work, the bundles that instanciates the Liquibase classes and loads and parses the changelog files, needs to do two things:
1. Add a Require-Capabilities header that requires all capabilities provided by the liquibase-core bundle (this must be done for all bundles that need to instantiate Liquibase classes)
2. Extract the XSD file(s) for the changelog schema version(s) used and add them to the bundle doing the parsing (this must be done for all bundles XML files are loaded from)

org.apache.felix maven-bundle-plugin 5.1.8 osgi.extender; filter:="(osgi.extender=osgi.serviceloader.processor)", osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.serializer.ChangeLogSerializer)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.parser.NamespaceDetails)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.database.Database)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.change.Change)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.database.DatabaseConnection)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.precondition.Precondition)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.serializer.SnapshotSerializer)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.configuration.AutoloadedConfigurations)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.diff.DiffGenerator)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.lockservice.LockService)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.changelog.ChangeLogHistoryService)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.datatype.LiquibaseDataType)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.configuration.ConfigurationValueProvider)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.logging.LogService)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.snapshot.SnapshotGenerator)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.parser.ChangeLogParser)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.servicelocator.ServiceLocator)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.diff.compare.DatabaseObjectComparator)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.command.LiquibaseCommand)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.license.LicenseService)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.diff.output.changelog.ChangeGenerator)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.executor.Executor)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.structure.DatabaseObject)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.parser.SnapshotParser)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.hub.HubService)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.command.CommandStep)"; cardinality:=multiple, osgi.serviceloader; filter:="(osgi.serviceloader=liquibase.sqlgenerator.SqlGenerator)"; cardinality:=multiple =target/classes, /www.liquibase.org/=target/dependency/www.liquibase.org/ org.apache.maven.plugins maven-dependency-plugin copy-liquibase-xsd validate unpack org.liquibase liquibase-core **/dbchangelog-3.5.xsd

In the above example only dbchangelog-3.5 is copied. If a different schema version is used, that version must be copied instead.

To copy all schemas, change includes to this (Disclaimer: not tested):

*/.xsd

Current problems under OSGi

License

Apart from the issues worked around in the previous section I see two problems currently:
1. I think JSON parsers and YAML parsers may not work in the current (4.17.1) version, based on these messages in karaf.log
#+begin_example 2022-10-30T11:09:18,242 | INFO | features-3-thread-1 | servicelocator | 167 - org.liquibase.core - 4.17.1 | Cannot load service: liquibase.parser.ChangeLogParser: liquibase.parser.core.json.JsonChangeLogParser Unable to get public no-arg constructor 2022-10-30T11:09:18,249 | INFO | features-3-thread-1 | servicelocator | 167 - org.liquibase.core - 4.17.1 | Cannot load service: liquibase.parser.ChangeLogParser: liquibase.parser.core.yaml.YamlChangeLogParser Unable to get public no-arg constructor 2022-10-30T11:09:18,344 | INFO | features-3-thread-1 | servicelocator | 167 - org.liquibase.core - 4.17.1 | Cannot load service: liquibase.change.Change: liquibase.change.core.LoadDataChange Unable to get public no-arg constructor 2022-10-30T11:09:18,346 | INFO | features-3-thread-1 | servicelocator | 167 - org.liquibase.core - 4.17.1 | Cannot load service: liquibase.change.Change: liquibase.change.core.LoadUpdateDataChange Unable to get public no-arg constructor #+end_example
2. I sometimes see messages like the one below in the karaf.log. I think the problems causing the messages may be caused by multiple threads interfering with each other and may be this liquibase issue: https://github.com/liquibase/liquibase/issues/2248
#+begin_example 2022-09-10T13:47:54,302 | ERROR | CM Configuration Updater (ManagedServiceFactory Update: factoryPid=[org.ops4j.datasource]) | HandleregProductionDbLiquibaseRunner | 125 - no.priv.bang.handlereg.db.liquibase.production - 1.0.0.SNAPSHOT | Failed to create handlereg derby test database liquibase.exception.LiquibaseException: java.lang.RuntimeException: Cannot end scope cpkebkpkfa when currently at scope bbldyrztji at liquibase.Liquibase.runInScope(Liquibase.java:2419) ~[?:?] at liquibase.Liquibase.update(Liquibase.java:209) ~[?:?] at liquibase.Liquibase.update(Liquibase.java:195) ~[?:?] at liquibase.Liquibase.update(Liquibase.java:191) ~[?:?] at liquibase.Liquibase.update(Liquibase.java:183) ~[?:?] at no.priv.bang.handlereg.db.liquibase.HandleregLiquibase.applyLiquibaseChangelist(HandleregLiquibase.java:42) ~[?:?] at no.priv.bang.handlereg.db.liquibase.HandleregLiquibase.createInitialSchema(HandleregLiquibase.java:28) ~[?:?] at no.priv.bang.handlereg.db.liquibase.production.HandleregProductionDbLiquibaseRunner.prepare(HandleregProductionDbLiquibaseRunner.java:55) ~[?:?] at org.ops4j.pax.jdbc.config.impl.DataSourceRegistration.(DataSourceRegistration.java:88) ~[?:?] at org.ops4j.pax.jdbc.config.impl.DataSourceConfigManager.lambda$null$4(DataSourceConfigManager.java:95) ~[?:?] at org.ops4j.pax.jdbc.config.impl.ServiceTrackerHelper$1.addingService(ServiceTrackerHelper.java:132) ~[?:?] at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:943) ~[osgi.core-8.0.0.jar:?] at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:871) ~[osgi.core-8.0.0.jar:?] at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256) ~[osgi.core-8.0.0.jar:?] at org.osgi.util.tracker.AbstractTracked.trackInitial(AbstractTracked.java:183) ~[osgi.core-8.0.0.jar:?] at org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:321) ~[osgi.core-8.0.0.jar:?] at org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:264) ~[osgi.core-8.0.0.jar:?] at org.ops4j.pax.jdbc.config.impl.ServiceTrackerHelper.track(ServiceTrackerHelper.java:141) ~[?:?] at org.ops4j.pax.jdbc.config.impl.DataSourceConfigManager.lambda$null$5(DataSourceConfigManager.java:91) ~[?:?] at org.ops4j.pax.jdbc.config.impl.ServiceTrackerHelper$1.addingService(ServiceTrackerHelper.java:132) ~[?:?] at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:943) ~[osgi.core-8.0.0.jar:?] at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:871) ~[osgi.core-8.0.0.jar:?] at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256) ~[osgi.core-8.0.0.jar:?] at org.osgi.util.tracker.AbstractTracked.trackInitial(AbstractTracked.java:183) ~[osgi.core-8.0.0.jar:?] at org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:321) ~[osgi.core-8.0.0.jar:?] at org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:264) ~[osgi.core-8.0.0.jar:?] at org.ops4j.pax.jdbc.config.impl.ServiceTrackerHelper.track(ServiceTrackerHelper.java:141) ~[?:?] at org.ops4j.pax.jdbc.config.impl.ServiceTrackerHelper.track(ServiceTrackerHelper.java:86) ~[?:?] at org.ops4j.pax.jdbc.config.impl.DataSourceConfigManager.lambda$updated$6(DataSourceConfigManager.java:90) ~[?:?] at org.ops4j.pax.jdbc.config.impl.ServiceTrackerHelper.track(ServiceTrackerHelper.java:147) ~[?:?] at org.ops4j.pax.jdbc.config.impl.ServiceTrackerHelper.track(ServiceTrackerHelper.java:86) ~[?:?] at org.ops4j.pax.jdbc.config.impl.DataSourceConfigManager.updated(DataSourceConfigManager.java:89) ~[?:?] at org.apache.felix.cm.impl.helper.ManagedServiceFactoryTracker.updated(ManagedServiceFactoryTracker.java:159) ~[?:?] at org.apache.felix.cm.impl.helper.ManagedServiceFactoryTracker.provideConfiguration(ManagedServiceFactoryTracker.java:93) ~[?:?] at org.apache.felix.cm.impl.ConfigurationManager$ManagedServiceFactoryUpdate.provide(ConfigurationManager.java:1264) ~[?:?] at org.apache.felix.cm.impl.ConfigurationManager$ManagedServiceFactoryUpdate.run(ConfigurationManager.java:1208) ~[?:?] at org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:122) ~[?:?] at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:84) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.lang.RuntimeException: Cannot end scope cpkebkpkfa when currently at scope bbldyrztji at liquibase.Scope.exit(Scope.java:230) ~[?:?] at liquibase.Scope.child(Scope.java:191) ~[?:?] at liquibase.Scope.child(Scope.java:179) ~[?:?] at liquibase.Scope.child(Scope.java:158) ~[?:?] at liquibase.Liquibase.runInScope(Liquibase.java:2414) ~[?:?] ... 38 more #+end_example

This maven project is licensed with the Apache v 2.0 license.

The details of the license can be found in the LICENSE file.

The liquibase-slf4j jar is covered with the MIT license, copyright 2012-2015 Matt Bertolini. This license and copyright also covers the rebundled version of the jar that results from the "com.mattbertolini.liquibase-slf4j-osgi" maven module.