Modelstore: a typless modeling framework
What is this?
This is a small Java library intended to hold a model consisting of objects and relations in memory.
The main use case for this library is to be used as a generic model container for an Eclipse GEF editor. Because of this, and because it does no harm, and because I kind of like OSGi, this library is packaged as an OSGi bundle.
The API is meant to be as clutter-free as possbile in use: it should throw no exceptions, and it should return no nulls (instead of null it will return the empty string, empty collections and zero values for the numeric types).
The PropertySet interface defines the API of objects that forms both objects and relationships of the model.
Development stuff
This library is licensed under the Eclipse Public License. See the
file LICENSE for the terms and specifics of the license.
file:https://github.com/steinarb/modelstore/actions/workflows/modelstore-maven-ci-build.yml/badge.svg
file:https://coveralls.io/repos/steinarb/modelstore/badge.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=alert_status#.svg
file:https://sonarcloud.io/images/project_badges/sonarcloud-white.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=sqale_index#.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=coverage#.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=ncloc#.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=code_smells#.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=sqale_rating#.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=security_rating#.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=bugs#.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=vulnerabilities#.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=duplicated_lines_density#.svg
file:https://sonarcloud.io/api/project_badges/measure?project=steinarb_modelstore&metric=reliability_rating#.svg
Installing on karaf
The modelstore can't be used for anything, yet. But it can be installed on apache karaf:
- Clone and build the project (requires git, java >= 1.8 and maven installed):
#+BEGIN_EXAMPLE
git clone https://github.com/steinarb/modelstore.git
cd modelstore
mvn clean install
#+END_EXAMPLE
- Install apache karaf using the karaf quick start
- In the karaf console give the following commands:
#+BEGIN_EXAMPLE
feature:repo-add mvn:no.priv.bang.modeling.modelstore/modelstore.backend/LATEST/xml/features
feature:install modelstore.backend
#+END_EXAMPLE
Roadmap [18/55]
DONE Fix the test coverage of all property values (hashCode, equals and toString ruined coverage)
DONE Put javadoc from travis builds on the web
TODO Add javadocs to everything
DONE Rename "propertyvalue" to "value" in type names
DONE Refactor the propertyvalue creation into something that makes sense for a library bundle
DONE Create typespecific add and set operations to the ValueList interface
DONE Create primitive-value setters for Propertyset
DONE Change "propertyName" to "propertyname" in arguments
DONE Add clone or copy constructor operation to Propertyset
DONE Add clone or copy constructor to ValueList
DONE Add copy-on-set for complex properties and list values
DONE Add list creation method on the PropertysetManager interface, to make it possible to create ValueList instances outside of the bundle
DONE Put some basic aspects in place (object, relationship, model, aspectcontainer)
DONE Split off ModelContext [5/5]
DONE Persist and instantiation should use PropertysetContext
DONE Add methods to the PropertysetManager to store and receive PropertysetContexts
DONE Create a merge operation that will merge all aspects and propertysets
DONE Rename the PropertysetManager interface to Modelstore
DONE Rename PropertysetContext to ModelContext
DONE Store exception errors in the Modelstore
DONE Add UUIDs of built-in aspects to Modelstore
DONE Switch from Jsr330Activator to OSGi Declarative Service (DS)
TODO Replace logging to ErrorBean with the OSGi LogService
TODO Separate the modelstore implementation holding model in memory into an OSGi library bundle (maybe)
TODO create an OSGi bundle modelstore.client
TODO Create modelstore.db.liquibase bundle to define the JDBC schema
TODO use the Modelstore DatabaseService in modelstore.backend
TODO create a modelstore.web.security OSGi bundle (connect with shiro and authservice)
TODO create a modelstore.web.api OSGi bundle providing a REST API
TODO Create a modelstore-specific DatabaseService interface in modelstore.services
TODO Create modelstore.db.derbytest OSGi bundle
TODO Create modelstore.db.postgresql OSGi bundle
TODO Connect a minimal hardcoded model to eclipse GEF
TODO Implement JSON storage for eclipse GEF models
TODO Split ModelContext objects
TODO Order propertysets by dependency when serializing
TODO Introduce a DateTime primitive type in value
TODO Add verification code for aspects
TODO Add AspectViwer (connected to aspect container and used as a filter)
TODO Create a read-only propertyset wrapper with defensive copy-on-read for complex properties and lists
TODO Create a proxy aspect
TODO Test serialization/deserialization using YAML (YAML has object id and object reference)
TODO Storage based on SQL for relationships and references and individual JSON files
TODO Individual Propertyset files git versioned
TODO Storage based on PostgreSQL with native JSON support
TODO Get PropertysetManager with storage running in Karaf
TODO Move interface definitions to a separate bundle
TODO Move Jackson serialization to a separate bundle (maybe a library bundle?)
TODO Create a RESTful API and a storage/persist mechanism on top of it
TODO Create a query language (or find something usable and implement/use)
TODO Make an s-expression-factory for jackson
TODO Make merge operation thread safe
TODO Add propertyvalue creation methods on the ModelStore interface, to make them accessible to the world
TODO Decide if the PropertysetRecordingSaveTime should compare equal to a PropertysetRecordingSaveTime from a different ModelContext
TODO Switch to defensive copy on read for list and complex properties (have to think about this)
TODO Rename Propertyset to Valueset
TODO Wrap the propertysets and aspects returned from the metadata-setting ModelContext
Having "property" in the name doesn't make sense for the list of values, and make the type names longer than they have to be.
- Maybe a static method on the PropertyValue interface, if that's possible?
- Or make that creation methods on the interface, backed by a class with static methods?
- Primitives and lists are immutable and can be shared
- Lists and complex properties should not be shared
- For complex properties this can be accomplished with a clone operation
- For list properties, probably a clone here as well
- Modelstore will be the access point for creating and saving ModelContext instances
- This is an approach at making the system multithreaded and performant
- Minimal locking on the propertysets themselves, because there is only on thread using them at a time
- No need to copy the property values, because they are immutable (except for complexproperty and listproperty, that is...)
- Use shallow copy on list and complex object property get, perhaps?
- Merge will not touch the id property so merging with an empty object with a different id will be to effectively make a copy with a different id
- This may be useful
- The parsing and file/stream operations give a lot of possible error situation that right now go untracked
- Add a logError method to the ModelContext
- Let the ModelContext pass the error to the Modelstore
- Create an ErrorBean with getters only and a constructor initializing the fields:
- Date when the error occurred
- ModelContext where the error occurred
- Errormessage
- Exception caught
- Create an interface with methods
- reportError(String message, Exception e)
- boolean hasErrors()
- Collection listErrors()
- Let the ModelContext and Modelstore interfaces inherit this interface
- Should be thread safe with a minimal locked critical region
- Wrap the error list in a synchronized list
- Synchronize on the list before doing a shallow copy in getErrors()
- Create an interface with the getters for these IDs (a "protocol")
- Let Modelstore inherit this interface
- Try the following implementation: create a class implementing this interface and let ModelstoreBase inherit it, as well as implementing the Modelstore
- Can use the same approach for value creation if of interest
- <2019-08-12 man. 11:54> The single jar was split into modelstore.services defining the OSGi services and a modelstore.backend containing the DS component
- <2019-08-12 man. 11:56> The gogoshell stuff was deleted and karaf features were created instead
- <2019-08-12 man. 20:54> modelstore.backend is to become a DS component that initially saves to and restores from disk
- <2019-08-12 man. 20:56> need a good name for the model-in-memory library before I can create the model
- <2019-08-12 man. 21:02> The serialization/deserialization code doesn't need to be part of this library
- <2019-08-12 man. 21:03> modelstore.model is probably a good name for the library,
- <2019-08-12 man. 21:20> classes that should be migrated to modelstore.model, are:
- Aspects
- BooleanValue
- BuiltinAspectsBase
- ComplexValue
- DoubleValue
- EmptyValue
- EmptyValueList
- IdValue
- ListValue
- LongValue
- NilValue
- PropertysetImpl
- PropertysetNil
- Propertysets
- PropertysetValueBase
- ReferenceValue
- StringValue
- ValueArrayList
- ValueBase
- Values
- <2019-08-12 man. 21:21> classes that should not be migrated to modelstore.model, are:
- JsonGeneratorWithReferences
- JsonPropertysetPersister
- ModelstoreProvider (this is the DS component)
- <2019-08-12 man. 21:24> classes I'm unsure of should be migrated to modelstore.model, are:
- ModelContextImpl
- ModelContextRecordingMetadata
- ModelContexts
- ModelstoreBase
- PropertysetRecordingSaveTime
- <2019-08-12 man. 21:38> Looks like not all classes in modelstore.model should be visible
- <2019-08-12 man. 21:39> A static creator class and/or singleton is not a good pattern for OSGi: then it's better to create a DS component
- <2019-08-12 man. 21:49> What should the inteface exposed by the DS component be called?
- <2019-08-12 man. 21:50> Some name candidates for the interface:
- Model (probably wrong. Model should be a parent object containing other objects)
- ModelFactory (more correct, but suffixing with "Factory" is overused, and suffixing is bad practice anyway)
- ObjectFactory (most correct. However, maybe too "overused"...?)
- ModelBuilder (sounds good, but might make people expect the builder pattern...?)
- ModelProducer
- ObjectProducer
- <2023-04-03 Mon 13:37> Current plan:
- First create a ValueCreator interface in modelstore.services, that contains all the methods in the static class Values
- Then create a new bundle modelstore.values that contains a DS component that implements ValueCreator
- Then (maybe) create an Aspects interface (not sure what should go into it?)
- Then (again: maybe) create a new bundle modelstore.aspects that contains a DS component for Aspects
- Then create a Models interface for creating and composing Models in modelstore.services
- Then create a bundle modelstore.models containing a DS component that provides a DS component for Models and requires the ValueCreator service
- <2019-08-12 man. 21:27> This is a to be a convenient starting point for using modelstore
- <2019-08-12 man. 21:30> Design:
- Create an interface in modelstore.services called ModelstoreClient (maybe just a subtype of Modelstore?)
- in modelstore.client create a DS component that receives a Modelstore service and exposes a ModelstoreClient service
- This allows for having an in-process modelstore or a modelstore accessed through a REST API
- For an in-process modelstore this should be a thin wrapper
- For a remote modelstore this library should maintain the in-memory model
- <2019-08-12 man. 22:14> The objects themselves should be stored to disk and/or a git blob store
- <2019-08-12 man. 22:15> The schema should define object interconnection and metadata (but I don't have clear vision of how it should look)
- Separate out a propertyset and all the propertysets it depends on to a separate ModelContext
- Should be thread safe before it is set to complete
- Since the metadata object will be first in all files, it is a good place to put machine and human readable version information
- The idea is that objects that aren't created locally, and have been locally modified are the ones that needs to be saved back to a remote server
- Aspects should come before propertysets referencing them
- Base aspects should come before aspects inheriting them
- Propertysets being referenced should come before propertysets referencing them
- Contents of a container should come before the container
- Endpoints of a relationship should come before the
- Propertyset fronted by graphical proxy propertyset should come before the proxies
- Not so easy, since JSON doesn't have a syntactic marker for this
- The metadata object stores and restores Date objects with millisecond accuracy, but the values are stored as JSON strings with a custom format (human readable)
- This could be something on the aspect, but I don't see how to do this cleanly during parsing
- It could be parsed as a string value, and then converted to a DateTime value on access or when an aspect is applied
- Check a propertyset to see if it has the required propertysets for an aspect
- Check the propertyset to see if it brings anything meaningful to the table (ie. property definitions)
- Use this with the built-in aspects
- Contains a single property that is a reference to a different propertyset
- Think about how a proxy should be handled in an aspect container
- It would be nice if the actual application of the aspect could "pass through" to the proxied propertyset
- The graphical information (position, symbol) should be added to the proxy
- Per propertyset load time
- Per propertyset last modified time
- Propertyset delete time
- This may be necessary when creating more components in a server setting (servlet component, and SQL server component)
- The functionality of JsonPropertysetPersister must be available in some fashion from the Modelstore
- JsonPropertysetPersister parsing and unparsing will be needed for:
- parsing JSON messages from clients (REST requests)
- Creating JSON messages to send to clients (REST responses)
- Loading and saving individual objects in a jgit based versioned storage
- Possibilities
- I like s-expressions
- Is there something in JSON that could be used
- Just implement something as nested complex objects and let its JSON representation be the wire format
Maybe actually two separate:
- S-expression directly on jackson
- sxml on top of the existing XML serialization/deserialization
- <2015-07-25 lør 15:44> Not doing this for now, too hard to be certain with the current implementation
- Not sure if this is necessary with the primitive value setters in place for both Propertyset and Valuelist?
- <2015-07-14 tir 17:49> I couldn't compare two propertsets that should have been equal with assertEquals() in a test
- I can't decide what's the correct thing to do here, so I compared the unwrapped propertysets instead
- This is the only (practical) way to track changes to list and complex properties
- Wrap the complex properties and the lists will be just too much work
- Don't know if I will go through with this...?
- What was this about? Is this something other than the current wrapping?