When handling model structures from systems like thrift and protobuf, one problem that we’ve come across is the fact that we want to define the data structures as close to where it is being used as possible:

  • If the model structure is used multiple places, it should be possible to define it once, and use it in multiple project. The type of inclusion should be as close to native dependency management for that language as possible. E.g. in java you want to use maven, python with pip, php with composer etc.
  • If using the structures goes across multiple programming languages, we still want to define it once, and then include it with a native dependency management system where needed.
  • If possible, we want each project using these structures to be able to define the parameters for generating the source code to be used to handle the model structures. E.g. an android app may want the tiny variant, while the java backend probably want the normal java variant.

Neither thrift nor protobuf has solved this problem, and relied on the build system (make, maven, bazel) to be able to gather all the relevant model definitions and compile the appropriate output.

Solving all these problems in once is not a trivial task, but I’ve gotten pretty close to something that should be able to solve all the requirements.

By using maven as internal dependency management, model definitions (like thrift files) should be able to depend on models defined in other projects. By using the maven classifier pattern it is possible to publish custom artifacts, this time containing the model definition files (the ‘.thrift’ files). E.g.:

<dependency>
    <groupId>net.morimekta.providence</groupId>
    <artifactId>providence-reflect</artifactId>
    <version>0.1.2</version>
    <classifier>providence</classifier>
    <type>zip</type>
</dependency>

This Points at the providence definitions of the internal models that providence use to define thrift itself. With this dependency, and using the providence-maven-plugin, you can include structures from the model.thrift file directly (you may also need to depend in the providence-reflect jar file for the compiled models, unless you want to compile these yourself).

Currently (at version v0.1.2) only the maven plugin supports using maven dependencies. My plan for v0.1.3 is to have the pvdc code generator be able to do two things:

  • Take in maven artifacts in order to fulfill dependency includes. This is already done by the maven plugin using the standard maven dependencies.
  • Take in a maven artifact and compile that code locally. This needs to be added both to the maven plugin and the compiler.

Sadly because of the way thrift defines namespaces, all thrift definition files (including dependencies) needs to have the namespace {language} ... property set for the language you want to compile for. And This, I have not found a solution around yet.

Note that providence only support java at the moment, so the namespace problem is not as urgent as solving the dependency problem. If the .thrift files are available globally, it should be possible to solve. On the other hand, I would like to add support for at least one more language in the “near” future.