Introducing Conveyor

Hello world! Conveyor is a new tool that makes distributing desktop and command line apps as easy as shipping a web app. It generates self-upgrading packages for Windows, macOS and Linux using each platform’s native package formats, you don’t need to have those operating systems to build them, and it looks like this:

It can do all this from whatever operating system you like because it implements all the packaging, signing and Apple notarization logic itself. Drop it into any continuous build system or run it from your laptop. The results will be the same: a simple config file goes in one end, an incremental and parallel build system processes it and out the other end comes a fully fledged repository ready for publishing. Try installing a sample app and see for yourself.

Outputs

Windows users have traditionally dealt many different ad-hoc installers and updaters. This is especially tough for IT departments, who need consistency and must thus painfully “repackage” apps, slowing things down. Conveyor fixes this because it generates both an ordinary non-self updating zip and an MSIX package. MSIX (also known as AppX) is the Windows 10 package management engine and has some great modern features. Updates happen transparently and silently even when the app isn’t running, but can also be forced on a schedule or even on every app start. An embedded block map allows downloaded data to be shared across apps from entirely different sources, and shared files are hard-linked to save disk space. That in turn is made possible by automatic containerization of installed apps, and MSIX packages are easily deployed across corporate networks or incorporated into pre-canned Windows OS images.

Mac users get a fully signed, notarized and Sparkle-enabled app bundle. Sparkle is a slick auto update framework widely used across the Mac ecosystem. It asks the user once on first use whether to allow background updates, and from then on updates apply silently whilst the app is running. Admins can easily update apps without running them in the background using the dedicated Sparkle tooling.

Linux users get a tarball and a Debian apt repository (with RPM support in development). The .deb packages automatically install GPG keys and apt sources, so users don’t need to use the command line if they don’t want to - installing packages through the GUI is sufficient.

Finally, Conveyor generates a static download page that ties it all together with operating system and CPU architecture auto-detection. The repositories are just a set of flat files, so you can publish them via GitHub Releases. Installs will automatically upgrade to the latest released version.

You can read more about the output formats.

Any kind of app

Get started fast with four project templates that can be generated from the command line:

  1. A native app written in C++ that demonstrates using CMake, OpenGL pixel shaders and the GLFW library.
  2. A hello world Electron project.
  3. A JVM app written in Kotlin using the Jetpack Compose Desktop toolkit. This is the same next-gen reactive toolkit developed for Android by Google, but ported to the desktop by JetBrains.
  4. A JVM app written in Java using the mature and desktop focused JavaFX toolkit.

Conveyor isn’t only useful for desktop apps. It can also package command line tools and servers. In fact, Conveyor is packaged with itself and we use it to manage our servers.

Both types of program get some special enhancements. For servers Conveyor can generate systemd units, as well as the package scripts needed to start/stop a server at install time and across upgrades. The systemd configs support sandboxing using the DynamicUsers feature, and the packages can also include sample Apache/nginx reverse proxy configs. We use this feature to manage our own servers.

JVM apps get a special launcher that makes it way easier to write portable CLI Apps. For example, on Windows the console is configured to ensure Unicode and ANSI colors just work out of the box without extra libraries.

Free for open source projects

During the introductory period Conveyor is free for everyone. Eventually it will require a license for use with non-open source projects. The model is trust but verify. Getting a license key is fully automatic: just build a project which doesn’t supply the URL of an open source version control repository and a license key will be added to your config automatically. Once charging begins you’ll be able to pay asynchronously, ensuring you are never blocked by slow corporate processes. We also offer source licenses for companies that want to purchase the code for forking and internal modification, or to expose Conveyor’s functionality as a service.

Signing and notarization

Code signing has a reputation for being awkward. Conveyor eliminates the technical pain involved in signing.

If you don’t have or don’t want signing certificates, no problem. This can be appropriate when targeting developers, or making internal apps. Your download site will include shell and PowerShell scripts that can be triggered using a curl | bash style command (try it out for yourself), as well as GUI instructions for macOS. The apps are self-signed not unsigned so different versions can be linked together and operating systems won’t forget granted permissions when the app upgrades.

To get a standard download experience without security errors or terminal commands, you need to sign your software. If you already have certificates Conveyor can use them, including if your keys are stored in a hardware USB token or other PKCS#11 key store. If you don’t then it will generate the CSR files that you can upload to a certificate authority. You can then obtain the right credentials for Windows and macOS using just a web browser - you don’t need those operating systems at any point to deploy to them.

And like with everything else, you can sign your packages from any operating system. That’s especially useful for building packages in a Linux CI system, or rapidly creating lightweight niche apps from your laptop. Conveyor is the only packaging tool that can do this.

In recent years Apple has introduced a new requirement on top of signing called notarization. Conveyor implements the full notarization protocol whilst following all Apple’s rules.

Electron, simplified

If you’ve ever distributed an Electron app, you already know that it’s way harder than it should be. Tutorials frequently link you to programs that are abandoned. The official Squirrel updater is deprecated and without any full time maintainers and hasn’t done a macOS release for over 5 years. Also, Mac development stopped completely in 2021 and it isn’t Apple Silicon compatible. Many other advertised alternatives to Squirrel are also abandoned, largely undocumented or both. Switching to Sparkle (the framework Conveyor uses on macOS) was requested by users but got no response: the issue thread contains descriptions of some of the problems the various packaging options present.

Even when these tools work they often require custom update servers with dedicated databases, making the Electron community heavily reliant on a centralized update service that is only offered to open source apps. Finally, Windows has for some years been migrating to purely declarative Linux style packages, and some OS features require that approach to be used. Conveyor generates packages that fully integrate with the modern Windows approaches, thus also benefiting from the best possible support for IT-managed networks.

Conveyor’s support for Electron apps eliminates all that complexity. Here’s what a config looks like:

include required("/stdlib/electron/electron.conf")

// This section imports metadata like the name, 
// app version and which version of Electron to use.
package-lock {
  include "package-lock.json"
}

app {
  display-name = Template App
  rdns-name = com.example.template-app
  site.base-url = "localhost:8899"
  icons = "icons/icon-*.png"
}

You can then upload the resulting directory to any static HTTP file server. Every setting is documented, the update engines are all well maintained and because proprietary apps will be charged for in future, incentives are correctly aligned to ensure long term maintenance.

Deep JVM integration

Cross-platform packaging makes a lot of sense when paired with a great cross-platform runtime, like a modern JVM. Mature UI toolkits, pauseless garbage collectors and many supported languages make them a great choice for building GUI apps fast, without needing the web stack. That’s why Conveyor has deep support for both GUI and command line JVM applications. Everything needed is bundled with the app and the user won’t see any indication of what technologies you’re using. A custom launcher makes apps work better, especially for command line tools. There’s automatic integration with Gradle and Maven projects, and it’s fully integrated with the module system - jdeps and jlink are both used to create a minimal download automatically. JARs are optimized in various ways to (optionally) strip debug info, make them deterministic, and to extract native libraries ahead of time. Library extraction avoids messy and slow runtime unpacking, ensures code signing works properly and shrinks downloads by automatically deleting libraries that are for the wrong CPU or operating system.

Simple configuration

There are over 120 available settings but smart defaults means you’ll probably never need them. Here’s the only configuration needed for the sample Jetpack Compose Desktop note taking app:

// Use Java 17 and import config from Gradle.
include required("/stdlib/jdk/17/openjdk.conf")
include required("#!./gradlew -q printConveyorConfig")

app {
  // Releasing using GitHub Releases is a good idea if that's where your code is.
  site.base-url = github.com/hydraulic-software/eton-desktop/releases/latest/download

  // Make the name of the program a bit clearer than just "Eton".
  display-name = Eton Notes

  // When source code is released under an open source license, Conveyor is free.
  // We just have to point to our source repository.
  vcs-url = github.com/hydraulic-software/eton-desktop
  license = Apache 2

  // The 'icons' directory contains rescaled versions of the Jetpack Compose logo.
  icons = icons
}

// This line is added automatically, and allows the tool to change  whilst 
// preserving backwards compatibility.
conveyor.compatibility-level = 1

One small step towards our glorious post-web future

The founding belief of Hydraulic is this: HTML has served developers well for many years, but it’s time to start imagining what comes next. There are many people out there with ideas for new and better platforms, but they’re constrained by the difficulty of distributing software outside the browser. Developing interesting new kinds of app/document runtime is a long term project and thus we use an incremental approach, starting with making it as easy to ship apps to desktops as it is to ship a web app. When it gets easier to make powerful apps for powerful platforms we’ll will do it more, and that in turn can improve the privacy, security, usability and performance of our software.

On that foundation can be built a new generation of ‘neo-browsers’ that explore new ways to solve the problems that web browsers solve. We have our own ideas about what that might look like, but by starting with a generic solution to the downloads-and-updates problem we want to empower everyone to experiment on their own.

We plan to write more about this vision in a series of blog posts.

Sound good?

Outstanding. First, subscribe to this blog or our Twitter feed. We’ll only notify you when we have something interesting to say, and that’s a promise. Then visit the documentation to learn how to download and use Conveyor. If you get stuck, visit our GitHub discussion forum or say hello in our Zulip chatroom. And if you find post-web platforms as interesting as we do, take a look at our jobs page.

.