Building and Packaging

How to build the collector natively, as Docker images and as portable release binaries for x86_64 and 64-bit ARM, how releases are published, and how to run the resulting binary. For the development workflow (tests, CI, the simulator) see the Developer Guide; for deploying with systemd see Deployment.

What gets built

The workspace produces two binaries:

• collector-app, the collector (crate crates/collector-app);
• sunspec-sim, the SunSpec Modbus TCP simulator (crate crates/sunspec-sim), for local development and tests.

--locked builds with the committed Cargo.lock, as CI and the Dockerfile do. The workspace uses Cargo's default release profile.

Toolchain and MSRV

The minimum supported Rust version is 1.88, set as rust-version in the workspace Cargo.toml. The Docker builder stage uses the same version (rust:1.88-slim-bookworm), so the Docker build checks that the MSRV still works; the other CI jobs use the latest stable Rust. When the MSRV changes, change both places.

Prerequisites

Debian / Ubuntu

sudo apt-get update
sudo apt-get install -y build-essential cmake pkg-config libssl-dev libcurl4-openssl-dev

These are the packages CI installs (build-essential is preinstalled on the CI runners).

macOS

xcode-select --install       # compiler, linker and SDK (the SDK provides the curl headers)
brew install cmake openssl@3

openssl-sys finds Homebrew's OpenSSL without further configuration. Native macOS binaries link libssl and libcrypto from Homebrew dynamically, so openssl@3 must stay installed on the machine that runs them.

Why cmake, OpenSSL and curl

The Kafka client is rdkafka 0.39 with the features cmake-build, ssl, tokio and zstd:

• cmake and a C/C++ toolchain. cmake-build compiles librdkafka 2.12 (bundled in rdkafka-sys 4.10.0+2.12.1) from source with CMake. No system librdkafka is used, so the version is the same on every platform.
• OpenSSL. ssl builds librdkafka with TLS support against the system OpenSSL, which needs the headers at build time (libssl-dev) and the library at run time (libssl3), unless the vendored-openssl feature is used (see below).
• curl headers. librdkafka is built with curl disabled and the binary does not link libcurl. However, librdkafka 2.12's rdkafka_conf.c includes <curl/curl.h> under #ifdef WITH_OAUTHBEARER_OIDC, and the generated config.h always defines that macro (as 0 or 1), so the header has to be present for the build to compile.
• zstd and SQLite need nothing from the system: zstd-sys compiles its bundled zstd, and sqlx's sqlite feature compiles a bundled SQLite.

The first build compiles librdkafka and therefore takes noticeably longer than later builds.

Statically linked OpenSSL

collector-app has an opt-in feature vendored-openssl (forwarded to avro-kafka and rdkafka's ssl-vendored). With it, OpenSSL is compiled from source and linked statically, so the binary does not depend on the OpenSSL of the machine it runs on. Building OpenSSL needs perl and make.

cargo build --release --locked -p collector-app --features vendored-openssl

The release builds use it; native and Docker builds do not.

Native builds

cargo build --workspace                                       # debug, all crates
cargo build --release --locked -p collector-app               # release collector
cargo build --release --locked -p collector-app -p sunspec-sim

A native Linux release build links OpenSSL 3 and glibc dynamically; check the full list for your system with ldd target/release/collector-app. To install it, copy the binary, for example to /usr/local/bin/collector-app for the systemd unit described in Deployment.

Docker images

The Dockerfile has these stages:

docker build -t sunspec-collector .                         # collector (default target)
docker build --target simulator -t sunspec-simulator .      # simulator

Build cache

The Dockerfile starts with # syntax=docker/dockerfile:1 and uses BuildKit cache mounts for the cargo registry (/usr/local/cargo/registry) and the target directory (/app/target), so rebuilds after a source change reuse the compiled dependencies, librdkafka included. Cache mounts need BuildKit, which is the default builder in current Docker Engine and Docker Desktop releases. Because the target directory is a cache mount and not part of the image, the binaries are copied to /out in the same step. Both images share the builder stage, which builds both binaries in one cargo invocation.

.dockerignore keeps target/, .git/, .github/, docs/, docker/ and SQLite files out of the build context.

Running the collector image

docker run -d --name sunspec-collector \
  -p 9090:9090 \
  -v sunspec-data:/app/data \
  -e SUNSPEC_STATIC_DEVICES=192.168.1.20:1 \
  -e SUNSPEC_KAFKA_BROKERS=kafka.example.internal:9092 \
  sunspec-collector

• Buffer. The image does not set SUNSPEC_BUFFER_PATH. The collector runs in /app/data, so the default relative buffer.path (sunspec-buffer.sqlite) is /app/data/sunspec-buffer.sqlite, and a buffer.path from a config file or SUNSPEC_BUFFER_PATH is used as given (relative paths resolve against /app/data). Keep /app/data on a volume so buffered samples survive container replacement. A new named volume takes its ownership from the image, so it is writable by collector; a bind-mounted host directory must be made writable for that user yourself.

• Upgrading from older images. Images that set SUNSPEC_BUFFER_PATH=/app/data/buffer.sqlite stored the buffer under that name. To keep sending rows already buffered in an existing volume, set SUNSPEC_BUFFER_PATH=/app/data/buffer.sqlite (or move the file and its -wal and -shm files to sunspec-buffer.sqlite while the collector is stopped).

• Config file. Mount it and set SUNSPEC_CONFIG, or pass --config: the image's entrypoint is /app/collector-app, so arguments after the image name go to the collector:

  docker run -d -v "$PWD/collector.toml:/app/collector.toml:ro" -v sunspec-data:/app/data \
    -p 9090:9090 sunspec-collector --config /app/collector.toml
  

• Environment files. With --env-file, put comments on their own lines: Docker does not strip trailing # ... from values, and the collector rejects values it cannot parse.

• Health check. curl -fsS "http://localhost:${SUNSPEC_METRICS_PORT:-9090}/healthz" every 30 s (timeout 3 s, start period 20 s, 3 retries), so it follows SUNSPEC_METRICS_PORT. If you set the port only in a config file ([metrics] port), also set SUNSPEC_METRICS_PORT or override the check with --health-cmd.

• Stopping. docker stop sends SIGTERM, which starts the graceful shutdown (pollers stop, queued samples are written, the uplink drains for up to buffer.shutdown_drain_timeout_ms). Docker sends SIGKILL after 10 s by default; docker stop -t 30 matches the 30 s the systemd unit allows.

• Schema and version. docker run --rm sunspec-collector --print-schema prints the Avro schema of the published records, and --version the collector version.

Running the simulator image

docker run --rm -p 1502:1502 sunspec-simulator --units 1,2

Arguments replace the default --bind 0.0.0.0:1502 --units 1; the bind address defaults to 0.0.0.0:1502 anyway. See the Developer Guide for all flags.

Docker Compose stack

docker-compose.yml runs a complete local pipeline without hardware:

docker compose up --build            # build the images and start everything
curl localhost:9090/healthz          # 200 with a JSON body once the pipeline runs
curl -s localhost:9090/metrics | grep '^sunspec_uplink_records_total'

# Record keys on the topic (the values are binary Avro)
docker compose exec kafka /opt/kafka/bin/kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 --topic sunspec.telemetry --from-beginning \
  --max-messages 10 --property print.key=true --property print.value=false

docker compose down -v               # stop and remove the containers and volumes

docker compose down without -v keeps the volumes, and with them the collector's buffer and the Kafka data. docker compose stop gives containers 10 s before killing them; use docker compose stop --timeout 30 collector for a full graceful stop, as CI does.

Release builds

Release binaries are built with cross for x86_64-unknown-linux-gnu and aarch64-unknown-linux-gnu, in images derived from the cross-rs 0.2.5 images (Ubuntu 16.04). They need glibc 2.18 or newer and link nothing else that a target system might lack: OpenSSL is vendored, librdkafka, SQLite, zstd and zlib are compiled in, and the only shared libraries are glibc's and libgcc_s.

cargo install cross --locked
target=aarch64-unknown-linux-gnu
docker build -t "sunspec-cross-$target" -f docker/Dockerfile.cross docker
# x86_64: pass the pinned image from the release workflow's build matrix
# docker build -t sunspec-cross-x86_64-unknown-linux-gnu -f docker/Dockerfile.cross \
#   --build-arg BASE_IMAGE=ghcr.io/cross-rs/x86_64-unknown-linux-gnu:0.2.5@sha256:<digest> \
#   --build-arg DEB_ARCH=amd64 --build-arg MULTIARCH=x86_64-linux-gnu docker
cross build --release --locked --target $target -p collector-app -p sunspec-sim \
  --features collector-app/vendored-openssl
readelf -d target/$target/release/collector-app | grep NEEDED    # glibc and libgcc_s only

• Cross.toml maps each target to the image tag sunspec-cross-<target>; build the image under that tag first.
• docker/Dockerfile.cross starts from BASE_IMAGE, the target's ghcr.io/cross-rs/<target>:0.2.5 image pinned by digest (the default is the aarch64 one; the release workflow's build matrix lists both). It adds the target's Debian architecture if it is not the image's own, installs cmake, make, perl, pkg-config and libcurl4-openssl-dev for the target (the curl headers librdkafka needs at compile time), and points PKG_CONFIG_LIBDIR and PKG_CONFIG_PATH at the target's multiarch directory.
• --features collector-app/vendored-openssl is required: the images' OpenSSL is older than the version librdkafka 2.12 needs. OpenSSL is compiled from source (hence make and perl).

Archives

scripts/package-release.sh <version> <target> <bin-dir> writes dist/sunspec-collector-<version>-<target>.tar.gz, one directory holding collector-app, sunspec-sim, README.md, CHANGELOG.md, LICENSE-MIT, LICENSE-APACHE, LICENSE-SUNSPEC-MODELS (the bundled model definitions), config.example.toml, sunspec-collector.env and sunspec-collector.service. It also copies the binaries to dist/bin/<amd64|arm64>/ for the release images.

scripts/install.sh installs an archive on a Linux machine; see Deployment. Its SUNSPEC_ROOT, SUNSPEC_TARGET and SUNSPEC_BASE_URL variables install under another directory, for another target, or from local files, which is how CI tests it.

Release images

docker build --build-arg BINARIES=prebuilt packages the binaries in dist/bin/<arch>/ instead of compiling them, so a multi-arch image needs no compilation under emulation:

docker buildx build --platform linux/amd64,linux/arm64 --build-arg BINARIES=prebuilt \
  --target collector -t sunspec-collector .

The published images are ghcr.io/imrans110/sunspec-collector and ghcr.io/imrans110/sunspec-simulator, tagged with the version (0.5.0), the minor series (0.5), latest for the newest non-prerelease, and sha-<commit>.

Release workflow

.github/workflows/release.yml runs on pushes to main, on pull requests that change the Dockerfiles, Cross.toml, scripts/ or the workflow, on manual dispatch, and on v* tags:

Every run except a tag push is a dry run: nothing is pushed or published. The archives and the release files are kept as the run's artifacts (archive-<target> and release). A manual run (gh workflow run release.yml --ref <branch>) also attests its release files, to test the attestation steps; pull request and main dry runs skip them.

Verifying a release

See Verifying a release.

Cutting a release

1. Pick the version (the Kafka record format and the metric names are part of the public interface, so changing them needs a new minor version before 1.0).

2. In a pull request, set version under [workspace.package] in Cargo.toml, run cargo check so Cargo.lock follows, and rename ## [Unreleased] in CHANGELOG.md to ## [<version>] - <date> with a new empty ## [Unreleased] above it.

3. After it merges, tag the merge commit on main and push the tag:

   git switch main && git pull
   git tag -a v0.6.0 -m "v0.6.0"
   git push origin v0.6.0
   

4. The workflow publishes the images and the release. The GHCR packages are linked to this public repository and take its visibility, so they can be pulled without logging in.

Running the binary

See Command-line options for the options, the environment, files, ports, signals and exit status.