Schema Registry

All Kafka event schemas and REST API contracts are registered in a self-hosted Apicurio Registry instance. Apicurio was chosen because it is open source (Apache 2.0) and provides first-class support for Protobuf, OpenAPI, AsyncAPI, and JSON Schema in a single store.

ss3-protos Repository#

All .proto files for all services live in a single dedicated Git repository: ss3-protos. No service owns its schemas in its own repo.

ss3-protos/
  buf.yaml
  buf.gen.yaml
  io/shopstar/
    common/v1/common.proto
    catalog/v1/catalog_service.proto
    catalog/v1/events.proto
    inventory/v1/inventory_service.proto
    inventory/v1/events.proto
    order/v1/order_service.proto
    order/v1/events.proto
    ... (one directory per bounded context)
    audit/v1/events.proto

The repository uses Buf for linting and compatibility checking. Generated Java code is compiled and published to the internal artifact registry (Nexus/Artifactory) as a versioned Gradle dependency:

dependencies {
    implementation("io.shopstar:ss3-protos:1.4.0")
}

Services pin this version. Upgrading the dependency is the mechanism for picking up schema changes.

Artifact Organisation#

Apicurio organises artifacts by Group + Artifact ID. The platform uses two groups:

Kafka event schemas — io.shopstar.events#

One artifact per Kafka topic. Artifact ID matches the topic name exactly.

Compatibility rule: FULL — no field removals, no type changes; additions only. Applied both by buf breaking in CI (pre-merge) and by Apicurio server-side on registration (post-merge). Both gates must pass.

REST API specs — io.shopstar.rest#

One artifact per service that exposes a REST API. Quarkus generates the OpenAPI spec automatically at build time (/q/openapi); CI registers it in Apicurio.

Compatibility rule: BACKWARD — REST APIs are consumed by external integrations and the admin SPA at their own upgrade pace, so FULL is too strict here.

CI/CD Pipeline#

ss3-protos (on every PR and on merge to main)#

PR opened
  └─ buf lint                         style + proto best practices
  └─ buf breaking --against main      FULL compatibility check
  └─ block merge if either fails

Merge to main
  └─ buf generate                     compile Java sources
  └─ ./gradlew publish                publish versioned artifact to Nexus/Artifactory
  └─ register Kafka event schemas     PUT to Apicurio io.shopstar.events group
                                      Apicurio enforces FULL compatibility server-side

Per-service (on every build)#

Service build
  └─ Quarkus generates OpenAPI spec at /q/openapi
  └─ POST spec to Apicurio io.shopstar.rest/{service-name}
     Apicurio enforces BACKWARD compatibility

Kafka Serializer Configuration#

Services use the Apicurio Protobuf serde library. Schema IDs are embedded in Kafka message headers; consumers resolve schemas from Apicurio at decode time.

// build.gradle.kts
dependencies {
    implementation("io.apicurio:apicurio-registry-serdes-protobuf-serde:${apicurioVersion}")
}

# Producer — application.properties
mp.messaging.outgoing.order-placed.value.serializer=\
  io.apicurio.registry.serde.protobuf.ProtobufKafkaSerializer
mp.messaging.outgoing.order-placed.apicurio.registry.url=${APICURIO_REGISTRY_URL}
mp.messaging.outgoing.order-placed.apicurio.registry.auto-register=false
mp.messaging.outgoing.order-placed.apicurio.registry.find-latest=true

# Consumer — application.properties
mp.messaging.incoming.order-placed.value.deserializer=\
  io.apicurio.registry.serde.protobuf.ProtobufKafkaDeserializer
mp.messaging.incoming.order-placed.apicurio.registry.url=${APICURIO_REGISTRY_URL}

auto-register=false on all producers — schema registration is owned exclusively by the CI pipeline, never by a running service.

Schema Evolution#

Field numbers are never reused once assigned. Deprecating and reserving is the permanent end state — there is no “remove after migration” step.