Registration Service Refactor

Goal

Research and propose an approach to consolidating the Registration Service APIs so that applications can complete onboarding through a coherent, reduced set of endpoints and new services can be accommodated without introducing additional complexity.

Today the registration data is spread across five entity groups (Client, Application, ClientApplication, Extractor, AllowedExtension), each with its own CRUD endpoints — 21+ endpoints in total. Onboarding a single client requires 4 sequential calls whose inputs depend on the previous response’s IDs, and adding a new pipeline capability (e.g. classifier, splitter) would mean adding yet another set of endpoints.

This spike proposes:

A single registration resource that wraps Client, ClientApplication, Extractors, and AllowedExtensions into one schema — so the full onboarding payload is sent and returned as a single document.
A features array within that registration schema where each pipeline stage (FileIntake, Classifier, Splitter, Extractor) is an independently configured, toggleable feature. New services are added as new feature entries with their own featureConfig shape — no new endpoints.
A reduced endpoint set (21 → 6) organised around /v2/registrations as a CRUD resource, plus a reference-data lookup for allowed file extensions.

Acceptance Criteria

Propose a forward-compatible schema that covers current services and can absorb future services
Identify Registration-to-Policy Engine integration contract, given that service opt-in decisions made at through registration will likely inform policy evaluation

1. Current State

1.1 Domain Model

Client (1) ──────┐
                  ├──> ClientApplication (M) ──> ClientApplicationExtractor (M)
Application (1) ─┘                          └──> ClientApplicationAllowedExtension (M)
                                                        │
                                                        └──> AllowedFileExtension (lookup)

The service exposes 21+ endpoints across 5 route groups (/v1/client, /v1/clientApplication, .../extractors, .../extensions, extension lookup). Full endpoint list is in openapi.yaml.

1.2 Problems

4 sequential calls to onboard: POST /client → POST /clientApplication → POST .../extractors → POST .../extensions. Each depends on the previous response’s IDs. Failure at any step leaves partial state.
Confusing surface: Client/Application are auto-created internally (createIfNotExists) yet have separate CRUD endpoints — unclear whether consumers should manage them directly.
No extensibility model: Adding a new pipeline capability (classifier, splitter) would require its own CRUD endpoints, further fragmenting the surface.

1.3 Feature Enablement & Policy Engine

The registration schema includes a features array — four pipeline stages that can be toggled per client:

Feature	Description
FileIntake	File ingestion — validates extensions, routes to processing folders
Classifier	Document classification — categorises incoming documents by type
Splitter	Document splitting — breaks multi-page documents into logical sections
Extractor	Data extraction — extracts structured data via extraction endpoints

Registration and Policy Engine are two separate concerns:

Registration	Policy Engine (“capacitor”)
What	Stores the declaration — “Client ABC has extractor enabled”
When	At onboarding time
Owns	The `features` array (what clients opted into + config)

  Registration                          Policy Engine                    Pipeline
  ┌──────────────┐   publishes event   ┌───────────────┐   "what to do   ┌──────────┐
  │ stores       │ ────────────────>   │ reads feature │   with this    │ executes │
  │ features     │   "features changed"│ state, builds │ <──────────── │ stages   │
  │ array        │                     │ execution     │ ────────────> │          │
  └──────────────┘                     │ plan          │   run plan    └──────────┘
                                       └───────────────┘

The registration spike defines the features array schema and publishes feature-change events. The Policy Engine runtime logic is a separate body of work.

How it fits in the system

                    onboarding time                          runtime
                    ──────────────                          ───────
  
┌──────────┐  POST /v2/registrations  ┌──────────────┐
│  Caller  │ ───────────────────────> │ Registration │
│          │ <─────────────────────── │   Service    │
└──────────┘   201 + features state   └──────┬───────┘
                                             │
                                             │ publishes event:
                                             │ "Client ABC features changed"
                                             │
                                             ▼
                                      ┌──────────────┐       ┌──────────┐
                                      │    Policy     │ <──── │ Pipeline │
                                      │    Engine     │ ────> │          │
                                      │ ("capacitor") │       └──────────┘
                                      └──────────────┘
                                        "what stages           "run fileIntake
                                         for this doc?"         then extractor,
                                                                skip classifier"

1.4 Current vs Proposed Onboarding

Current (v1) — 4 sequential calls:

  POST /v1/client/          POST /v1/clientApplication/   POST .../extractors        POST .../extensions
  ┌─────────────┐           ┌─────────────┐               ┌─────────────┐            ┌─────────────┐
  │ Create      │ clientId  │ Create      │ clientAppId   │ Add         │ clientAppId│ Set allowed │
  │ Client      │ ────────> │ ClientApp   │ ────────────> │ Extractor   │ ─────────> │ Extensions  │
  └─────────────┘           └─────────────┘               └─────────────┘            └─────────────┘

Proposed (v2) — one registration resource, features-driven extensibility:

  POST /v2/registrations                    ┌─────────────────────────────┐
  ┌────────────────────────────────────┐     │  registration (response)    │
  │  registration (request)            │     │  ┌───────────────────────┐  │
  │  ┌──────────────────────────────┐  │     │  │ clientId              │  │
  │  │ clientName                   │  │     │  │ clientName            │  │
  │  │ clientApplicationName        │  │     │  │ clientApplicationId   │  │
  │  │ features [                   │  │     │  │ clientApplicationName │  │
  │  │   { fileIntake, enabled,     │  │     │  │ dropFolderId          │  │
  │  │     featureConfig: {         │  │ ──> │  │ features [            │  │
  │  │       allowedExtensions,     │  │     │  │   { fileIntake:       │  │
  │  │       deliveryUrl, ... } }   │  │     │  │     allowedExts,      │  │
  │  │   { extractor, enabled,      │  │     │  │     deliveryUrl ... } │  │
  │  │     featureConfig: {...} }   │  │     │  │   { extractor ... }   │  │
  │  │   { classifier, enabled }    │  │     │  │   { classifier ... }  │  │
  │  │ ]                            │  │     │  │ ]                     │  │
  │  └──────────────────────────────┘  │     │  │ ]                     │  │
  └────────────────────────────────────┘     │  └───────────────────────┘  │
                                             └─────────────────────────────┘

Reduced endpoint set:

  POST   /v2/registrations          Full onboarding in one call
  GET    /v2/registrations/:id      Retrieve complete registration
  PATCH  /v2/registrations/:id      Update any part of the registration
  DELETE /v2/registrations/:id      Deactivate registration
  GET    /v2/registrations          Query/filter registrations
  GET    /v2/extensions             Reference data (lookup)

How this meets the goal:

Goal	How
Coherent, reduced endpoints	21 → 6, all organised around a single `registration` resource
Complete onboarding in fewer calls	4 sequential calls → 1 POST (no intermediate IDs to chain)
New services without additional complexity	New pipeline stages are entries in the `features` array — no new endpoints, no schema-level changes

2. Proposed Registration Schema

2.1 Request Schema (`POST /v2/registrations`)

{
  "type": "object",
  "required": ["clientName", "clientApplicationName"],
  "additionalProperties": false,
  "properties": {
    "clientId": {
      "type": "string",
      "format": "uuid",
      "description": "If provided, updates existing client. If omitted, creates new."
    },
    "clientName": {
      "type": "string",
      "minLength": 1,
      "maxLength": 500,
      "pattern": "^(?!\\s*$).+"
    },
    "clientApplicationId": {
      "type": "string",
      "format": "uuid",
      "description": "If provided, updates existing application. If omitted, creates new."
    },
    "clientApplicationName": {
      "type": "string",
      "minLength": 1,
      "maxLength": 500,
      "pattern": "^(?!\\s*$).+"
    },
    "features": {
      "type": "array",
      "items": {
        "type": "object",
        "required": ["name", "enabled"],
        "additionalProperties": false,
        "properties": {
          "name": {
            "type": "string",
            "enum": ["fileIntake", "classifier", "splitter", "extractor"]
          },
          "enabled": { "type": "boolean" },
          "featureConfig": {
            "type": "object",
            "description": "Shape varies by feature name"
          }
        }
      }
    }
  }
}

2.2 Feature Configuration Shapes

Feature	`featureConfig` fields	Notes
fileIntake	`allowedExtensions` (UUID[]), `maxFileSizeInBytes` (integer, max 2 GB), `deliveryUrl` (URI)	Maps to current ClientApplication + AllowedExtension
classifier	(to be defined)	Config shape TBD
splitter	(to be defined)	Config shape TBD
extractor	`extractors` (array of `{ extractionUrl, pollingUrl }`)	Maps to current ClientApplicationExtractor

2.3 Example

What the request looks like

{
  "clientName": "Company ABC",
  "clientApplicationName": "Invoice Processing",
  "features": [
    {
      "name": "fileIntake",
      "enabled": true,
      "featureConfig": {
        "allowedExtensions": ["<extension-uuid>"],
        "maxFileSizeInBytes": 2147483648,
        "deliveryUrl": "http://example.com/delivery"
      }
    },
    {
      "name": "extractor",
      "enabled": true,
      "featureConfig": {
        "extractors": [{
          "extractionUrl": "https://example.com/extract",
          "pollingUrl": "https://example.com/poll"
        }]
      }
    },
    { "name": "classifier", "enabled": false },
    { "name": "splitter", "enabled": false }
  ]
}

One call. The server creates Client, Application, ClientApplication, sets up folders, creates extractors, sets allowed extensions, and returns the full registration with all generated IDs.

Response (201 Created):

{
  "clientId": "d411997e-5fb9-4dd3-bb4a-b968fe711ab3",
  "clientName": "Company ABC",
  "clientApplicationId": "c311997e-5fb9-4dd3-bb4a-b968fe711ab3",
  "clientApplicationName": "Invoice Processing",
  "dropFolderId": "d311997e-5fb9-4dd3-bb4a-b968fe711ab3",
  "uploadFolderId": "u311997e-5fb9-4dd3-bb4a-b968fe711ab3",
  "internalFolderId": "f311997e-5fb9-4dd3-bb4a-b968fe711ab3",
  "features": [
    {
      "name": "fileIntake",
      "enabled": true,
      "featureConfig": {
        "allowedExtensions": ["a1b2c3d4-e5f6-7890-abcd-ef1234567890"],
        "maxFileSizeInBytes": 2147483648,
        "deliveryUrl": "http://example.com/delivery"
      }
    },
    {
      "name": "extractor",
      "enabled": true,
      "featureConfig": {
        "extractors": [{
          "clientApplicationExtractorId": "x711997e-5fb9-4dd3-bb4a-b968fe711ab3",
          "extractionUrl": "https://example.com/extract",
          "pollingUrl": "https://example.com/poll"
        }]
      }
    },
    { "name": "classifier", "enabled": false },
    { "name": "splitter", "enabled": false }
  ]
}

3. Feature Storage

New table to persist feature state alongside existing registration data:

ClientApplicationFeature
  ClientApplicationFeatureId   uuid PK
  ClientApplicationId          uuid FK → ClientApplication
  FeatureName                  varchar (fileIntake, classifier, splitter, extractor)
  Enabled                      bit
  FeatureConfig                nvarchar(max) — JSON
  IsActive                     bit

Why a new table (not a separate service):

GET /v2/registrations/:id needs to return features — without local storage it would depend on an external call
Follows existing pattern — every other registration entity is in this DB
Registration is the source of truth; Policy Engine subscribes to events

Why FeatureConfig as JSON (not separate columns):

Each feature has a different config shape — columns per type would require schema migrations for every new feature
AJV validates the shape at the API layer before it hits the DB
SQL Server JSON_VALUE() / OPENJSON() available if querying is needed

Dual-write to existing tables: The controller writes to both the new ClientApplicationFeature table and the existing normalised tables (ClientApplication, ClientApplicationExtractor, ClientApplicationAllowedExtension) — keeping v1 queries and backward compatibility intact.

4. Migration Strategy

Phase	Action	Breaking
1	Add v2 `POST /v2/registrations` alongside v1	No
2	Build remaining v2 endpoints (GET, PATCH, DELETE)	No
3	Migrate consumers to v2	No (coordination needed)
4	Remove v1	Yes

5. Open Questions

#	Question
1	Should `POST /v2/registrations` be fully transactional (rollback on failure) or best-effort with partial state?
2	Upsert (create-or-update via `clientId`) or create-only with separate PATCH?
3	If upsert — should extractors/extensions append or replace?
4	Where does the Policy Engine live — within registration service, or as a separate service?
5	Should `features` be required or optional on initial registration?
6	What are the `featureConfig` schemas for `classifier` and `splitter`?
7	Should feature changes publish a separate Service Bus event to notify downstream services?
8	Would existing Service Bus consumers (`FLUXX_REGISTRATION_TOPIC_UPDATED_REGISTRATION`) break if the event shape changes?