Extractor Service

This service receives a file event from the accepted-file topic, looks up the registered extractor configuration for the client application, submits the file to the extractor’s HTTP endpoint, and polls for a result. When the extractor reports completion, the result is stored in blob storage and a job status event is published.

Two extractor types are supported: DOCSAI and DBRICKS. Each has distinct submission and polling contracts.

Registration Contract

Extractor configuration is read from registration data using this precedence:

1. features[].featureName == "extractor" and featureConfig.extractors
2. extractors in the registration payload
3. Fallback lookup to /v1/clientApplication/{clientApplicationId}/extractors

Each configured extractor is expected to provide these fields.

Only the first extractor in the configured list is processed.

Submission Contract

The service sends an HTTP POST request to extractionUrl.

• Authentication: the request includes Authorization: Bearer <token>.
• Transport success: any HTTP 2xx response is treated as successful transport by the HTTP client.
• Transport failure: non-2xx responses and transport errors are retried up to 5 attempts with a fixed 2000 ms delay.

Submission Payload by Extractor Type

DOCSAI

Request body:

{
  "jobId": "<job-uuid>",
  "fileUrl": "<signed-blob-url>",
  "DocumentHash": "<file-hash>",
  "FileLocationUrl": "<file-location-url>"
}

Expected response payload shape:

{
  "ExtractedEntityId": 123,
  "status": "<string>",
  "message": "<string>"
}

runId is derived from ExtractedEntityId.

DBRICKS

Request body:

{
  "name": "extract-<fileId>-<jobId>",
  "client_id": "<azure-client-id>",
  "created_by": "<azure-client-id>",
  "metadata": {
    "output_path": "/results/<clientApplicationId>/<fileId>/<jobId>"
  },
  "files": [
    {
      "file_path": "<source-path>",
      "agent": "<agent>"
    }
  ]
}

Expected response payload shape:

{
  "id": "<run-id>",
  "name": "<string>",
  "overall_status": "<string>",
  "created_at": "<iso-timestamp>",
  "task_count": 1
}

runId is derived from id.

For DBRICKS, agent is required. Missing or blank agent fails submission.

Polling Contract

Polling starts after submission when a job has runId and extractor pollingUrl.

Polling URL format:

GET {pollingUrl}/{encodedRunId}

If pollingUrl is missing, polling fails and the job is marked failed. If runId is missing, polling fails and the job is marked failed.

Polling Status Mapping

DOCSAI

• Reads Status from polling response.
• Status == "complete" maps to Succeeded with Payload as result.
• Status == "failed" maps to Failed.
• Status == "processing" maps to Processing.
• Missing Status maps to Processing and keeps polling.
• Polling request errors map to Processing and keep polling.

DBRICKS

• Reads overall_status from polling response.
• COMPLETED maps to Succeeded.
• FAILED or CANCELLED maps to Failed.
• ENQUEUED or PROCESSING map to Processing.
• Missing or unrecognized status maps to Processing.
• Polling request errors map to Processing and keep polling.

Polling Retry and Stop Conditions

Polling behavior is controlled by resolved job tracking configuration.

Configuration precedence:

1. extractor.jobTrackingConfig.*
2. Global app configuration keys:
   • FLUXX_EXTRACTOR_MAX_POLLING_ATTEMPTS
   • FLUXX_EXTRACTOR_POLL_INTERVAL_MS
   • FLUXX_EXTRACTOR_MAX_DELAY_MS
   • FLUXX_EXTRACTOR_MAX_POLL_DURATION_MS

Backoff formula:

where:

• effectiveMaxDelayMs = maxDelayMs when maxDelayMs > 0
• otherwise effectiveMaxDelayMs = baseIntervalMs

Polling stops and job fails when either condition is met:

• attemptNumber >= maxPollingAttempts
• jobAgeMs > maxDurationMs

Result Processing

When polling returns Succeeded, result handling is extractor-specific.

DOCSAI

• Inline object payloads are serialized to result.json.
• Data URL payloads are decoded and uploaded.
• HTTP URL payloads are downloaded and uploaded.
• Upload target container: results
• Upload target path: $<clientApplicationId>/<fileId>/<jobId>/result.json

DBRICKS

• Uses tasks[].result_url to resolve blob location.
• Reads result blobs from container results.
• Sends each blob payload to Palantir ingestion.
• Returns result_url when available.

Status Events

The service publishes job status events to the job status topic with statuses:

• Queued
• Started
• Processing
• Succeeded
• Failed
• Canceled

The service publishes extracted result events after successful result handling.