The full flow is: upload → extract text → extract signals → review → export. Every request includes your API key when key protection is enabled.

API=http://localhost:8000/api/v1
KEY="sk_cleanextract_internal_123456"

# 1. Upload a document
DOC=$(curl -s -X POST "$API/documents/upload" -H "x-api-key: $KEY" \
  -F "file=@report.pdf" | jq -r .id)

# 2. Extract text, then signals
curl -s -X POST "$API/documents/$DOC/extract-text"    -H "x-api-key: $KEY"
curl -s -X POST "$API/documents/$DOC/extract-signals" -H "x-api-key: $KEY"

# 3. List the extracted signals
curl -s "$API/documents/$DOC/signals" -H "x-api-key: $KEY"

Or just use the workspace UI — upload from the Documents page, run extraction, and review signals visually.

Signal

A single structured data point extracted from a document — a typed value with the evidence that supports it. Example: a recommendation with the verbatim sentence it came from.

Evidence grounding

Every signal carries an evidence snippet from the source. Each snippet is located in the document text (ignoring case, spacing and quote style) and its character offsets and page are recorded. If a model proposes a signal whose evidence can't be found, it is markedgrounded: false, its confidence is capped at 0.4 and it is sent for review — or discarded entirely when DROP_UNGROUNDED=true. This is the primary guard against hallucinations.

Confidence

A 0.0–1.0 score per signal. Signals below the REVIEW_THRESHOLD, and all ungrounded signals, are marked needs_review and can be listed with?needs_review=true.

Set globally via EXTRACTION_MODE, or rely on the default. The mode controls how deterministic rules and the LLM combine.

In hybrid and llm, if the LLM is unreachable or over quota the rule-based pass still returns results — extraction never hard-fails on the model.

Two families of signals are supported: precise entities and higher-order claim-level statements.

Entities

Claim-level

The engine is built around one principle: recall first, precision second. Generate candidate signals broadly, then ground and verify strictly.

1. 1Ingest. Parse PDF / email / docx to text, preserving layout and character offsets.
2. 2Extract. Rule anchors and the LLM each propose candidate signals (high recall).
3. 3Ground. Each candidate's evidence is located in the source; if it can't be found, it's flagged for review (or dropped).
4. 4Merge. Duplicates across strategies are collapsed into one signal.
5. 5Review. Confidence routes uncertain signals to a human approve / reject queue.
6. 6Export. Approved signals leave as JSON, CSV or Excel with page and character offsets.

Each signal has a review_status that moves through human-in-the-loop states:

Approving, rejecting or editing a signal clears its needs_review flag. Export only approved and edited signals with /export-approved.json, .csv or.xlsx, or add ?approved_only=true to any export.

curl -X PATCH "$API/signals/$SIGNAL_ID/review" \
  -H "x-api-key: $KEY" -H "content-type: application/json" \
  -d '{"review_status":"approved","reviewed_by":"analyst@org"}'

All routes are under /api/v1. When REQUIRE_API_KEY is enabled, send your key in the x-api-key header.

Configured through environment variables (a .env file on the backend).

# Extraction
EXTRACTION_MODE=hybrid          # rule_based | hybrid | llm
REVIEW_THRESHOLD=0.6

# LLM provider (Ollama or Anthropic)
LLM_PROVIDER=ollama             # ollama | anthropic
LLM_ENDPOINT=http://localhost:11434
LLM_MODEL=llama3.1
LLM_API_KEY=                    # required for cloud / Anthropic

# Storage & limits
DATABASE_URL=sqlite:///./signalextract.db
MAX_UPLOAD_MB=20
ALLOWED_EXTENSIONS=.txt,.pdf,.docx,.eml,.csv

# Security
REQUIRE_API_KEY=true
API_KEY=replace-me

Does it depend on the LLM?

No. In hybrid mode the rule-based extractor always runs, so a down or rate-limited model degrades gracefully instead of failing.

Can I use a local model?

Yes — point LLM_PROVIDER=ollama at a local Ollama server, or at Ollama Cloud / Anthropic by setting the endpoint and key.

How are hallucinations prevented?

Evidence grounding: any signal whose evidence can't be matched back to the source text is discarded before it reaches review.