Operating loop

Core Workflow

The early release focuses on a compact operating loop: set up a dataset, route real queries, process the query/extract queue, then use the resulting insights to create and curate clusters.

1. Setup dataset

Start by identifying the corpus you want Pelorus to observe and represent. A dataset points at that source material and is attached to a Dataset Group, which controls shared settings such as embedding space and storage.

There are two ways to bring a corpus in. Add Dataset points Pelorus at a folder of documents (the folder and everything under it), which it parses and chunks itself. Import takes a pre-chunked RAG Connector Dataset: Pelorus lists any Dataset it finds in the shared Datasets directory, keeps its chunks exactly as cut and under their own ids, and only ever reads the folder. Because a dataset is its chunking, an imported dataset cannot have files added or be re-ingested; the chunk ids are the Dataset's, and keeping them is what lets citations agree with the test sets RAGauge and testset-kit build on the same corpus. The Pelorus Space example is loaded this way.

During setup, configure the extraction template (the Extract Config and prompt). The template controls what each Extract should capture, how the fields are used, and how the resulting information packet stays faithful to the corpus while addressing a query.

Setup also covers the core server settings — models, retrieval, and processing. In normal Admin usage, these persisted settings become the source of truth after startup.

Models

Configure embedding models and extract LLMs. Keep public config files pointed at environment variable names instead of raw keys.

Retrieval

Tune extract thresholds, chunk thresholds, max results, and cluster matching behavior after checking real query results.

Processing

Use manual batches while validating prompts. Move to realtime only after demand quality and extraction behavior are acceptable.

llm_configs:
  - name: gemini-flash
    provider: google
    model: gemini-3-flash-preview
    api_key_env: GEMINI_API_KEY
    key_source: env

retrieval_config:
  query_extract_threshold: 0.98
  chunks_threshold: 0.45
  extraction_processing_mode: manual_batch

2. Route queries

Enable the dataset when it is ready to serve. Application traffic can then be routed through a connector, such as the Python client, so Pelorus can answer requests and observe the query patterns emerging from real usage.

Routing traffic is what turns the system from static retrieval into an adaptive context layer. Queries can match chunks, Query Extracts, and Cluster Extracts while also creating the signals used to improve future coverage.

Only live traffic leaves a trace. Queries from the Python client and MCP count as live by default; queries from the Admin Query tab and direct POST /query calls do not, so they record no hits, no utterances and no demand on any tier unless you mark them Treat as live query. That keeps a day of testing from inflating the hit counts you later curate from.

3. Process the extract queue (Extract Processing)

As queries reveal repeated needs, Pelorus can queue work to create or update query/extract pairs. Processing the queue turns observed demand into Extracts that are tailored to those queries and true to the corpus.

Queue processing may be manual while you validate the strategy and prompts, or more automated as confidence grows. The important control point is that new Extracts are created from the corpus with an explicit query in mind. Pelorus authors every Extract itself, with the extract LLM you configure; a client or agent can ask for one, but never writes one. Before a question is queued, Pelorus checks whether an existing Extract or cluster already answers it under the current thresholds and records a hit instead; similar in-flight questions merge into one backlog item.

A known limit. The default extraction strategy, index_guided, hands the extract LLM an outline of every document in the corpus so it can decide which sections to read. That outline is cached across calls, but it still has to fit in one context window, which puts a measured ceiling on corpus size at roughly 150 to 250 documents depending on outline length. Retrieval and curation work on larger corpora; extraction does not yet. Scoping the index per question is the planned fix.

4. View insights and curate clusters

Use the Query Map and related insight views to see where traffic is concentrating, where gaps are appearing, and where similar queries form useful neighborhoods.

From there, create and curate clusters. Cluster Extracts let you serve broader areas of demand, while manual curation lets you refine cluster boundaries, handle overlap, and keep coverage aligned with how user traffic changes over time.

You can also curate Extracts directly. For example, a subject-matter expert can add trusted context, references, or clarifications that should be available across related queries without editing the source corpus and reingesting documents. This is often most useful on a cluster, where the curated information can serve a neighborhood of related queries.