Routing traffic

Connectors

Connectors are how application or agent traffic reaches Pelorus. They let you route queries through enabled datasets so Pelorus can return chunks, Query Extracts, or Cluster Extracts while observing demand.

Looking for the other direction? This page is about traffic coming into Pelorus — you are the caller. If you have a retrieval system and you want Pelorus to read it, that is the RAG Connector contract.

Python client

Use the Python client when application code should send runtime queries to a local or remote Pelorus engine. The default client surface is intentionally small: connect, query, look up an Extract, inspect read-only state, and close the connection.

from pelorus import PelorusClient

pq = PelorusClient.connect("http://127.0.0.1:9090")
result = pq.query("Which warranty terms apply?", dataset="support")

for document in result.documents:
    print(document.metadata.get("type"), document.content[:160])

Dataset setup, diagnostics, and development controls live on client.dev(). Keeping them separate makes it clear which calls belong in application request paths and which calls are for setup, tests, or operations.

from pelorus import Pelorus

with Pelorus.local(config="config.yaml") as pq:
    dev = pq.dev()
    dataset = dev.create_managed_dataset(
        name="support-docs", source_folder="/data/support-docs"
    )

    # A new dataset lands disconnected. Connect binds it to a datasource group
    # and creates its vectors; enable is a pure gate that turns serving on.
    dev.connect_dataset_group(dataset["id"], group_id, vector_policy="reuse_valid")
    dev.enable_dataset(dataset=dataset["id"])

    answer = pq.query("How do returns work?", dataset=dataset["id"])

group_id is the id of a datasource group, which you create in the admin console. Every dataset in a group shares one immutable embedding configuration, so its vectors are all in the same embedding space. If that configuration later changes, the dataset reads Re-embed Required and is held back from serving until you reconnect it with vector_policy="revectorize_all".

External Extract Processing workers use a third surface, client.extract_worker(). Read-only notebook and reporting workflows use client.analysis() to pull Query Maps, clusters, vectors, coverage, and similarity data without mutating the dataset.

See Client API for the current Python client and MCP-facing method signatures.

Download the example notebook: Pelorus Demand Coverage Audit.

MCP

Use the MCP connector when an MCP-capable agent should query Pelorus, inspect runtime state, or act as an external Extract Processing worker. MCP lets an agent workflow use the same dataset and retrieval surface without embedding Pelorus directly into the agent process.

pelorus-mcp --url http://127.0.0.1:9090 --transport stdio

The MCP tools are intentionally a thin facade over the Python client. Agents can query, poll Extract state, inspect dataset state, submit or list extraction demand, and run external Extract Processing leases. In the developer release, this is a trusted-local workflow: the connector does not implement authentication, so run it on localhost, a private machine, or behind your own access boundary.

Configure Claude Code

Start the Pelorus management API first, then let Claude Code launch pelorus-mcp as a local stdio MCP server.

python -m pelorus.serve --config config.yaml --host 127.0.0.1 --port 9090
claude mcp add pelorus -- pelorus-mcp --url http://127.0.0.1:9090 --transport stdio

For a shared project configuration, put this in .mcp.json at the project root. Claude Code will ask you to approve the server the first time the project opens it; use /mcp inside Claude Code to confirm that pelorus is connected.

{
  "mcpServers": {
    "pelorus": {
      "type": "stdio",
      "command": "pelorus-mcp",
      "args": ["--url", "http://127.0.0.1:9090", "--transport", "stdio"]
    }
  }
}

Use MCP as a client

An agent can call pelorus_query, choose raw chunks with chunks_only, poll queued Extract state with pelorus_get_extract, and inspect the selected dataset with pelorus_introspect. This is the right fit for agent applications that need source-grounded context from Pelorus but should not own the engine lifecycle.

Use MCP as a queue worker

An agent can submit demand, list backlog counts, claim work with pelorus_claim_extraction_task, author a draft from the returned query and source chunks, then complete, fail, or nack the task. The queue on/off control governs automatic internal processing; a trusted MCP worker claiming a task is treated as an explicit operator action.

Current split. MCP is the agent-facing surface for retrieval and external extract queue work. Cluster creation and curation stay in the Admin UI, management API, and Python analysis/admin surfaces for this release. Adding MCP cluster tools is a later extension, not part of the developer-preview contract.

Process the extract queue with your agent subscription

Extract Processing is the one place Pelorus calls an LLM, and in dev and test that API spend adds up fast. The MCP queue tools exist so it doesn't have to: set the extraction executor to external and the engine stops calling the extract LLM entirely — your MCP-connected agent (Claude Code, or any agent runtime billed through a subscription you already pay for) authors the Extracts instead.

retrieval_config:
  extraction_executor_mode: external   # engine never calls the extract LLM

The agent loop is: pelorus_claim_extraction_task (use a generous lease_seconds, e.g. 900+, for interactive agents), author a source-grounded draft from the task's query and source chunks, then pelorus_complete_extraction_task — or pelorus_nack_extraction_task when the corpus can't support an answer. Repeat until claim returns nothing. Pair a local embedding model (fastembed) with external execution and the whole dev loop runs without a paid API call.

Today the claimed task is a compact authoring packet, not the full internal extractor runtime. It includes the canonical query, representative example queries, lease metadata, the configured executor mode, and the top retrieved source_chunks. It does not currently expose the dataset's extractor prompt, the document outline index, or full-corpus context through MCP. That means the external agent authors from the supplied source chunks and its own MCP/task instructions; it is not yet running Pelorus' full-corpus extractor or the index-guided extraction planner over MCP.

Drafts, not answers. Agent-authored completions land as drafts pending source-grounded validation. They are never served as trusted Extracts until a validation or approval step promotes them.
Cluster curation still belongs to the admin/API surface. MCP query calls can ask Pelorus to include cluster matching, and the Python analysis surface can read cluster maps, coverage, vectors, and membership. MCP is intentionally not the cluster curation surface yet: it does not currently expose tools to create clusters, edit cluster extracts, regenerate labels, tune thresholds, or manage membership. Those operations are available through the Admin UI and management API today, and routing a small cluster-curation toolset to MCP is a natural follow-up once the developer workflow needs it.