The corpus
28 documents from a travel agency that doesn't exist.
These 28 documents are the corpus: the policies, guides, flyers and notices of Pelorus Voyages, a fictional interplanetary travel operator, printed the way an operator would print them. Real document sets arrive as PDFs, so this is what a RAG pipeline would be asked to ingest. They are written so that retrieval goes wrong in known, named ways; see how RAG fails.
Clean to extract
Pictures carry no words.
If an image, a header or a page number put one extra word into the extracted text, the extracted text would no longer be the document's own words. So the documents are designed around what they leave out.
- Wordless art. Planets, horizons and the logo are SVG with no text elements, so extraction and OCR find nothing in them.
- No page furniture. No running headers, footers, page numbers, captions or pull quotes.
- No invented titles. A document with no heading in its text gets none on the page.
- Tables stay tables. Markdown tables become ruled tables with the same cells in the same order. A header row is never repeated on the next page.
- No split words. Ligatures and hyphenation are off, and hyphenated words and ranges like
8–10never break across a line. - Checked on every build. Each PDF is read back and compared with the text it was built from, word by word and cell by cell. 28 of 28 pass.
