Documents & retrieval · BM25 search
PDF to searchable knowledge base, offline
Add up to five PDFs. This page reads their text page by page, cuts it into chunks that stay on one page, and searches with BM25. Every result carries a page citation. The export is one HTML file that searches offline. This tool does not upload them.
Build a knowledge pack from your PDFs
Files are read in this tab. The tool does not upload them. Your PDFs and their text are not stored; only the files you download are saved.
1. Add PDFs
No files yet| File | Citation title | Year | Pages | No text layer |
|---|
The citation title starts as the PDF's Title field, or the file name. The year is optional and only used for the (Title, Year, p. N) format.
What is read, and what is not
- The text layer only. PDF.js returns the text objects on each page. Text that exists only as a picture is not read, and such a page is flagged.
- No OCR. Recognising text in images needs an engine such as Tesseract. This page does not run one.
- Reading order is approximate. In tagged PDFs, the PDF specification says logical and content order may differ. Two columns at the same height can merge into one line, so check the full page text before you quote.
- This tool does not upload your files. It has no upload code. Extraction and search run in this tab. The export is built here after one same-origin request for the search script, then saved by your browser.
2. Check the pack
3. Chunking and ranking
The defaults match Elasticsearch and Lucene. They are defaults, not optima: try other values on your own question and see which passage moves up.
4. Search and cite
Add PDFs firstPage numbers are positions in the PDF file, not printed page labels. The LLM block asks the model to cite its labels; check every quote against the PDF.
5. Export
The offline pack is one HTML file: the text, a copy of the shared search code and the page logic, all inline. It opens from your disk with no connection, and its security policy blocks network requests. The text file has a "## Page N" heading for each page. The chunks file lists every chunk with its page.
How to use PDF to Knowledge Pack
- Add PDFs. Drop or choose up to five files, or load the built-in sample. Each file's title, page count and pages with no text appear in the table.
- Check the pack. Read the check panel. A flagged page has little or no text, so a search cannot find words that are not there.
- Search and cite. Type a question. Each result shows its page, a highlighted snippet and a Copy citation button. Show full page text opens the whole page.
- Export. Download the offline pack (.html). Open the saved file from your disk; it searches with no connection.
What the text extraction does
PDF.js describes itself as "a general-purpose, web standards-based platform for parsing and rendering PDFs". This page uses it for one job: reading the text layer of each PDF. The library returns the text of one page at a time, as a list of text items. Each item carries its string, its position matrix, a width, a height and a flag that says whether a line break follows. Fields checked in PDF.js 3.11.174, the version shipped here, and in the PDF.js source on the master branch. Names may change between releases.
The page rebuilds lines from the positions. Items on the same baseline form one line, within two to three points depending on the item height. Inside a line, items are sorted from left to right, and a space is inserted where the gap is larger than a quarter of the item height. A line that ends in a hyphen, followed by a line that starts in lower case, is joined without the hyphen, so a word broken as "sam-" and "pling" comes out as "sampling". A gap well above the page's typical line spacing starts a new paragraph.
These are rules of thumb. The PDF 1.7 specification describes a logical structure tree. For tagged PDFs it says that the logical structure order and the page content order "are logically distinct and may or may not coincide". Many PDFs are not tagged. For those, it notes that in some cases the extracted text "can vary depending on the grouping of glyphs into strings". So this tool's line order is a rebuild by position, and two readers may return different text for the same page. The check panel and the "Show full page text" view exist so you can see the result before you rely on it.
Pages that return fewer than 25 characters, not counting spaces, are flagged as "no text layer (scanned or image-only?)". The flag comes from a character count. It cannot tell a scanned page from a blank one. Text that exists only as a picture needs optical character recognition. Tesseract is described in its manual as an open source text recognition (OCR) engine. This page does not run OCR, so a page that is only a picture stays unsearchable here.
How the search scores a passage
Each page is cut into chunks of about 120, 220 or 380 words. A chunk never crosses a page boundary, so every citation points at one page. Neighbouring chunks on the same page share about 15% of their words, which keeps a sentence that straddles two chunks findable. A short trailing piece is merged into the chunk before it.
Chunks are scored with BM25, using the shared text engine on apprun.icu. The BM25 formula sums, over the query terms, an inverse document frequency weight times a saturated term frequency. Robertson and Zaragoza describe BM25 as the "best-known instantiation" of the probabilistic relevance framework, and their review says a term's contribution to a document score "cannot exceed a saturation point". The table lists every rule the page applies.
| Part | What it does here | Setting |
|---|---|---|
| Stemming and stop words | Stems words with the Porter rules, so "citations" also matches "citation". Common words such as "the" and "of" are ignored. | Fixed |
| Inverse document frequency | Rare words weigh more than common ones. The form is the one Lucene uses: ln(1 + (N − n + 0.5) / (n + 0.5)). | Fixed |
| k1 | Controls term-frequency saturation: how fast a repeated word stops adding to the score. | 1.2 by default |
| b | Controls length normalisation: how strongly a longer chunk is penalised. | 0.75 by default |
| Heading guess | Adds the page's first line to the text that is searched, and shows it as a section guess under each result when that line looks like a heading. | Off by default |
The k1 and b defaults used here match the defaults in Elasticsearch, which documents k1 as "The default value is 1.2" and b as 0.75, and in Lucene's BM25Similarity. They are defaults, not optima. The Robertson and Zaragoza review says that "many experiments suggest a somewhat lower value of k1 and a somewhat higher value of b". Use the k1 and b fields on your own questions and see which passage moves to the top.
The score shown on each result is the BM25 score for that chunk. A higher number means more matching words, weighted by rarity. It is not a probability that the passage answers the question.
For the trade-off between exact-word ranking and embedding search on a small set of documents, read BM25 vs embeddings: which is enough for a small set of documents? To see how the chunk size changes what a model is given, try the Chunking Strategy Comparator.
How citations and page numbers work
Every citation names the page by its position in the PDF file. The first page of the file is page 1. Printed page labels, such as a roman-numbered preface or a cover that is not counted, are not read. If your reference needs the printed number, take it from the page itself.
The title comes from the file's Title field when the file has one. Otherwise it is the file name without ".pdf". You can edit the title in the document table. The year is optional, and it is used only for the parenthetical format, (Title, Year, p. 12). The square-bracket format, [Title, p. 12], needs no year.
Copy citation puts the chosen format on your clipboard. The Markdown format quotes the chunk and cites it: > quote — Title, p. 12. Purdue OWL's summary of APA style says to "include the page number at the end of the parenthetical citation", writes "p." for one page and "pp." for several, and uses "an en dash for page ranges" (Purdue OWL). The (Title, Year, p. 12) format is not APA, which puts the author's name in the parenthetical.
Copy as LLM context gathers the top passages for the query. Each passage is labelled in square brackets, for example [Title p.12], and the block asks the model to cite those labels after each claim. A model can still misquote a passage or cite the wrong label, so check each quote against the PDF.
A longer walk-through of the citation formats is in How to search a PDF offline with page citations.
Privacy: what happens to your PDFs
A file input lets the user "choose one or more files from their device storage". Once chosen, the files "can be uploaded to a server" through a form submission, or "manipulated using JavaScript code and the File API", as the MDN reference for the file input explains. The file input alone therefore does not prove where a file goes. What matters is the code that runs after the choice.
The tool's own code reads your PDF in this tab, through the browser's File API, which lets web content "ask the user to select local files and then read the contents of those files". The bytes go to PDF.js in this tab. The tool's code contains no upload request, and the search runs in the tab too.
The export is built in this tab after one same-origin request for the search script. It is a Blob, which MDN describes as a "file-like object of immutable, raw data". It is given a temporary URL, and that URL is saved through a link with a download attribute. The temporary URL is then released, because MDN says to call revokeObjectURL() to release an object URL.
You can check this yourself. Open the browser's network panel, choose a PDF and watch the requests. Choosing a PDF starts requests only for the PDF.js script and its worker, both from this site. Any request, including an image load, sends your browser's usual details, such as your IP address, to the host it goes to. The site-wide store widget and popup are separate from this tool. They load their own images and data from apprun.icu and Gumroad, and the tool never passes them your files.
Limits and what this cannot tell you
The tool cannot read text that exists only as an image, and it does not run OCR. Pages with no text layer stay empty in the pack, and the check panel says so.
Reading order is approximate. Lines are grouped by baseline, so two columns that sit at the same height can merge into one line. Tables are not rebuilt into rows and columns: they come out as the text lines that PDF.js returns. Check the "Show full page text" view for any page you quote.
The hyphen rule is a guess. It joins a line-end hyphen to a lower-case continuation, so a real compound such as "well-" and "known" becomes "wellknown". The original line break is not kept in the text.
BM25 matches words, not meanings. A search for "car" will not find a passage that only says "automobile". Synonyms, paraphrases and negations are not understood, and a passage with the right words in the wrong sense can rank first.
The defaults for k1 and b are not the best values for every collection. Some documents will rank better with other settings, and the page does not search for the best ones for you.
Only Latin-script PDFs were tested. Some CJK PDFs rely on predefined character maps, which this page does not load, so their text may come out wrong or empty.
Size limits: five files at a time, 60 MB in total and 1,000 pages in total. A pack above 25 MB is refused. Here 1 MB means 1,048,576 bytes. Extraction runs on the page for simplicity, yielding between pages. Moving it to a worker is possible but is not done in this version.
To map the claims in a paper to the sentences that back them, use the Claim–Evidence Map. To merge, split or compress the PDF before you load it, use PDF Tools.
Questions
Is my data uploaded?
The tool's code reads your PDF in this tab, makes no network request with its contents, and stores nothing unless you download the pack. The exported pack is a single HTML file with a CSP of default-src none, so it makes no requests at all. The site-wide promo widget is separate and never receives your files. To check, open the network panel, choose a PDF and watch the requests.
Does the offline pack work without a connection?
Yes. The pack is one HTML file. Its styles, a copy of the shared search code and your text are all inside it, so it opens from your disk with no server. It also carries a content security policy that tells the browser to block network requests, so the pack cannot fetch anything after it opens.
How accurate is this?
It is a rule-based keyword ranking, not an AI model. BM25 counts the query words in each chunk, gives rare words more weight and stops rewarding repeats after a point. It does not know that two words mean the same thing, so a synonym will not match. A top result can still be the wrong passage. Open the cited page and read it before you rely on a quote.
Why does a page show no text?
A page with fewer than 25 characters, not counting spaces, is flagged as having no text layer. The count cannot tell a scanned page from a blank one, and this tool does not run OCR. Text that exists only as a picture needs an OCR engine such as Tesseract. The check panel lists every flagged page.
Does copied PDF text always come out in reading order?
No, not always. In tagged PDFs, the PDF 1.7 specification says the logical structure order and the page content order "may or may not coincide". Many PDFs are not tagged. For those, the spec notes that in some cases the extracted text "can vary depending on the grouping of glyphs into strings". This tool rebuilds lines from text positions, which helps but cannot fix every layout. Two columns at the same height can merge into one line. Use Show full page text to check the order before you quote.
Which page number does a citation use?
It uses the position of the page in the PDF file, so the first page of the file is page 1. It does not read printed page labels such as a roman-numbered preface. If your reference needs the printed number, take it from the page. Purdue OWL's summary of APA style writes "p." for one page and "pp." for a range, with an en dash. APA's parenthetical citation also names the author, which this tool does not add.
What do k1 and b mean in BM25?
k1 sets how quickly repeated occurrences of a word stop adding to a score, which is called saturation. b sets how strongly a longer chunk is penalised. The Elasticsearch reference and Lucene both default to k1 = 1.2 and b = 0.75. Those are defaults, not the best settings for every collection. The Robertson and Zaragoza review says experiments suggest a somewhat lower k1 and a somewhat higher b. Try the fields on your own question.
Can I add a year and an author name to citations?
The year is optional. Type a four-digit year in the document table, and the (Title, Year, p. N) format becomes available for that document. The Author field from the file's metadata appears in the pack's file table but is not added to citations, so type the name yourself when you write the reference.
How big a PDF can I use?
Up to five files at a time, 60 MB in total and 1,000 pages in total. Sizes use binary megabytes: 1 MB here is 1,048,576 bytes. The reader handles one page at a time and yields to the browser between pages, so the screen stays responsive. An offline pack above 25 MB is refused with advice to split the work. Very large files use more memory in the tab.
Read more about this
Two plain-language guides that go deeper than this page can, with sources.