vault_search only indexed title + path + the auto-summary, and the summary
is written from the first 3000 runes of the file. Anything the summary left
out (room names, prices, codes) could never be found. On top of that the
FTS query used plainto_tsquery, which ANDs every word, so a normal question
like "what is the price of the Deluxe Ocean Suite?" matched nothing even when
the key words were indexed.
- Add vault_document_chunks (migration 98): the file body split into
chunks with their own tsvector and embedding. body_indexed_hash on
vault_documents records which content_hash the chunks came from, so
unchanged files are not re-chunked or re-embedded.
- The enrich worker rebuilds chunks when a file changes. This needs no LLM,
so it runs even when no provider is configured.
- Rescan backfills chunks for docs indexed before this change, since they
already have a summary and never go back through the worker.
- FTS matches any query word; ts_rank still ranks docs with more matching
words first. Both FTS and vector search look at the doc and its chunks
and score each doc by its best hit.
SQLite is unchanged: its vault search is LIKE on title/path only.
Chandan Kumar
65dbbcfd59