Incremental Compilation

KaaS uses incremental compilation to avoid redundant work — only new or changed content triggers reprocessing. This keeps compilation fast and cost-efficient as your knowledge base grows.

How It Works

When you submit content to KaaS, the system determines whether it has already been processed:

1. Content fingerprinting — each submission is hashed. If the hash matches a previously compiled item, recompilation is skipped.
2. Selective pipeline execution — only new or modified content enters the 4-phase pipeline (Extract → Classify → Write → Index).
3. Article merging — when new content relates to an existing wiki article, the Write phase merges new information into the existing article rather than creating a duplicate.
4. Incremental index updates — the master index is updated to reflect new or changed articles without rebuilding the entire index.

What Gets Preserved

When new content arrives:

• Existing wiki articles remain untouched unless new content directly relates to them.
• Previously built indexes are updated incrementally, not rebuilt from scratch.
• Cross-article links remain valid — new articles are woven into the existing link structure.

When Recompilation Triggers

Recompilation happens when:

• A new piece of content is submitted (text, file, or URL) that has not been seen before.
• Previously submitted content is resubmitted with modifications (different hash).

Recompilation does not trigger when:

• Identical content is submitted again.
• You browse or query the existing knowledge base.

Forcing Full Recompilation

In some cases you may want to rebuild the entire knowledge base from scratch — for example, after upgrading KaaS to a version with improved compilation logic.

To force a full recompilation, remove the compile state and resubmit:

Docker:

# Stop the service
docker stop kaas

# Remove compile state (wiki articles are preserved in data/kb/)
rm -f data/compile.db

# Restart — all content will be reprocessed on next submission
docker start kaas

CLI:

# Stop the service, then:
rm -f data/compile.db
kaas serve

TIP

After a forced recompilation, existing wiki articles are overwritten with freshly compiled versions. Back up data/kb/ first if you have manual edits you want to preserve.

Benefits

Aspect	Without Incremental	With Incremental
Speed	Full pipeline on every submission	Only new content processed
Cost	LLM calls for all content each time	LLM calls only for new/changed content
Stability	Articles may fluctuate between runs	Existing articles remain stable
Scale	Slows linearly with corpus size	Constant time per new submission