Technical Details

How it works

Every run works through the same five steps for every platform — AI classification only where a change was actually found.

1. 01
   Fetch

   Retrieves each platform's page by the most appropriate method: plain HTTP, JS-rendered headless browser, full Playwright automation, or PDF text extraction.

2. 02
   Compare

   Computes a SHA-256 fingerprint of the page content and compares outgoing links against the previous snapshot. Detects even minor edits.

3. 03
   Log

   Appends the result to an append-only audit log. Changed pages get a full diff with line counts and link deltas; unchanged pages get a brief status record.

4. 04
   AI Classify

   An LLM reviews each diff and classifies it as noise, cosmetic, link change, content update, or structural.

5. 05
   Commit

   Every run is permanently archived: a tamper-evident git commit records exactly what changed, when, its content fingerprint, and the AI classification. Full page snapshots are kept alongside on the server filesystem.

Fetcher tiers

25 platforms across 33 endpoints, including 5 bilingual EN+DE catalogue pairs — fetched with one of four tiers depending on how the page renders.

Under the hood

Runs are triggered manually via the admin panel or CLI, or run automatically on a set interval via the admin panel's built-in scheduler daemon (or systemd/cron on the server). Each run fetches every platform, compares against the previous version, runs AI classification on changes, and commits the result to git. Nothing is ever deleted or overwritten. Every change has a timestamp, a content fingerprint, and a line-by-line record of exactly what was different.