How it works¶

The code¶

Package	What's in it
`core/`	Settings, the Celery app, health checks, rate limits, security headers, the security log, caching
`feed/`	Feeds and entries: fetching (`ingest.py`), search (`search.py`), "more like this" (`related.py`), retention, OPML and Atom
`library/`	Bookmarks, saved searches and email digests
`users/`	The custom user model and sign-up rules (django-allauth, email login)
`pages/`	Static pages, `robots.txt` and the sitemap
`templates/`	All templates; htmx is vendored in `static/js/`
`deploy/`	The entrypoint, update, backup and docs scripts, and Caddyfiles

Fetching feeds¶

Celery beat starts a refresh every 30 minutes. The schedules live in the database (django-celery-beat), so they are edited in the admin, not in code. Each feed is fetched with a conditional GET, so an unchanged feed costs one small request. New articles are then downloaded and their text extracted with trafilatura.

Every download has a size cap and an overall deadline, and every Celery task has a time limit. Failing feeds back off exponentially.

An article belongs to the first feed that saw it. Links that differ only trivially (tracking parameters, http/https, www., a trailing slash) count as the same article, and canonical links merge duplicates across feeds.

Search¶

Each entry has a stored tsvector column with a GIN index. The text-search configuration is chosen per feed language: English, Swedish, or simple. A query is parsed with websearch_to_tsquery in all three configurations, OR'ed together, so it matches articles in any of the languages.

Ranking weighs title over summary over body.
Snippets come from ts_headline, which is the costly part of a search, so it only runs where snippets are shown. Matches are marked with control characters, and the template escapes the text before turning them into <mark>.
"Did you mean" compares the query's words with a materialized view of the words in the index (ts_stat, with fuzzystrmatch).
Queries are cut to a maximum length before any of this runs.

Caching¶

Anonymous visitors get cached pages. The cache is keyed on a "content changed" marker that ingestion and admin edits bump, so a page is never older than the content it shows. Atom feeds answer conditional GETs from the same marker. If Redis goes down, the site keeps working without the cache (and without rate limits), and staff are emailed.

Security¶

Outgoing requests go through an SSRF guard that checks every redirect and the address actually connected to, so feeds can only reach public addresses. OPML is parsed with defusedxml.
Content-Security-Policy allows scripts and styles only from the site's own static files.
Rate limits cover searching, search feeds and feed suggestions; allauth limits sign-in attempts and locks out repeated failures.
Staff must use an authenticator app or a passkey before they reach the admin, whose path is secret.
The security log records sign-ins, failures and rate-limit hits by account id, in the logs and in the database (read-only in the admin).
In production the app runs as an unprivileged user on a read-only filesystem and connects to a database role that isn't a superuser.

Privacy¶

The site stores as little as it can. Search statistics are per-day counts per query, never who searched. Readers can download everything stored about them and delete their account. The site's privacy page lists exactly what is kept, for how long, and which cookies are set.

Monitoring¶

/healthz answers Docker's and Caddy's health checks. It also checks heartbeats from the feed refresh and the daily backup, and emails ADMINS when one is late. That check runs from the web app, not Celery, so it still works when Celery is down.