Typesense Search + Sync

Optional Typesense backends, Athena table sync jobs, freshness monitoring, and Athena-routed search endpoints.

Athena can optionally project Postgres-backed table data into Typesense for low-latency search. The integration is fully opt-in: nothing changes until you apply the Typesense schema and configure at least one backend plus one sync job.

What gets provisioned

Apply sql/typesense.sql or the equivalent section in sql/provision.sql to the Athena logging database. That creates:

public.typesense_profiles for saved Typesense backends
public.typesense_sync_jobs for Athena table-to-collection bindings
public.typesense_sync_runs for run history and troubleshooting
public.typesense_sync_job_status for stale-state and health-oriented reads

These tables live in the logging database so the integration stays optional and operator-managed rather than becoming a hard runtime dependency for every client.

API routes

Athena exposes four Typesense routes:

POST /typesense/backends/test
POST /typesense/collections/list
POST /typesense/sync-jobs/{job_id}/run
POST /typesense/search

Use them to validate a Typesense node, inspect collections, trigger sync jobs manually, and execute search through a configured Athena binding.

Web app workflow

The web app exposes a dedicated Search page for the full operator flow:

Add a backend in Backends with Typesense base URL, API key, and timeout.
List collections to inspect the remote search cluster before binding data.
Create a Sync Job that maps an Athena client + schema + table into a Typesense collection.
Choose the document id column, included columns, queryable fields, sync interval, and stale threshold.
Review recent runs and stale status directly from the sync-job status view.
Use Live Search to query the bound collection through Athena.

On the first successful sync, Athena automatically creates the target Typesense collection when it does not already exist.

Freshness and staleness

Each sync job stores operational fields such as:

last_synced_at
next_sync_at
last_sync_status
last_sync_error
last_source_row_count
last_imported_count
last_failed_count

The typesense_sync_job_status view turns those into operator-friendly freshness signals such as is_stale and sync_health, which the web app uses to highlight jobs that need attention.

Search binding behavior

POST /typesense/search resolves a sync job either by:

explicit job_id, or
source_client_name + source_table_name (+ optional source_schema_name)

That means callers can keep using Athena concepts such as logical clients and table bindings while Athena routes the actual query to Typesense under the hood. If no matching enabled job exists, Athena returns a not-found style response.

Auth requirements

Typesense routes require either:

ATHENA_KEY_12 via X-Athena-Key or X-Athena-Admin-Key, or
a gateway API key with the gateway.typesense_proxy right

Authentication runs before JSON parsing, so invalid or missing credentials still return 401 Unauthorized even if the request body is malformed.

Worker behavior

Athena also starts an optional background sync worker that checks for due jobs and runs them automatically when enabled.

ATHENA_TYPESENSE_SYNC_WORKER_ENABLED
ATHENA_TYPESENSE_SYNC_WORKER_POLL_MS
ATHENA_TYPESENSE_ALLOW_HTTP

The worker is intended for scheduled freshness maintenance, while the manual run endpoint is useful for backfills, initial collection creation, or operator-driven recovery after a failed import.

Reference

See Reference -> Athena API Reference -> typesense for exact request and response payloads.