Add archivist incremental sync state

Request:
Add archivist support for remembering which IPO archive stages have already been synced and which stages should be updated next.

Changes:
- Add sync_runs, ticker_sync_state, sync_tasks, and price_performance tables to the archive schema.
- Add scripts/update_sync_state.py to derive per-ticker stage status and rebuild the next-sync task queue.
- Export the new sync-state tables as Git-friendly CSV snapshots.
- Document the incremental archive flow in the archivist skill and README.

Verification:
- Ran scripts/bootstrap_historical_data.py.
- Ran scripts/update_sync_state.py with a deterministic as-of timestamp.
- Checked SQLite integrity and DB-to-snapshot row counts with Python sqlite3.
- Parsed Python scripts with ast.parse.
- Ran git diff --check and checked for temporary SQLite/cache files.

Next useful context:
- Current derived queue has 2 open tasks for 06658 and 15 waiting_until_due tasks for future stages.
This commit is contained in:
2026-06-15 06:29:54 +00:00
parent fb715cd446
commit 08db218b6d
10 changed files with 601 additions and 3 deletions
+4
View File
@@ -303,8 +303,12 @@ def main() -> None:
"ipo_master",
"offering_terms",
"ipo_demand",
"price_performance",
"source_refs",
"data_gaps",
"sync_runs",
"ticker_sync_state",
"sync_tasks",
]:
export_snapshot(conn, table)