Add archivist incremental sync state
Request: Add archivist support for remembering which IPO archive stages have already been synced and which stages should be updated next. Changes: - Add sync_runs, ticker_sync_state, sync_tasks, and price_performance tables to the archive schema. - Add scripts/update_sync_state.py to derive per-ticker stage status and rebuild the next-sync task queue. - Export the new sync-state tables as Git-friendly CSV snapshots. - Document the incremental archive flow in the archivist skill and README. Verification: - Ran scripts/bootstrap_historical_data.py. - Ran scripts/update_sync_state.py with a deterministic as-of timestamp. - Checked SQLite integrity and DB-to-snapshot row counts with Python sqlite3. - Parsed Python scripts with ast.parse. - Ran git diff --check and checked for temporary SQLite/cache files. Next useful context: - Current derived queue has 2 open tasks for 06658 and 15 waiting_until_due tasks for future stages.
This commit is contained in:
@@ -303,8 +303,12 @@ def main() -> None:
|
||||
"ipo_master",
|
||||
"offering_terms",
|
||||
"ipo_demand",
|
||||
"price_performance",
|
||||
"source_refs",
|
||||
"data_gaps",
|
||||
"sync_runs",
|
||||
"ticker_sync_state",
|
||||
"sync_tasks",
|
||||
]:
|
||||
export_snapshot(conn, table)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user