geometrybase/hk-ipo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add archivist incremental sync state

Browse Source 
Request:
Add archivist support for remembering which IPO archive stages have already been synced and which stages should be updated next.

Changes:
- Add sync_runs, ticker_sync_state, sync_tasks, and price_performance tables to the archive schema.
- Add scripts/update_sync_state.py to derive per-ticker stage status and rebuild the next-sync task queue.
- Export the new sync-state tables as Git-friendly CSV snapshots.
- Document the incremental archive flow in the archivist skill and README.

Verification:
- Ran scripts/bootstrap_historical_data.py.
- Ran scripts/update_sync_state.py with a deterministic as-of timestamp.
- Checked SQLite integrity and DB-to-snapshot row counts with Python sqlite3.
- Parsed Python scripts with ast.parse.
- Ran git diff --check and checked for temporary SQLite/cache files.

Next useful context:
- Current derived queue has 2 open tasks for 06658 and 15 waiting_until_due tasks for future stages.
This commit is contained in:
geometrybase
2026-06-15 06:29:54 +00:00
parent fb715cd446
commit 08db218b6d
10 changed files with 601 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
data/snapshots/sync_tasks.csv
+18
View File
@@ -0,0 +1,18 @@
task_id,ticker,stage,task_type,task_status,due_date,data_gap_id,last_sync_run_id,updated_at,notes
06658_D1,06658,D1,archive_price_performance,open,2026-06-15,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06658_T2_grey_market,06658,T2_grey_market,archive_grey_market_result,open,2026-06-15,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06675_T1_allotment,06675,T1_allotment,archive_allotment_results,waiting_until_due,2026-06-16,06675_allotment_results_pending_2026_06_15,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Allotment result facts are not archived yet.
06675_D1,06675,D1,archive_price_performance,waiting_until_due,2026-06-17,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06675_T2_grey_market,06675,T2_grey_market,archive_grey_market_result,waiting_until_due,2026-06-17,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06658_D5,06658,D5,archive_price_performance,waiting_until_due,2026-06-19,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06675_D5,06675,D5,archive_price_performance,waiting_until_due,2026-06-21,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06106_T1_allotment,06106,T1_allotment,archive_allotment_results,waiting_until_due,2026-06-23,06106_allotment_results_pending_2026_06_15,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Allotment result facts are not archived yet.
06106_D1,06106,D1,archive_price_performance,waiting_until_due,2026-06-24,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06106_T2_grey_market,06106,T2_grey_market,archive_grey_market_result,waiting_until_due,2026-06-24,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06106_D5,06106,D5,archive_price_performance,waiting_until_due,2026-06-28,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06658_D20,06658,D20,archive_price_performance,waiting_until_due,2026-07-04,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06675_D20,06675,D20,archive_price_performance,waiting_until_due,2026-07-06,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06106_D20,06106,D20,archive_price_performance,waiting_until_due,2026-07-13,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06658_D60,06658,D60,archive_price_performance,waiting_until_due,2026-08-13,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06675_D60,06675,D60,archive_price_performance,waiting_until_due,2026-08-15,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06106_D60,06106,D60,archive_price_performance,waiting_until_due,2026-08-22,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
1 task_id ticker stage task_type task_status due_date data_gap_id last_sync_run_id updated_at notes
2 06658_D1 06658 D1 archive_price_performance open 2026-06-15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
3 06658_T2_grey_market 06658 T2_grey_market archive_grey_market_result open 2026-06-15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
4 06675_T1_allotment 06675 T1_allotment archive_allotment_results waiting_until_due 2026-06-16 06675_allotment_results_pending_2026_06_15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Allotment result facts are not archived yet.
5 06675_D1 06675 D1 archive_price_performance waiting_until_due 2026-06-17 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
6 06675_T2_grey_market 06675 T2_grey_market archive_grey_market_result waiting_until_due 2026-06-17 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
7 06658_D5 06658 D5 archive_price_performance waiting_until_due 2026-06-19 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
8 06675_D5 06675 D5 archive_price_performance waiting_until_due 2026-06-21 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
9 06106_T1_allotment 06106 T1_allotment archive_allotment_results waiting_until_due 2026-06-23 06106_allotment_results_pending_2026_06_15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Allotment result facts are not archived yet.
10 06106_D1 06106 D1 archive_price_performance waiting_until_due 2026-06-24 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
11 06106_T2_grey_market 06106 T2_grey_market archive_grey_market_result waiting_until_due 2026-06-24 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
12 06106_D5 06106 D5 archive_price_performance waiting_until_due 2026-06-28 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
13 06658_D20 06658 D20 archive_price_performance waiting_until_due 2026-07-04 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
14 06675_D20 06675 D20 archive_price_performance waiting_until_due 2026-07-06 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
15 06106_D20 06106 D20 archive_price_performance waiting_until_due 2026-07-13 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
16 06658_D60 06658 D60 archive_price_performance waiting_until_due 2026-08-13 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
17 06675_D60 06675 D60 archive_price_performance waiting_until_due 2026-08-15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
18 06106_D60 06106 D60 archive_price_performance waiting_until_due 2026-08-22 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.