geometrybase/hk-ipo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add archivist incremental sync state

Browse Source 
Request:
Add archivist support for remembering which IPO archive stages have already been synced and which stages should be updated next.

Changes:
- Add sync_runs, ticker_sync_state, sync_tasks, and price_performance tables to the archive schema.
- Add scripts/update_sync_state.py to derive per-ticker stage status and rebuild the next-sync task queue.
- Export the new sync-state tables as Git-friendly CSV snapshots.
- Document the incremental archive flow in the archivist skill and README.

Verification:
- Ran scripts/bootstrap_historical_data.py.
- Ran scripts/update_sync_state.py with a deterministic as-of timestamp.
- Checked SQLite integrity and DB-to-snapshot row counts with Python sqlite3.
- Parsed Python scripts with ast.parse.
- Ran git diff --check and checked for temporary SQLite/cache files.

Next useful context:
- Current derived queue has 2 open tasks for 06658 and 15 waiting_until_due tasks for future stages.
This commit is contained in:
geometrybase
2026-06-15 06:29:54 +00:00
parent fb715cd446
commit 08db218b6d
10 changed files with 601 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
data/snapshots/ticker_sync_state.csv
+22
View File
@@ -0,0 +1,22 @@
ticker,stage,status,required,due_date,completed_at,last_source_id,data_gap_id,last_sync_run_id,updated_at,notes
06106,T0_prospectus,complete,1,2026-06-15,2026-06-15,06106_prospectus_candidate_2026_06_15,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Prospectus source and offering terms are archived.
06106,T1_allotment,pending_not_due,1,2026-06-23,,,06106_allotment_results_pending_2026_06_15,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Allotment result facts are not archived yet.
06106,T2_grey_market,pending_not_due,1,2026-06-24,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06106,D1,pending_not_due,1,2026-06-24,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06106,D5,pending_not_due,1,2026-06-28,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06106,D20,pending_not_due,1,2026-07-13,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06106,D60,pending_not_due,1,2026-08-22,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06658,T0_prospectus,complete,1,2026-06-05,2026-06-05,06658_prospectus_2026_06_05,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Prospectus source and offering terms are archived.
06658,T1_allotment,complete,1,2026-06-12,2026-06-12,06658_allotment_results_2026_06_12,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Allotment result facts are archived.
06658,T2_grey_market,pending_due,1,2026-06-15,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06658,D1,pending_due,1,2026-06-15,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06658,D5,pending_not_due,1,2026-06-19,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06658,D20,pending_not_due,1,2026-07-04,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06658,D60,pending_not_due,1,2026-08-13,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06675,T0_prospectus,complete,1,2026-06-09,2026-06-09,06675_prospectus_2026_06_09,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Prospectus source and offering terms are archived.
06675,T1_allotment,pending_not_due,1,2026-06-16,,,06675_allotment_results_pending_2026_06_15,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Allotment result facts are not archived yet.
06675,T2_grey_market,pending_not_due,1,2026-06-17,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06675,D1,pending_not_due,1,2026-06-17,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet.
06675,D5,pending_not_due,1,2026-06-21,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06675,D20,pending_not_due,1,2026-07-06,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
06675,D60,pending_not_due,1,2026-08-15,,,,sync_state_seed_2026_06_15,2026-06-15T06:30:00Z,Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
1 ticker stage status required due_date completed_at last_source_id data_gap_id last_sync_run_id updated_at notes
2 06106 T0_prospectus complete 1 2026-06-15 2026-06-15 06106_prospectus_candidate_2026_06_15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Prospectus source and offering terms are archived.
3 06106 T1_allotment pending_not_due 1 2026-06-23 06106_allotment_results_pending_2026_06_15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Allotment result facts are not archived yet.
4 06106 T2_grey_market pending_not_due 1 2026-06-24 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
5 06106 D1 pending_not_due 1 2026-06-24 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
6 06106 D5 pending_not_due 1 2026-06-28 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
7 06106 D20 pending_not_due 1 2026-07-13 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
8 06106 D60 pending_not_due 1 2026-08-22 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
9 06658 T0_prospectus complete 1 2026-06-05 2026-06-05 06658_prospectus_2026_06_05 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Prospectus source and offering terms are archived.
10 06658 T1_allotment complete 1 2026-06-12 2026-06-12 06658_allotment_results_2026_06_12 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Allotment result facts are archived.
11 06658 T2_grey_market pending_due 1 2026-06-15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
12 06658 D1 pending_due 1 2026-06-15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
13 06658 D5 pending_not_due 1 2026-06-19 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
14 06658 D20 pending_not_due 1 2026-07-04 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
15 06658 D60 pending_not_due 1 2026-08-13 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
16 06675 T0_prospectus complete 1 2026-06-09 2026-06-09 06675_prospectus_2026_06_09 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Prospectus source and offering terms are archived.
17 06675 T1_allotment pending_not_due 1 2026-06-16 06675_allotment_results_pending_2026_06_15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Allotment result facts are not archived yet.
18 06675 T2_grey_market pending_not_due 1 2026-06-17 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
19 06675 D1 pending_not_due 1 2026-06-17 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet.
20 06675 D5 pending_not_due 1 2026-06-21 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
21 06675 D20 pending_not_due 1 2026-07-06 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.
22 06675 D60 pending_not_due 1 2026-08-15 sync_state_seed_2026_06_15 2026-06-15T06:30:00Z Price/performance source is not archived yet. Due date uses calendar days until trading-calendar support is added.