fix(ci): merge all duration slices, not one
Each test slice uploads an artifact with the same file name, test_durations.json. The save-durations job downloaded the 12 artifacts with merge-multiple, so all extractions wrote to one path in parallel. This caused two faults: - A race between two extractions wrote two JSON documents into one file. The merge step then failed with 'JSONDecodeError: Extra data' (run 31382130252). - On green runs, the last write erased the other 11 slices. The merged cache held ~230 of ~2760 file durations. Remove merge-multiple so each artifact extracts into its own directory, and point the glob at durations/*/test_durations.json. A local merge of the 12 real artifacts from the failed run gives 2761 durations.
This commit is contained in:
@@ -155,18 +155,22 @@ jobs:
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Download all slice durations
|
||||
# Each slice uploads the same file name (test_durations.json).
|
||||
# With merge-multiple, the parallel downloads write to one path.
|
||||
# This causes two problems: a race can write two JSON documents
|
||||
# into one file, and the last write erases the other slices.
|
||||
# Without merge-multiple, each artifact gets its own directory.
|
||||
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
||||
with:
|
||||
pattern: test-durations-slice-*
|
||||
path: durations
|
||||
merge-multiple: true
|
||||
|
||||
- name: Merge into single durations file
|
||||
run: |
|
||||
python3 -c "
|
||||
import json, glob, os
|
||||
merged = {}
|
||||
for f in glob.glob('durations/*test_durations.json'):
|
||||
for f in glob.glob('durations/*/test_durations.json'):
|
||||
with open(f) as fh:
|
||||
merged.update(json.load(fh))
|
||||
with open('test_durations.json', 'w') as fh:
|
||||
|
||||
Reference in New Issue
Block a user