Skip to content

Stop using --strict with ASV benchmarks #5421

New issue
New issue
Closed
#5496
Closed
Stop using --strict with ASV benchmarks#5421
#5496
Labels
Type: PerformanceType: Testing
@trexfeathers

Description

@trexfeathers
trexfeathers
opened on Aug 14, 2023

The --strict parameter is being removed from ASV in the imminent v0.6 release (airspeed-velocity/asv#1205). We use this as parameter in our benchmark runs:

iris/benchmarks/bm_runner.py

Lines 29 to 32 in 3531f7b

ASV_HARNESS = (
"run {posargs} --attribute rounds=4 --interleave-rounds --strict "
"--show-stderr"
)

The good news: the parameter is being removed because it is now the default behaviour

The bad news: we need to stop making calls to strict, and also work out how to modify the times when we don't call strict:

iris/benchmarks/bm_runner.py

Lines 324 to 326 in 3531f7b

# C/SPerf benchmarks are much bigger than the CI ones:
# Don't fail the whole run if memory blows on 1 benchmark.
asv_command = asv_command.replace(" --strict", "")

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type: PerformanceType: Testing

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions