-
Notifications
You must be signed in to change notification settings - Fork 83
Implement tests on inference scenarios for models #1076
Copy link
Copy link
Closed
Description
🚀 Feature Request
We want to have tests that check different inference scenarios for models.
Proposal
Add tests in test_inference
on:
- Forecast on subset of segments:
- Check that forecast on subset of segments is equal to subset of forecast on all segments.
- Predict on subset of segments.
- Check that predict on subset of segments is equal to subset of predict on all segments.
- Forecast on new segments.
- Check that forecast fills the target without fail.
- Predict on new segments.
- Check that predict fills the target without fail (target will be already filled, but we want to check that it doesn't fail).
- Predict on future data with gap (we can try to do it like with
forecast
:TestForecastOutSamplePrefix
,
TestForecastOutSampleSuffix
)
- Check that prediction on prefix of future dataset is the same as prefix of prediction on bigger future dataset.
- Check that prediction on suffix of future dataset is the same as suffix of prediction on bigger future dataset.
If some tests are failed for some models mark this tests as to_be_fixed
if we want to fix them. If we understand that it isn't possible to fix this (model's properties) make a separate test case for such models.
We have already tested
- How
forecast
works on data in future with gap:TestForecastOutSampleSuffix
. - How
predict
works on data right after train:TestPredictOutSample
.
Test cases
No response
Additional context
No response
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Type
Projects
Status
Done