Skip to content

Implement tests on inference scenarios for models #1076

@Mr-Geekman

Description

@Mr-Geekman

🚀 Feature Request

We want to have tests that check different inference scenarios for models.

Proposal

Add tests in test_inference on:

  1. Forecast on subset of segments:
  • Check that forecast on subset of segments is equal to subset of forecast on all segments.
  1. Predict on subset of segments.
  • Check that predict on subset of segments is equal to subset of predict on all segments.
  1. Forecast on new segments.
  • Check that forecast fills the target without fail.
  1. Predict on new segments.
  • Check that predict fills the target without fail (target will be already filled, but we want to check that it doesn't fail).
  1. Predict on future data with gap (we can try to do it like with forecast: TestForecastOutSamplePrefix,
    TestForecastOutSampleSuffix)
  • Check that prediction on prefix of future dataset is the same as prefix of prediction on bigger future dataset.
  • Check that prediction on suffix of future dataset is the same as suffix of prediction on bigger future dataset.

If some tests are failed for some models mark this tests as to_be_fixed if we want to fix them. If we understand that it isn't possible to fix this (model's properties) make a separate test case for such models.

We have already tested

  1. How forecast works on data in future with gap: TestForecastOutSampleSuffix.
  2. How predict works on data right after train: TestPredictOutSample.

Test cases

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions