Skip to content

Refactored dataset saving for inference jobs #884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Feb 17, 2025
Merged

Refactored dataset saving for inference jobs #884

merged 4 commits into from
Feb 17, 2025

Conversation

aittalam
Copy link
Member

What's changing

Dataset saving is triggered inside the JobService only on inference jobs which explicitly provide a True store_to_dataset config variable.

Within an inference + evaluation workflow, inference jobs are called with an explicit store_to_dataset=False and dataset saving is immediately enqueued afterwards to guarantee the dataset is present before evaluation.

If this PR is related to an issue or closes one, please link it here.

Closes #877

How to test it

Steps to test the changes:

  1. run a full workflow on a dataset with GT
  2. verify that there's exactly one dataset generated by the inference job
  3. run an annotation job on a dataset without GT
  4. verify that there's exactly one dataset generated by the inference job

I already...

  • Tested the changes in a working environment to ensure they work as expected
  • Added some tests for any new functionality: there are no new functionalities, it's just a bug fix.
  • Updated the documentation: no need for updated prod documentation, but I updated comments in code
  • Checked if a (backend) DB migration step was required and included it if required: none required

@aittalam
Refactored dataset saving for inference jobs
6b1d9ae 
Dataset saving is triggered inside the JobService only on inference jobs which
explicitly provide a True `store_to_dataset` config variable.

Within an inference + evaluation workflow, inference jobs are called with an
explicit `store_to_dataset`=False and dataset saving is immediately enqueued
afterwards to guarantee the dataset is present before evaluation.
@aittalam aittalam linked an issue Feb 17, 2025 that may be closed by this pull request
[BUG]: a new workflow always generates two new datasets #877
Closed
1 task
@HareeshBahuleyan HareeshBahuleyan self-requested a review February 17, 2025 12:06
HareeshBahuleyan
HareeshBahuleyan approved these changes Feb 17, 2025
View reviewed changes
Copy link
Contributor

@HareeshBahuleyan HareeshBahuleyan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @aittalam for the fix! I tested the dataset creation and it works as per the steps 👍

@aittalam aittalam mentioned this pull request Feb 17, 2025
Open
@aittalam aittalam enabled auto-merge (squash) February 17, 2025 15:54
@aittalam aittalam merged commit b784985 into main Feb 17, 2025
15 checks passed
@aittalam aittalam deleted the 877-bug-double-datasets branch February 17, 2025 16:09
@ividal ividal mentioned this pull request Feb 17, 2025
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ividal ividal ividal left review comments

@HareeshBahuleyan HareeshBahuleyan HareeshBahuleyan approved these changes

@dpoulopoulos dpoulopoulos Awaiting requested review from dpoulopoulos

@njbrake njbrake Awaiting requested review from njbrake

Assignees
No one assigned
Labels
backend
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG]: a new workflow always generates two new datasets
3 participants
@aittalam @ividal @HareeshBahuleyan