Skip to content

Conversation

njbrake
Copy link
Contributor

@njbrake njbrake commented Feb 6, 2025

What's changing

Since the inference job was created which has support for running the OpenAI and Mistral completion service, the /completions route is duplicating effort and increasing the complexity of the codebase. Lumigator is now designed to be an evaluation platform, not a proxy to other LLMs.

With this in mind, this PR deletes the completion route: inference with OpenAI and Mistral is still supported via running the inference job, so this PR does not remove functionilty, but rather consolidates our LLM inference logic into a single location (the inference job)

How to test it

The SDK had a connection to the completions/ route but the frontend didn't so the functionality appears to be unused. Unit tests and integration tests passing seem like enough, as long as we all agree on removing it.

I already...

  • Tested the changes in a working environment to ensure they work as expected
  • Added some tests for any new functionality
  • Updated the documentation (both comments in code and product documentation under /docs)
  • Checked if a (backend) DB migration step was required and included it if required

@github-actions github-actions bot added documentation Improvements or additions to documentation sdk backend api Changes which impact API/presentation layer schemas Changes to schemas (which may be public facing) labels Feb 6, 2025
@njbrake njbrake marked this pull request as ready for review February 6, 2025 13:27
Copy link
Contributor

@ividal ividal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This makes sense. As discussed, we may bring back live interacting with a model later on, but if/when that happens we can see if inference as it is already covers the need.

@njbrake njbrake merged commit f4c403d into main Feb 6, 2025
16 checks passed
@njbrake njbrake deleted the brake/deletecompletions branch February 6, 2025 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Changes which impact API/presentation layer backend documentation Improvements or additions to documentation schemas Changes to schemas (which may be public facing) sdk
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants