feat: `Sidequest.build` from job #20

GiovaniGuizzo · 2025-07-28T22:48:25Z

Checklist for Pull Requests

All tests pass (yarn test:all)
Code follows the style guide and passes lint checks
Documentation is updated (README, docs, etc)
Linked to corresponding issue, if applicable

Summary of Changes

Creates the LazyBackend for lazy loading backend only on the first call (used by engine for lazy loading)
Injects Sidequest config when running a job. This allows us to enqueue jobs from within other jobs. It also enables using other backend stuff.

Closes #14

…eConfig; add dynamic Sidequest import

Copilot

Pull Request Overview

This PR introduces the ability to build Sidequest from within job execution environments and enables lazy loading of backend connections.

Creates a LazyBackend class that defers backend initialization until first use
Modifies the engine to inject Sidequest configuration into job contexts
Refactors ExecutorManager and related components to accept full configuration objects

Reviewed Changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
rollup.config.base.js	Adds optionalDependencies to external packages list
packages/engine/src/workers/main.ts	Simplifies ExecutorManager instantiation by passing full config
packages/engine/src/utils/index.ts	Exports new import utility module
packages/engine/src/utils/import.ts	Adds dynamic import function for sidequest module
packages/engine/src/shared-runner/runner.ts	Injects Sidequest config into job execution and updates runner signature
packages/engine/src/shared-runner/runner.test.ts	Updates tests to use new runner interface and adds config injection tests
packages/engine/src/shared-runner/runner-pool.ts	Passes configuration to worker pool and updates constructor
packages/engine/src/shared-runner/runner-pool.test.ts	Updates tests for new RunnerPool constructor signature
packages/engine/src/execution/executor-manager.ts	Refactors to accept full config object instead of individual parameters
packages/engine/src/execution/executor-manager.test.ts	Updates tests for ExecutorManager config changes
packages/engine/src/execution/dispatcher.test.ts	Updates dispatcher tests to use new ExecutorManager signature
packages/engine/src/engine.ts	Implements LazyBackend and moves queue configuration to start method
packages/engine/src/engine.test.ts	Removes queue configuration tests that moved to start method
packages/engine/package.json	Adds sidequest as peer dependency
packages/backends/backend/src/lazy-backend.ts	Implements LazyBackend proxy class for deferred backend initialization
packages/backends/backend/src/lazy-backend.test.ts	Comprehensive tests for LazyBackend functionality
packages/backends/backend/src/index.ts	Exports LazyBackend class
CHANGELOG.md	Formatting changes to bullet points

packages/backends/backend/src/lazy-backend.ts

packages/engine/src/execution/dispatcher.test.ts

packages/engine/src/shared-runner/runner.test.ts

packages/backends/backend/src/lazy-backend.test.ts

…Backend and Dispatcher tests

packages/engine/package.json

…sistency

…dequestConfig

# [1.0.0-next.15](v1.0.0-next.14...v1.0.0-next.15) (2025-07-30) ### Bug Fixes * author ([ec1734a](ec1734a)) * author ([e81372f](e81372f)) * Proper dashboard and Sidequest shutdown ([#16](#16)) ([ed1d853](ed1d853)) * release ([4b2f4e7](4b2f4e7)) * release ([#26](#26)) ([c6093ae](c6093ae)) * release.yml ([c37c57c](c37c57c)) * revert unlimited jobs ([#23](#23)) ([96ccc5d](96ccc5d)), closes [#21](#21) * Unhandled error on Executor Manager ([#24](#24)) ([81bd5e0](81bd5e0)) * using personal token ([#25](#25)) ([91c5834](91c5834)) * using sidequest release app ([1ecf1a7](1ecf1a7)) ### Features * `Sidequest.build` from job ([#20](#20)) ([d7f5286](d7f5286)) * unlimited jobs ([#21](#21)) ([0055f4a](0055f4a))

sidequest-release · 2025-07-30T01:15:16Z

🎉 This PR is included in version 1.0.0-next.15 🎉

The release is available on:

v1.0.0-next.15
GitHub release

Your semantic-release bot 📦🚀

# 1.0.0 (2025-08-01) ### Bug Fixes * add close method to Engine class for backend cleanup ([90ece50](90ece50)) * add engines field to specify required Node.js version ([#42](#42)) ([44c9523](44c9523)) * author ([ec1734a](ec1734a)) * author ([e81372f](e81372f)) * changed to patch and fixed job path call on table ([6e2399e](6e2399e)) * claim ([b535e98](b535e98)) * deduplication ([#58](#58)) ([e0d6ae7](e0d6ae7)) * default job state ([626882e](626882e)) * enqueue ([a1cc0aa](a1cc0aa)) * ensure consistent path formatting in buildPath function ([55de875](55de875)) * escape backslashes in file paths for consistency ([36cc48c](36cc48c)) * fix build of dashboard on windows ([e92f15d](e92f15d)) * Fix Dashboard stats and graph, and removes Redis ([#154](https://github.com/sidequestjs/sidequest/issues/154)) ([b3ab912](b3ab912)) * fix second shutdown attempt ([295f6b4](295f6b4)) * fixed a few inconsistencies ([54d338d](54d338d)) * fixed time range ([fb19cac](fb19cac)) * fixed yarn lock ([#140](https://github.com/sidequestjs/sidequest/issues/140)) ([68176ec](68176ec)) * format ([c51ecf6](c51ecf6)) * Future canceled jobs not running ([#34](#34)) ([85a5212](85a5212)) * improve job failure handling in RetryTransition ([#143](https://github.com/sidequestjs/sidequest/issues/143)) ([f448c35](f448c35)) * including migrations on backends ([a670e89](a670e89)) * make config parameter optional in Engine.start method ([2359040](2359040)) * min concurrency ([#28](#28)) ([5e9877b](5e9877b)) * new version ([c104052](c104052)) * Proper dashboard and Sidequest shutdown ([#16](#16)) ([ed1d853](ed1d853)) * publish ([17d036b](17d036b)) * publish to npm ([477e9a2](477e9a2)) * queue list ([b09d103](b09d103)) * release ([4b2f4e7](4b2f4e7)) * release ([d731dd2](d731dd2)) * release ([b736f50](b736f50)) * release ([#26](#26)) ([c6093ae](c6093ae)) * release next ([66b9329](66b9329)) * release next ([dd9f2af](dd9f2af)) * release next ([cc4b2bd](cc4b2bd)) * release next ([586ec83](586ec83)) * release.yml ([c37c57c](c37c57c)) * remove topological ([f25b881](f25b881)) * remove unused delete button from job view ([#36](#36)) ([eedd15f](eedd15f)) * removing returning usage from sql backend ([5d61c29](5d61c29)) * removing version from monorepo ([e4c5a76](e4c5a76)) * removing version script ([e7736a4](e7736a4)) * revert unlimited jobs ([#23](#23)) ([96ccc5d](96ccc5d)), closes [#21](#21) * sending config to sidequest main ([44a314c](44a314c)) * serialize errors ([#44](#44)) ([683b7e1](683b7e1)) * show more and show less not saving state on page refresh ([#41](#41)) ([fcc99e7](fcc99e7)) * states ([81be666](81be666)) * test ([1da410e](1da410e)) * test ([8444aef](8444aef)) * Unhandled error on Executor Manager ([#24](#24)) ([81bd5e0](81bd5e0)) * update default concurrency value in documentation ([dfc328d](dfc328d)) * update jobDefaults handling and add engine tests ([#4](#4)) ([bbd8dde](bbd8dde)) * update logger import to use @sidequest/core ([d1f7825](d1f7825)) * update staleJobs test to handle negative timeout values ([514b330](514b330)) * url ([74651c2](74651c2)) * using exec ([80e47be](80e47be)) * using personal token ([#25](#25)) ([91c5834](91c5834)) * using sidequest release app ([1ecf1a7](1ecf1a7)) * version ([be0e7d4](be0e7d4)) * versions ([903c601](903c601)) * versions ([abd5a80](abd5a80)) * yarn lock ([d9cdc68](d9cdc68)) ### Features * `Sidequest.build` from job ([#20](#20)) ([d7f5286](d7f5286)) * add better logging and tests for createBackendFromDriver function ([#142](https://github.com/sidequestjs/sidequest/issues/142)) ([00942cb](00942cb)) * add comprehensive test suite for job and queue management ([a775de8](a775de8)) * add job builder ([#39](#39)) ([5af07bb](5af07bb)) * add JobData to job object. ([#171](https://github.com/sidequestjs/sidequest/issues/171)) ([c61df01](c61df01)) * add nullable timeout column to sidequest_jobs table ([b2bacbd](b2bacbd)) * add timeout on executor start ([5c0164d](5c0164d)) * add updateJob method and related tests for job management ([6f7a024](6f7a024)) * added cli ([442b031](442b031)) * added dasy ui ([59411e9](59411e9)) * added job view - work in progress ([e274af6](e274af6)) * added mongodb backend ([#155](https://github.com/sidequestjs/sidequest/issues/155)) ([0149d82](0149d82)) * allow nested jobs ([#55](#55)) ([a8ff7f0](a8ff7f0)) * allowing to set maxAttempts and availabeAt on enqueuing job. ([#77](#77)) ([c469da2](c469da2)) * basic auth on dashboard ([#66](#66)) ([add020b](add020b)) * cancel job ([8434c70](8434c70)) * cleanup job and coverage ([#76](#76)) ([f58f2d0](f58f2d0)) * dashboard config ([e6741d2](e6741d2)) * enhance queue management with defaults and force update options ([#5](#5)) ([40acbec](40acbec)) * Facades and better transitions ([#161](https://github.com/sidequestjs/sidequest/issues/161)) ([8066f5d](8066f5d)) * implement backend initialization and configuration for dashboard ([34737b9](34737b9)) * implement staleJobs method for job management across backends ([a10959a](a10959a)) * improve DX ([#71](#71)) ([075b589](075b589)) * job flow control ([#42](#42)) ([f7dd600](f7dd600)) * job list ([5208e7c](5208e7c)) * job view ([31f6112](31f6112)) * mock shared-runner for improved test isolation in main.test.ts ([a1bfac2](a1bfac2)) * move magic numbers to Sidequest config ([#147](https://github.com/sidequestjs/sidequest/issues/147)) ([f701f1a](f701f1a)) * moving args to run function ([225bee3](225bee3)) * realising staled claimed and running jobs ([#59](#59)) ([148f64c](148f64c)) * recurring jobs ([#12](#12)) ([cd66663](cd66663)) * redis backend ([#129](https://github.com/sidequestjs/sidequest/issues/129)) ([f08e9b7](f08e9b7)) * refactor backend import paths and introduce setTestBackend function ([f5439da](f5439da)) * refactor uniquiness ([#67](#67)) ([fbea055](fbea055)) * removed dev server ([ffe153d](ffe153d)) * rerun jobs and fixing small issues ([6e2925c](6e2925c)) * routines ([6abf0d4](6abf0d4)) * run from dashboard ([b65b058](b65b058)) * unlimited jobs ([#21](#21)) ([0055f4a](0055f4a)) * Update branch triggers to include 'develop' for workflows ([#1](#1)) ([eef8a35](eef8a35)) * Update job arguments type to array in SidequestDashboard ([a920c27](a920c27)) ### Reverts * Revert "Chore/fix env ([#166](https://github.com/sidequestjs/sidequest/issues/166))" ([#167](https://github.com/sidequestjs/sidequest/issues/167)) ([6254959](6254959))

sidequest-release · 2025-08-01T14:55:49Z

🎉 This PR is included in version 1.0.0 🎉

The release is available on:

v1.0.0
GitHub release

Your semantic-release bot 📦🚀

GiovaniGuizzo added 2 commits July 28, 2025 18:54

feat: Implement LazyBackend class and add tests for its functionality

39dd3e1

feat: Refactor ExecutorManager and RunnerPool to use NonNullableEngin…

2a52bac

…eConfig; add dynamic Sidequest import

GiovaniGuizzo requested a review from Copilot July 28, 2025 22:48

GiovaniGuizzo requested a review from merencia as a code owner July 28, 2025 22:48

Copilot AI reviewed Jul 28, 2025

View reviewed changes

GiovaniGuizzo added 2 commits July 28, 2025 19:53

fix: Update test comments for clarity and adjust import paths in Lazy…

eeeaa1a

…Backend and Dispatcher tests

feat: Add peerDependencies for sidequest in yarn.lock

cf3d9da

merencia reviewed Jul 28, 2025

View reviewed changes

packages/engine/package.json Show resolved Hide resolved

GiovaniGuizzo changed the title ~~feat: Sidequest.build from job~~ feat: Sidequest.build from job Jul 28, 2025

GiovaniGuizzo added 2 commits July 28, 2025 20:14

feat: Add sidequest module declaration with configure method

400eb86

fix: Update branch name and PR title validation to use printf for con…

7fd406a

…sistency

GiovaniGuizzo changed the title ~~feat: Sidequest.build from job~~ feat: Sidequest.build from job Jul 28, 2025

fix: Correctly destructure Sidequest from importSidequest in injectSi…

ba1bc3e

…dequestConfig

GiovaniGuizzo requested a review from merencia July 29, 2025 12:22

GiovaniGuizzo force-pushed the feat/enqueue-from-job branch from 73a27bc to ba1bc3e Compare July 29, 2025 13:38

Merge branch 'develop' into feat/enqueue-from-job

8858b94

merencia approved these changes Jul 29, 2025

View reviewed changes

GiovaniGuizzo merged commit d7f5286 into develop Jul 29, 2025
2 checks passed

GiovaniGuizzo deleted the feat/enqueue-from-job branch July 29, 2025 14:01

sidequest-release bot added the released on @next label Jul 30, 2025

sidequest-release bot added the released label Aug 1, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: `Sidequest.build` from job #20

feat: `Sidequest.build` from job #20

Uh oh!

GiovaniGuizzo commented Jul 28, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sidequest-release bot commented Jul 30, 2025

Uh oh!

sidequest-release bot commented Aug 1, 2025

Uh oh!

Uh oh!

Uh oh!

feat: Sidequest.build from job #20

feat: Sidequest.build from job #20

Uh oh!

Conversation

GiovaniGuizzo commented Jul 28, 2025

Checklist for Pull Requests

Summary of Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sidequest-release bot commented Jul 30, 2025

Uh oh!

sidequest-release bot commented Aug 1, 2025

Uh oh!

Uh oh!

feat: `Sidequest.build` from job #20

feat: `Sidequest.build` from job #20