Skip to content

Check action performance with lot of commits #71

@josecelano

Description

@josecelano

The queue loads the messages from the git log output:

private async loadMessagesFromGit(): Promise<void> {
  await this.guardThatGitRepoHasBeenInitialized()

  const noCommits = !(await this.gitRepo.hasCommits()) ? true : false

  if (noCommits) {
    return
  }

  const logCommits = await this.gitRepo.log()

  const allMessages = CommittedMessageLog.fromGitLogCommits(logCommits.all)

  this.committedMessages = allMessages.filterCommitsByQueue(this.name)
}

I suppose SimpleGit loads all commits in memory when you use logCommits.all that could be a problem. I do not expect it to be too slow or too big for very big repositories with a lot of commits but at least we should know the limit. We could improve the process (if needed) with:

  • Not loading all commits in memory and processing and filtering them on the fly.
  • Stop processing commits once we load the active job (one-job version of the queue) or all active jobs (multiple-active-jobs version of the queue)
  • Maybe, if we modify the nextJob() or we even add a new method to get a given job getJob(jobId: JobId) we might need to re-process the git log to find the information for the missing job if we do not have it in memory. In this case, the CommitMessageLog attribute of the queue would be only a cache of the full queue information.

I'm going to add this issue to the roadmap for a future release. This issue is not intended to be the solution for the potential bottleneck problem. If we decide to do something we can discuss the solution and create a new issue for the implementation.

@da2ce7 @yeraydavidrodriguez

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions