feat: add import api support for multiple groups with a single alphas #9381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

mangalaman93 merged 5 commits into main from shiva/import-stream

May 11, 2025

Contributor

shivaji-kharse commented Apr 15, 2025 •

edited

Loading

Description

This PR adds import APIs for multiple shard groups of Dgraph Alpha, but it does not support HA clusters or multi-shard HA clusters. It only works with a single Alpha or a multi-shard single Alpha cluster.
Checklist

Code compiles correctly and linting passes locally
For all code changes, an entry added to the CHANGELOG.md file describing and linking to
this PR
Tests added for new functionality, or regression tests for bug fixes added as applicable
For public APIs, new features, etc., PR on
docs repo staged and linked here

Copilot AI review requested due to automatic review settings

April 15, 2025 07:11

shivaji-kharse requested a review from a team as a code owner

April 15, 2025 07:11

Copilot AI reviewed

View reviewed changes

Copilot AI left a comment

Copilot reviewed 16 out of 18 changed files in this pull request and generated 2 comments.

Files not reviewed (2)

dgraphtest/.gitignore: Language not supported
go.mod: Language not supported

worker/import.go Outdated Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

shivaji-kharse force-pushed the shiva/import-stream branch from c273224 to 32ac8bc Compare

April 21, 2025 06:02

github-actions bot added area/testing area/core go labels

github-advanced-security bot found potential problems

View reviewed changes

dgraph/cmd/dgraphimport/import_client.go Fixed Show fixed Hide fixed

trunk-io bot commented Apr 21, 2025 •

edited

Loading

_{View Full Report ↗︎ ⋅ Docs}

shivaji-kharse force-pushed the shiva/import-stream branch 2 times, most recently from cb1c51b to 98d59d2 Compare

April 23, 2025 10:16

mangalaman93 requested changes

View reviewed changes

protos/pb.proto Outdated Show resolved Hide resolved

dgraphapi/cluster.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraphtest/load.go Outdated Show resolved Hide resolved

dgraphtest/load.go Show resolved Hide resolved

dgraphtest/local_cluster.go Outdated Show resolved Hide resolved

dgraphtest/local_cluster.go Outdated Show resolved Hide resolved

dgraphtest/local_cluster.go Outdated Show resolved Hide resolved

dgraphtest/local_cluster.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Show resolved Hide resolved

shivaji-kharse force-pushed the shiva/import-stream branch from 98d59d2 to d4235b1 Compare

April 28, 2025 10:59

github-actions bot added the area/bulk-loader label

shivaji-kharse force-pushed the shiva/import-stream branch from d4235b1 to 5316b62 Compare

April 28, 2025 11:55

mangalaman93 changed the title ~~feat: add import api support for multiple groups with a single alphs~~ feat: add import api support for multiple groups with a single alphas

shivaji-kharse force-pushed the shiva/import-stream branch from 1fdaf06 to 4bc25f0 Compare

April 29, 2025 05:22

mangalaman93 requested changes

View reviewed changes

dgraph/cmd/dgraphimport/import_client.go Show resolved Hide resolved

dgraph/cmd/bulk/run.go Outdated Show resolved Hide resolved

worker/import.go Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

mangalaman93 requested changes

View reviewed changes

dgraph/cmd/dgraphimport/import_client.go Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

mangalaman93 requested changes

View reviewed changes

worker/import.go Outdated Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

worker/import.go Show resolved Hide resolved

worker/import.go Show resolved Hide resolved

shivaji-kharse force-pushed the shiva/import-stream branch 3 times, most recently from 13603e8 to 46e67d9 Compare

May 5, 2025 05:00

mangalaman93 requested changes

View reviewed changes

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

worker/import.go Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

shivaji-kharse force-pushed the shiva/import-stream branch from 3a9553d to 96c5e77 Compare

May 7, 2025 08:37

xqqp suggested changes

View reviewed changes

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

mangalaman93 requested changes

View reviewed changes

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

worker/import.go Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

worker/import.go Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved

mangalaman93 reviewed

View reviewed changes

Member

mangalaman93 left a comment

LGTM!

harshil-goel reviewed

View reviewed changes

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Outdated Show resolved Hide resolved

dgraph/cmd/dgraphimport/import_client.go Show resolved Hide resolved

worker/import.go Show resolved Hide resolved

worker/import.go Outdated Show resolved Hide resolved


          add import api support for multiple groups with a single alphs

ca8c843

shivaji-kharse added 3 commits

May 11, 2025 18:18


          resolve review comments

8d8fdbe


          resolve review comments

e2f31a0


          fix trunk

50eb712

shivaji-kharse force-pushed the shiva/import-stream branch from af4f733 to d36633d Compare

May 11, 2025 12:48

mangalaman93 approved these changes

View reviewed changes

mangalaman93 enabled auto-merge (squash)

May 11, 2025 13:01


          resolve review comments

c801bc7

shivaji-kharse force-pushed the shiva/import-stream branch from d36633d to c801bc7 Compare

May 11, 2025 13:06

mangalaman93 merged commit 24cb3f3 into main

13 checks passed

mangalaman93 deleted the shiva/import-stream branch

May 11, 2025 14:38

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/bulk-loader area/core area/testing go