Skip to content

Conversation

yunchipang
Copy link
Contributor

What changes were proposed in this pull request?

Move the schema-stubbing action to its corresponding test.

Why are the changes needed?

In setUp() of TestHadoopCatalogOperations, we stub schema files in a fixed‐length loop. As we add new tests that require additional schemas to be stubbed (e.g. schema30, schema31, ...) , this manual update of the loop's upper bound is inefficient and throws misleading test failures.

Fix: #7280

Does this PR introduce any user-facing change?

No

How was this patch tested?

by TestHadoopCatalogOperations itself

@@ -610,6 +606,7 @@ public void testAlterSchema() throws IOException {

@Test
public void testDropSchema() throws IOException {
stubSchema(20L);
String name = "schema20";
String comment = "comment20";
String catalogPath = TEST_ROOT_PATH + "/" + "catalog20";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mchades I was thinking about further extracting the schema id (in this case, 20) into a variable, so updating it automatically adjusts all related name, comment, catalogPath, etc. wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good

@mchades mchades changed the title [#7280] refactor: localize schema stubbing in TestHadoopCatalogOperations [#7280] refactor(UT): localize schema stubbing in TestHadoopCatalogOperations Jun 5, 2025
@mchades mchades added the branch-0.9 Automatically cherry-pick commit to branch-0.9 label Jun 5, 2025
@yunchipang yunchipang requested a review from mchades June 5, 2025 20:53
Copy link
Contributor

@mchades mchades left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plz resolve the code conflicts

Comment on lines 618 to 619
stubSchema(testId);
Schema schema = createSchema(schemaName, comment, catalogPath, null);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about moving the stubSchema call into createSchema method?

@mchades mchades removed the branch-0.9 Automatically cherry-pick commit to branch-0.9 label Jun 6, 2025
Schema schema = createSchema(testId);
Fileset fileset = createFileset(testId);
final NameIdentifier filesetIdent =
NameIdentifier.of("m1", "c1", schema.name(), fileset.name());
Copy link
Contributor Author

@yunchipang yunchipang Jun 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mchades I've overloaded createSchema and createFileset to take only a testId and stub the schema inside createSchema. The change is only applied to testListFilesetFilesWithNonExistentPath for now. Does this approach seem reasonable, or does the extra overload introduce too much complexity? Also, would switching to an idGenerator instead of manually assigning testId be a better option?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The primary purpose of this PR is to enable developers to write unit tests more easily by automatically generating IDs, allowing them to focus on the test logic rather than managing IDs manually. You can consider using the idGenerator if it achieves this goal.

doReturn(new SchemaIds(1L, 1L, testId))
.when(spySchemaMetaService)
.getSchemaIdByMetalakeNameAndCatalogNameAndSchemaName(
Mockito.anyString(), Mockito.anyString(), Mockito.eq(name));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mchades thanks for the comment. I've moved the schema-stubbing logic into createSchema so we no longer need to call stubSchema in every test. generateTestId func is added to utilize idGenerator to auto-generate test ids without manually defining a testId. Tests have all passes locally, let me know if you have any other suggestions on the design. thanks!

@mchades mchades closed this Jun 11, 2025
@mchades mchades reopened this Jun 11, 2025
@mchades
Copy link
Contributor

mchades commented Jun 11, 2025

Overall, it's LGTM.

This branch cannot be rebased due to conflicts. Could you plz fix it?

@yunchipang yunchipang force-pushed the refactor/localize-schema-stubbing branch from fad251d to fa5ddba Compare June 11, 2025 19:35
@yunchipang yunchipang force-pushed the refactor/localize-schema-stubbing branch from fa5ddba to ae757b9 Compare June 11, 2025 19:42
@yunchipang
Copy link
Contributor Author

@mchades rebased done. please take a look. thanks!

@mchades mchades added the branch-0.9 Automatically cherry-pick commit to branch-0.9 label Jun 12, 2025
@mchades mchades merged commit 9be5422 into apache:main Jun 12, 2025
30 checks passed
github-actions bot pushed a commit that referenced this pull request Jun 12, 2025
…erations (#7335)

### What changes were proposed in this pull request?

Move the schema-stubbing action to its corresponding test.

### Why are the changes needed?

In `setUp()` of `TestHadoopCatalogOperations`, we stub schema files in a
fixed‐length loop. As we add new tests that require additional schemas
to be stubbed (e.g. schema30, schema31, ...) , this manual update of the
loop's upper bound is inefficient and throws misleading test failures.

Fix: #7280 

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

by `TestHadoopCatalogOperations` itself
@yunchipang yunchipang deleted the refactor/localize-schema-stubbing branch June 12, 2025 15:28
jerryshao pushed a commit to jerryshao/gravitino that referenced this pull request Jul 8, 2025
…alogOperations (apache#7335)

### What changes were proposed in this pull request?

Move the schema-stubbing action to its corresponding test.

### Why are the changes needed?

In `setUp()` of `TestHadoopCatalogOperations`, we stub schema files in a
fixed‐length loop. As we add new tests that require additional schemas
to be stubbed (e.g. schema30, schema31, ...) , this manual update of the
loop's upper bound is inefficient and throws misleading test failures.

Fix: apache#7280 

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

by `TestHadoopCatalogOperations` itself
vishnu-chalil pushed a commit to vishnu-chalil/gravitino that referenced this pull request Jul 14, 2025
…alogOperations (apache#7335)

### What changes were proposed in this pull request?

Move the schema-stubbing action to its corresponding test.

### Why are the changes needed?

In `setUp()` of `TestHadoopCatalogOperations`, we stub schema files in a
fixed‐length loop. As we add new tests that require additional schemas
to be stubbed (e.g. schema30, schema31, ...) , this manual update of the
loop's upper bound is inefficient and throws misleading test failures.

Fix: apache#7280 

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

by `TestHadoopCatalogOperations` itself
hdygxsj pushed a commit to hdygxsj/gravitino that referenced this pull request Jul 15, 2025
…alogOperations (apache#7335)

### What changes were proposed in this pull request?

Move the schema-stubbing action to its corresponding test.

### Why are the changes needed?

In `setUp()` of `TestHadoopCatalogOperations`, we stub schema files in a
fixed‐length loop. As we add new tests that require additional schemas
to be stubbed (e.g. schema30, schema31, ...) , this manual update of the
loop's upper bound is inefficient and throws misleading test failures.

Fix: apache#7280 

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

by `TestHadoopCatalogOperations` itself
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-0.9 Automatically cherry-pick commit to branch-0.9
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Improvement] refactor schema stubbing in TestHadoopCatalogOperations
2 participants