Skip to content

Conversation

yuqi1129
Copy link
Contributor

What changes were proposed in this pull request?

This PR only add the KV backend interface for EntityStore

Why are the changes needed?

Currently, We only support storing entity in memory and can't keep it in persistent storage which will be used latter

Fix: #122

Does this PR introduce any user-facing change?

No

How was this patch tested?

Not yet

@yuqi1129 yuqi1129 requested a review from jerryshao July 25, 2023 12:29

@Override
public <R> R executeInTransaction(Executable<R> executable) throws IOException {
lock.lock();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lock is only workable for embedded storage, maybe we should change to transactions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I have add this as a TODO item.

@jerryshao
Copy link
Contributor

Seems like you've moved the interface to storage module. I'm wondering how will you manage the code, will you also put rocksdb implementation in the storage module, if so, how will you manage the code if you have different storage implementations?

I was thinking of putting interfaces in core module storage package. And the rocksdb implementation in new storage-rocksdb module. With this, we could create a new module storage-xxx when we have new implementations.

@jerryshao jerryshao merged commit 8b7c919 into apache:main Jul 26, 2023
yuqi1129 pushed a commit that referenced this pull request Aug 3, 2023
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

This PR rethrow the real exception when function fails in transaction

### Why are the changes needed?

Transaction caller needs to handle the real exception
Fix: #165 

### Does this PR introduce _any_ user-facing change?

Transaction caller should handle the exception which generated by
`Executable`

### How was this patch tested?
UT modified.
jerryshao pushed a commit that referenced this pull request Nov 20, 2023
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Add diqiu50 to maintainer

### Why are the changes needed?

Fix: #755

### Does this PR introduce _any_ user-facing change?

No
### How was this patch tested?

No
xunliu added a commit that referenced this pull request Nov 26, 2023
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

(Please outline the changes and how this PR fixes the issue.)

### Why are the changes needed?

(Please clarify why the changes are needed. For instance,
  1. If you propose a new API, clarify the use case for a new API.
  2. If you fix a bug, describe the bug.)

Fix: # (issue)

### Does this PR introduce _any_ user-facing change?

(Please list the user-facing changes introduced by your change,
including
  1. Change in user-facing APIs.
  2. Addition or removal of property keys.)

### How was this patch tested?

(Please test your changes, and provide instructions on how to test it:
1. If you add a feature or fix a bug, add a test to cover your changes.
2. If you fix a flaky test, repeat it for many times to prove it works.)
jerryshao pushed a commit that referenced this pull request Feb 3, 2024
)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

close #1724 

### Why are the changes needed?

Add server-side REST API support for fileset

Fix: #1724 

### Does this PR introduce _any_ user-facing change?

(Please list the user-facing changes introduced by your change,
including
  1. Change in user-facing APIs.
  2. Addition or removal of property keys.)

### How was this patch tested?
- add unit tests
jerryshao pushed a commit that referenced this pull request Feb 4, 2024
…2043)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Unify the style of configuration options

### Why are the changes needed?

Fix: #2042 

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Just doc.

Co-authored-by: Heng Qin <qqtt@123.com>
github-actions bot pushed a commit that referenced this pull request Feb 4, 2024
…2043)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Unify the style of configuration options

### Why are the changes needed?

Fix: #2042 

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Just doc.

Co-authored-by: Heng Qin <qqtt@123.com>
qqqttt123 added a commit that referenced this pull request Feb 4, 2024
…2045)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Unify the style of configuration options

### Why are the changes needed?

Fix: #2042 

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Just doc.

Co-authored-by: qqqttt123 <148952220+qqqttt123@users.noreply.github.com>
Co-authored-by: Heng Qin <qqtt@123.com>
jerryshao pushed a commit that referenced this pull request Feb 8, 2024
…uri (#2136)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Change `gravitino.url` to `gravitino.uri`.

### Why are the changes needed?

The example in trino-connector/installation.md uses different keys for
gravitino link.


https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/docs/trino-connector/installation.md?plain=1#L75-L81

From source code, it should be `gravitino.uri`.


https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/trino-connector/src/main/java/com/datastrato/gravitino/trino/connector/GravitinoConfig.java#L22

We also need to fix the one in trino-connector/configuration.md.


https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/docs/trino-connector/configuration.md?plain=1#L13

Fix: #2118

### Does this PR introduce _any_ user-facing change?

N/A

### How was this patch tested?

Check there is no `gravitino.url` in the document.

Signed-off-by: PoAn Yang <payang@apache.org>
github-actions bot pushed a commit that referenced this pull request Feb 8, 2024
…uri (#2136)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Change `gravitino.url` to `gravitino.uri`.

### Why are the changes needed?

The example in trino-connector/installation.md uses different keys for
gravitino link.


https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/docs/trino-connector/installation.md?plain=1#L75-L81

From source code, it should be `gravitino.uri`.


https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/trino-connector/src/main/java/com/datastrato/gravitino/trino/connector/GravitinoConfig.java#L22

We also need to fix the one in trino-connector/configuration.md.


https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/docs/trino-connector/configuration.md?plain=1#L13

Fix: #2118

### Does this PR introduce _any_ user-facing change?

N/A

### How was this patch tested?

Check there is no `gravitino.url` in the document.

Signed-off-by: PoAn Yang <payang@apache.org>
qqqttt123 pushed a commit that referenced this pull request Feb 8, 2024
…uri (#2142)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Change `gravitino.url` to `gravitino.uri`.

### Why are the changes needed?

The example in trino-connector/installation.md uses different keys for
gravitino link.


https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/docs/trino-connector/installation.md?plain=1#L75-L81

From source code, it should be `gravitino.uri`.


https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/trino-connector/src/main/java/com/datastrato/gravitino/trino/connector/GravitinoConfig.java#L22

We also need to fix the one in trino-connector/configuration.md.


https://github.com/datastrato/gravitino/blob/edfcdbea89dbf16d8e48833573c8917f9bcd5972/docs/trino-connector/configuration.md?plain=1#L13

Fix: #2118

### Does this PR introduce _any_ user-facing change?

N/A

### How was this patch tested?

Check there is no `gravitino.url` in the document.

Signed-off-by: PoAn Yang <payang@apache.org>
Co-authored-by: PoAn Yang <payang@apache.org>
yuqi1129 added a commit that referenced this pull request Feb 11, 2024
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Dependencies follow alphabetical order

### Why are the changes needed?
For better readability
Fix: #738 
### Does this PR introduce _any_ user-facing change?
N/A
### How was this patch tested?

N/A

---------

Co-authored-by: Qi Yu <yuqi@datastrato.com>
FANNG1 pushed a commit that referenced this pull request Feb 16, 2024
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Replace LinkedList with ArrayList.

### Why are the changes needed?

Address linter issues mentioned in #2163.

Fix #2163

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Covered by existing tests.
yuqi1129 pushed a commit that referenced this pull request Feb 17, 2024
…#2138)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

close #1955

### Why are the changes needed?

(Please clarify why the changes are needed. For instance,
  1. If you propose a new API, clarify the use case for a new API.
  2. If you fix a bug, describe the bug.)

Fix: #1955 

### Does this PR introduce _any_ user-facing change?
- no

### How was this patch tested?

- unit
yuqi1129 pushed a commit that referenced this pull request Feb 20, 2024
…n atomic to keep thread safe (#2139)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

close #1953 #1946 #1941 #1920 #1910 

### Why are the changes needed?


Fix: #1953 #1946 #1941 #1920 #1910 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- ut
jerryshao pushed a commit that referenced this pull request Feb 26, 2024
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

- close #2154 

### Why are the changes needed?


- Fix: #2154 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

`./gradlew build` and UTs
mchades added a commit that referenced this pull request Mar 1, 2024
…2403)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

- close #2367 

### Why are the changes needed?


- Fix: #2367 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew build`

---------

Co-authored-by: mchades <liminghuang@datastrato.com>
FANNG1 pushed a commit that referenced this pull request Mar 5, 2024
…for consistency (#2437)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Make all Spark SQL keywords used in spark connector integration test
uppercase

### Why are the changes needed?

Fix: #2424 

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
`./gradlew build -PskipTests`
yuqi1129 added a commit that referenced this pull request Mar 14, 2024
…le clean (#2531)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Remove directory `spark-warehouse` when exectue gradle clean in moudle
catalog-lakehouse-iceberg.
Remove directory `src/main/resources` when execute gradle clean in
module server, since which is generated by gradle build.

### Why are the changes needed?

Fix: #2508

### Does this PR introduce _any_ user-facing change?

N/A

### How was this patch tested?

`./gradlew :catalogs:catalog-lakehouse-iceberg:clean`
`./gradlew :server:clean`

---------

Co-authored-by: Qi Yu <yuqi@datastrato.com>
diqiu50 pushed a commit that referenced this pull request Mar 18, 2024
…rdedByInstance error-prone (#2514)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

`StaticGuardedByInstance` amis to avoid `public static` fields. Because
this field is not thread-safe. See more details at
https://errorprone.info/bugpattern/StaticGuardedByInstance.

In this MR, I remove some test code in
`com.datastrato.gravitino.trino.connector.GravitinoPlugin` and
`com.datastrato.gravitino.trino.connector.GravitinoConnectorFactory`. At
the same time, I enable `StaticGuardedByInstance` error-prone.

### Why are the changes needed?

- Fix: #2167 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- original unit tests.
hdygxsj pushed a commit to hdygxsj/gravitino that referenced this pull request Jul 15, 2025
…expression (apache#7582)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

This is a refactor of builder method for `LiteralDTO`,
`FieldReferenceDTO`, `FuncExpressionDTO`, and `UnparsedExpressionDTO`
that is proposed in the
[comments](apache#7498 (review))
of the previous PR.

### Why are the changes needed?

We need to support Column and its default value in python client.

apache#5202

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Unit tests

---------

Signed-off-by: George T. C. Lai <tsungchih.hd@gmail.com>
hdygxsj pushed a commit to hdygxsj/gravitino that referenced this pull request Jul 15, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

clean up unused method in `NameIdentifierUtil` class

### Why are the changes needed?

close issue apache#7588

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

using existing tests
hdygxsj added a commit to hdygxsj/gravitino that referenced this pull request Jul 15, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Support table authorization.

### Why are the changes needed?

Fix: apache#7542

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?


`org.apache.gravitino.client.integration.test.authorization.TableAuthorizationIT`

---------

Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com>
Co-authored-by: 1161623489@qq.com <1161623489@qq.com>
hdygxsj added a commit to hdygxsj/gravitino that referenced this pull request Jul 15, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Support fileset auth with annotation.

### Why are the changes needed?

To support fileset auth.

Fix: apache#7545

### Does this PR introduce _any_ user-facing change?

Yes.

### How was this patch tested?

FilesetAuthorizationIT

---------

Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com>
Co-authored-by: 1161623489@qq.com <1161623489@qq.com>
hdygxsj added a commit to hdygxsj/gravitino that referenced this pull request Jul 15, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Support fileset auth with annotation.

### Why are the changes needed?

To support fileset auth.

Fix: apache#7545

### Does this PR introduce _any_ user-facing change?

Yes.

### How was this patch tested?

FilesetAuthorizationIT

---------

Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com>
Co-authored-by: 1161623489@qq.com <1161623489@qq.com>
hdygxsj added a commit to hdygxsj/gravitino that referenced this pull request Jul 15, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Support table authorization.

### Why are the changes needed?

Fix: apache#7542

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?


`org.apache.gravitino.client.integration.test.authorization.TableAuthorizationIT`

---------

Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com>
Co-authored-by: 1161623489@qq.com <1161623489@qq.com>
hdygxsj added a commit to hdygxsj/gravitino that referenced this pull request Jul 15, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Support fileset auth with annotation.

### Why are the changes needed?

To support fileset auth.

Fix: apache#7545

### Does this PR introduce _any_ user-facing change?

Yes.

### How was this patch tested?

FilesetAuthorizationIT

---------

Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com>
Co-authored-by: 1161623489@qq.com <1161623489@qq.com>
jerryshao pushed a commit that referenced this pull request Jul 15, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Support table authorization.

### Why are the changes needed?

Fix: #7542

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?


`org.apache.gravitino.client.integration.test.authorization.TableAuthorizationIT`

---------

Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com>
Co-authored-by: 1161623489@qq.com <1161623489@qq.com>
jerryshao pushed a commit that referenced this pull request Jul 15, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Support fileset auth with annotation.

### Why are the changes needed?

To support fileset auth.

Fix: #7545

### Does this PR introduce _any_ user-facing change?

Yes.

### How was this patch tested?

FilesetAuthorizationIT

---------

Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com>
Co-authored-by: 1161623489@qq.com <1161623489@qq.com>
yuqi1129 pushed a commit that referenced this pull request Jul 30, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

add StarRocks Catalog Implement

### Why are the changes needed?

To support StarRocks Catalog.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

By E2E test, the test is in another pr

#7792
jerqi pushed a commit to qqqttt123/gravitino that referenced this pull request Jul 30, 2025
…expression (apache#7582)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

This is a refactor of builder method for `LiteralDTO`,
`FieldReferenceDTO`, `FuncExpressionDTO`, and `UnparsedExpressionDTO`
that is proposed in the
[comments](apache#7498 (review))
of the previous PR.

### Why are the changes needed?

We need to support Column and its default value in python client.

apache#5202

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Unit tests

---------

Signed-off-by: George T. C. Lai <tsungchih.hd@gmail.com>
jerqi pushed a commit to qqqttt123/gravitino that referenced this pull request Jul 30, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

clean up unused method in `NameIdentifierUtil` class

### Why are the changes needed?

close issue apache#7588

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

using existing tests
jerqi pushed a commit to qqqttt123/gravitino that referenced this pull request Jul 30, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Support table authorization.

### Why are the changes needed?

Fix: apache#7542

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?


`org.apache.gravitino.client.integration.test.authorization.TableAuthorizationIT`

---------

Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com>
Co-authored-by: 1161623489@qq.com <1161623489@qq.com>
jerqi pushed a commit to qqqttt123/gravitino that referenced this pull request Jul 30, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Support fileset auth with annotation.

### Why are the changes needed?

To support fileset auth.

Fix: apache#7545

### Does this PR introduce _any_ user-facing change?

Yes.

### How was this patch tested?

FilesetAuthorizationIT

---------

Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com>
Co-authored-by: 1161623489@qq.com <1161623489@qq.com>
jerqi pushed a commit to qqqttt123/gravitino that referenced this pull request Jul 30, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

add StarRocks Catalog Implement

### Why are the changes needed?

To support StarRocks Catalog.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

By E2E test, the test is in another pr

apache#7792
tsungchih pushed a commit to tsungchih/gravitino that referenced this pull request Aug 2, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

add StarRocks Catalog Implement

### Why are the changes needed?

To support StarRocks Catalog.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

By E2E test, the test is in another pr

apache#7792
yuqi1129 pushed a commit that referenced this pull request Aug 4, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Fix the incorrect path in OpenAPI docs for list partitions.

### Why are the changes needed?

The existing path has a typo.
Fix: #7888 

### Does this PR introduce _any_ user-facing change?

Yes, user-facing OpenAPI doc.

### How was this patch tested?

Rendering `openapi.yaml`.

Signed-off-by: Yunchi Pang <yunchipang@gmail.com>
Co-authored-by: Yunchi Pang <yunchipang@gmail.com>
jerqi pushed a commit that referenced this pull request Aug 8, 2025
…th automatic token management (#7930)

[#7925] feat(oauth): implement OIDC authentication flow with automatic
token management

- Add OIDC login/logout components and event handlers
- Support automatic silent token renewal and session cleanup
- Gives provider agnostic login interface using OIDC

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

This PR implements a provider-agnostic OIDC authentication flow for the
Gravitino web frontend.

- Adds OIDC login and logout components using oidc-client-ts
- This will work with all standard providers (like Azure) who support
OIDC on top of OAuth2.

### Why are the changes needed?

- Current OAuth login is limited, where clients need to provide
client_id,secret and scope explicitly which means all users need to know
these sensitive parameters and it's not a smooth login experience. OIDC
helps with one-click login using the sign-on provider
- Enables secure, standards-based authentication for Gravitino using
OIDC
- Improves user experience with automatic token refresh and session
management
- Makes it easy to switch identity providers by updating configuration
only
Fix: #7925 

### Does this PR introduce _any_ user-facing change?
Yes:
- Users can now log in and log out using any OIDC-compliant provider
- The UI displays the authenticated user's name and status
- Authentication errors and logout messages are shown in the interface

### How was this patch tested?

Manual testing of login, logout, and token refresh flows with Azure AD
Verified that JWT tokens are stored and sent to the backend for
validation
Confirmed that session cleanup and error handling work as expected
- Tested backward compatibility, ie. when provider is set to "generic" ,
it still gives the old login UI with client_id,secret and scope and
backend configured with the default sign key. I used the [dummy oauth
server](https://gravitino.apache.org/docs/0.9.1/security/how-to-authenticate#example)
for testing this case.
- Used Azure AD based OAuth for testing the OIDC login workflow


New workflow with oidc when we use
gravitino.authenticator.oauth.provider = oidc


https://github.com/user-attachments/assets/eaa50d6f-1b56-4db0-8e6b-a46804b76f57

Old workflow works as well when we set
gravitino.authenticator.oauth.provider = default


https://github.com/user-attachments/assets/ce76b2a6-8ecf-4572-bde8-7c15b54863ad
yuqi1129 pushed a commit to yuqi1129/gravitino that referenced this pull request Aug 15, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

add StarRocks Catalog Implement

### Why are the changes needed?

To support StarRocks Catalog.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

By E2E test, the test is in another pr

apache#7792
yuqi1129 pushed a commit to yuqi1129/gravitino that referenced this pull request Aug 18, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[apache#123] feat(operator): support xxx"
     - "[apache#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[apache#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

add StarRocks Catalog Implement

### Why are the changes needed?

To support StarRocks Catalog.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

By E2E test, the test is in another pr

apache#7792
yuqi1129 pushed a commit that referenced this pull request Aug 18, 2025
…gOperations.java (#8132)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?
- Fixed a bug in the `equals` method where `scheme` was incorrectly
compared to `that.authority`.
  - Updated the logic to correctly compare `scheme` with `that.scheme`

### Why are the changes needed?
   This fix ensures object equality is consistent and accurate.  

Fix: #8110

### Does this PR introduce _any_ user-facing change?
  - No user-facing changes.  
  - This fix only corrects the internal equality logic.

### How was this patch tested?

Verified that objects with different `scheme` values are not considered
equal.
jerryshao pushed a commit that referenced this pull request Aug 18, 2025
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

add StarRocks Catalog Implement

### Why are the changes needed?

To support StarRocks Catalog.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

By E2E test, the test is in another pr

#7792
diqiu50 added a commit that referenced this pull request Aug 28, 2025
…or (#8246)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?
To fix possible NPEs in GravitinoConnector.java, added
Preconditions.checkNotNull for internalConnector and transaction handle
in beginTransaction method before their use.


### Why are the changes needed?
An Improvement to prevent possible NPEs in GravitinoConnector.java

Fix: #8198 

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?
Unit testing

---------

Co-authored-by: Justin Mclean <justin@classsoftware.com>
Co-authored-by: Yuhui <hui@datastrato.com>
yuqi1129 pushed a commit that referenced this pull request Sep 5, 2025
…8240)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Fixed the `DorisUtils.extractBucketNum()` method to properly handle
`BUCKETS AUTO` parsing for `DISTRIBUTED BY RANDOM` statements. The
method was incorrectly calling `matcher.find(5)` which advances the
matcher position, causing the bucket value extraction to fail and
resulting in a `NumberFormatException` when parsing null values.

**Changes:**
- Changed `if (matcher.find(5))` to `if (matcher.group(5) != null)` in
`DorisUtils.extractBucketNum()`
- Added test case for `DISTRIBUTED BY RANDOM BUCKETS AUTO` in
`TestDorisUtils.testDistributedInfoPattern()`

### Why are the changes needed?

This fixes a critical bug where Doris table operations fail when
encountering `DISTRIBUTED BY RANDOM BUCKETS AUTO` syntax. The root cause
was that `matcher.find(5)` advances the matcher position after the
initial match, and since there are no subsequent matches, it returns
`false` and never reaches the bucket value extraction logic. This leaves
`bucketValue` as null, causing `Integer.valueOf(bucketValue)` to throw
`NumberFormatException: Cannot parse null string`.

The bug affects users trying to load tables with auto bucket
distribution using random strategy, which is a valid Doris SQL syntax.

  Fix: #8218 

### Does this PR introduce _any_ user-facing change?

No user-facing API changes. This is a bug fix that enables proper
parsing of existing Doris SQL syntax that was previously failing. Users
can now successfully work with tables using `DISTRIBUTED BY RANDOM
BUCKETS AUTO` without encountering parsing errors.

### How was this patch tested?


1. **Existing tests**: Verified all existing Doris utility tests
continue to pass, ensuring no regression
2. **New test case**: Added
`TestDorisUtils.testDistributedInfoPattern()` test case specifically for
`DISTRIBUTED BY RANDOM BUCKETS AUTO` to prevent future regressions
3. **Manual verification**: Created and ran a standalone test to
reproduce the original bug and verify the fix resolves the
`NumberFormatException`
4. **Code formatting**: Applied Spotless formatting to ensure code style
compliance

**Test command:**
```
./gradlew :catalogs:catalog-jdbc-doris:test --tests="*TestDorisUtils*"
```

All tests pass successfully with the fix applied.
github-actions bot pushed a commit that referenced this pull request Sep 5, 2025
…8240)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Fixed the `DorisUtils.extractBucketNum()` method to properly handle
`BUCKETS AUTO` parsing for `DISTRIBUTED BY RANDOM` statements. The
method was incorrectly calling `matcher.find(5)` which advances the
matcher position, causing the bucket value extraction to fail and
resulting in a `NumberFormatException` when parsing null values.

**Changes:**
- Changed `if (matcher.find(5))` to `if (matcher.group(5) != null)` in
`DorisUtils.extractBucketNum()`
- Added test case for `DISTRIBUTED BY RANDOM BUCKETS AUTO` in
`TestDorisUtils.testDistributedInfoPattern()`

### Why are the changes needed?

This fixes a critical bug where Doris table operations fail when
encountering `DISTRIBUTED BY RANDOM BUCKETS AUTO` syntax. The root cause
was that `matcher.find(5)` advances the matcher position after the
initial match, and since there are no subsequent matches, it returns
`false` and never reaches the bucket value extraction logic. This leaves
`bucketValue` as null, causing `Integer.valueOf(bucketValue)` to throw
`NumberFormatException: Cannot parse null string`.

The bug affects users trying to load tables with auto bucket
distribution using random strategy, which is a valid Doris SQL syntax.

  Fix: #8218 

### Does this PR introduce _any_ user-facing change?

No user-facing API changes. This is a bug fix that enables proper
parsing of existing Doris SQL syntax that was previously failing. Users
can now successfully work with tables using `DISTRIBUTED BY RANDOM
BUCKETS AUTO` without encountering parsing errors.

### How was this patch tested?


1. **Existing tests**: Verified all existing Doris utility tests
continue to pass, ensuring no regression
2. **New test case**: Added
`TestDorisUtils.testDistributedInfoPattern()` test case specifically for
`DISTRIBUTED BY RANDOM BUCKETS AUTO` to prevent future regressions
3. **Manual verification**: Created and ran a standalone test to
reproduce the original bug and verify the fix resolves the
`NumberFormatException`
4. **Code formatting**: Applied Spotless formatting to ensure code style
compliance

**Test command:**
```
./gradlew :catalogs:catalog-jdbc-doris:test --tests="*TestDorisUtils*"
```

All tests pass successfully with the fix applied.
yuqi1129 pushed a commit that referenced this pull request Sep 5, 2025
…8447)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?

Fixed the `DorisUtils.extractBucketNum()` method to properly handle
`BUCKETS AUTO` parsing for `DISTRIBUTED BY RANDOM` statements. The
method was incorrectly calling `matcher.find(5)` which advances the
matcher position, causing the bucket value extraction to fail and
resulting in a `NumberFormatException` when parsing null values.

**Changes:**
- Changed `if (matcher.find(5))` to `if (matcher.group(5) != null)` in
`DorisUtils.extractBucketNum()`
- Added test case for `DISTRIBUTED BY RANDOM BUCKETS AUTO` in
`TestDorisUtils.testDistributedInfoPattern()`

### Why are the changes needed?

This fixes a critical bug where Doris table operations fail when
encountering `DISTRIBUTED BY RANDOM BUCKETS AUTO` syntax. The root cause
was that `matcher.find(5)` advances the matcher position after the
initial match, and since there are no subsequent matches, it returns
`false` and never reaches the bucket value extraction logic. This leaves
`bucketValue` as null, causing `Integer.valueOf(bucketValue)` to throw
`NumberFormatException: Cannot parse null string`.

The bug affects users trying to load tables with auto bucket
distribution using random strategy, which is a valid Doris SQL syntax.

  Fix: #8218 

### Does this PR introduce _any_ user-facing change?

No user-facing API changes. This is a bug fix that enables proper
parsing of existing Doris SQL syntax that was previously failing. Users
can now successfully work with tables using `DISTRIBUTED BY RANDOM
BUCKETS AUTO` without encountering parsing errors.

### How was this patch tested?


1. **Existing tests**: Verified all existing Doris utility tests
continue to pass, ensuring no regression
2. **New test case**: Added
`TestDorisUtils.testDistributedInfoPattern()` test case specifically for
`DISTRIBUTED BY RANDOM BUCKETS AUTO` to prevent future regressions
3. **Manual verification**: Created and ran a standalone test to
reproduce the original bug and verify the fix resolves the
`NumberFormatException`
4. **Code formatting**: Applied Spotless formatting to ensure code style
compliance

**Test command:**
```
./gradlew :catalogs:catalog-jdbc-doris:test --tests="*TestDorisUtils*"
```

All tests pass successfully with the fix applied.

Co-authored-by: Mathieu Baurin <86267428+mbaurin@users.noreply.github.com>
yuqi1129 pushed a commit that referenced this pull request Sep 7, 2025
…rgHiveCachedClientPool when closing multiple instances (#8455)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?
Add a null check before shutting down scheduledExecutorService in close
method.


### Why are the changes needed?
clientPoolCache is a static field, meaning it is shared across all
instances of IcebergHiveCachedClientPool.
Only the first instance that calls init() will initialize both
clientPoolCache and scheduledExecutorService.
Any subsequent instances will find clientPoolCache already initialized,
so they will not initialize scheduledExecutorService (it remains null
for those instances).When we create multiple instances of
IcebergHiveCachedClientPool, only the first instance will have a
non-null scheduledExecutorService.The other instances will have
scheduledExecutorService as null.
When we call close() on all instances, each will attempt to call
scheduledExecutorService.shutdownNow()
For instances where scheduledExecutorService is null, this results in a
NullPointerException.

Fix: (#8382 )

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
unit testing
github-actions bot pushed a commit that referenced this pull request Sep 7, 2025
…rgHiveCachedClientPool when closing multiple instances (#8455)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?
Add a null check before shutting down scheduledExecutorService in close
method.


### Why are the changes needed?
clientPoolCache is a static field, meaning it is shared across all
instances of IcebergHiveCachedClientPool.
Only the first instance that calls init() will initialize both
clientPoolCache and scheduledExecutorService.
Any subsequent instances will find clientPoolCache already initialized,
so they will not initialize scheduledExecutorService (it remains null
for those instances).When we create multiple instances of
IcebergHiveCachedClientPool, only the first instance will have a
non-null scheduledExecutorService.The other instances will have
scheduledExecutorService as null.
When we call close() on all instances, each will attempt to call
scheduledExecutorService.shutdownNow()
For instances where scheduledExecutorService is null, this results in a
NullPointerException.

Fix: (#8382 )

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
unit testing
yuqi1129 pushed a commit that referenced this pull request Sep 7, 2025
…rgHiveCachedClientPool when closing multiple instances (#8460)

<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
   Examples:
     - "[#123] feat(operator): support xxx"
     - "[#233] fix: check null before access result in xxx"
     - "[MINOR] refactor: fix typo in variable name"
     - "[MINOR] docs: fix typo in README"
     - "[#255] test: fix flaky test NameOfTheTest"
   Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->

### What changes were proposed in this pull request?
Add a null check before shutting down scheduledExecutorService in close
method.


### Why are the changes needed?
clientPoolCache is a static field, meaning it is shared across all
instances of IcebergHiveCachedClientPool.
Only the first instance that calls init() will initialize both
clientPoolCache and scheduledExecutorService.
Any subsequent instances will find clientPoolCache already initialized,
so they will not initialize scheduledExecutorService (it remains null
for those instances).When we create multiple instances of
IcebergHiveCachedClientPool, only the first instance will have a
non-null scheduledExecutorService.The other instances will have
scheduledExecutorService as null.
When we call close() on all instances, each will attempt to call
scheduledExecutorService.shutdownNow()
For instances where scheduledExecutorService is null, this results in a
NullPointerException.

Fix: (#8382 )

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
unit testing

Co-authored-by: Sambhavi Pandey <48976443+pandeysambhi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Subtask]Add KV backend storage interface for EntityStore
2 participants