Skip to content

Conversation

qqqttt123
Copy link
Contributor

@qqqttt123 qqqttt123 commented Dec 21, 2023

What changes were proposed in this pull request?

Now, ConfigEntry lacks the ability of check config options. So we add the support of this feature referring to Spark.

Why are the changes needed?

Fix: #1221

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Add new ut.

@qqqttt123 qqqttt123 requested review from Clearvive, jerryshao, FANNG1, xunliu, diqiu50, yuqi1129 and mchades and removed request for jerryshao December 21, 2023 10:09
@jerryshao
Copy link
Contributor

Probably be better to add the checkValue to all the configurations that can be checked, not only the string configurations.

@qqqttt123
Copy link
Contributor Author

Probably be better to add the checkValue to all the configurations that can be checked, not only the string configurations.

OK, I will try to add more checks.

@qqqttt123
Copy link
Contributor Author

Probably be better to add the checkValue to all the configurations that can be checked, not only the string configurations.

OK, I will try to add more checks.

Added.

@qqqttt123 qqqttt123 self-assigned this Dec 22, 2023
StringUtils.isNotBlank(icebergConfig.get(IcebergConfig.CATALOG_URI)),
"Catalog uri can't be blank");
// The method `get` will call the method `checkValue`, just check the values of the config
// options
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not proper to add the comment here. It explains the behavior of config::get, unrelated to iceberg here.

Copy link
Contributor Author

@qqqttt123 qqqttt123 Dec 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I will remove 'The method get will call the method checkValue'. But it's also confusing if we only call the method get, but we don't use it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got your point. I will change the comment

The method `get` will check the values of config options.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

package com.datastrato.gravitino.config;

public interface ConfigConstants {
String NOT_BLANK_ERROR_MSG = "The value can't be the blank";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"The value can't be blank"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

if (validator == null) {
return;
}
optionValue.ifPresent(value -> validator.accept(value));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please optimize the logic here? It is a bit confusing to have both null and optional check here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if optionValue is empty, we will skip the check.

Copy link
Contributor Author

@qqqttt123 qqqttt123 Dec 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add some comments here.

// If optionValue is empty, the config entry will skip the check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link
Contributor

@jerryshao jerryshao Dec 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can change like this:

Stream.of(Optional.ofNullable(validator), optionValue).allMatch(Optional::isPresent) {
validator.accept(value)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

package com.datastrato.gravitino.config;

public interface ConfigConstants {
String NOT_BLANK_ERROR_MSG = "The value can't be blank";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using 'Missing config' can more clearly indicate the error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you configure " ", the validator will report error, too.

@@ -42,8 +42,7 @@ public class IcebergTableOps implements AutoCloseable {
public IcebergTableOps(IcebergConfig icebergConfig) {
String catalogType = icebergConfig.get(IcebergConfig.CATALOG_BACKEND);
if (!IcebergCatalogBackend.MEMORY.name().equalsIgnoreCase(catalogType)) {
// The method `get` will call the method `checkValue`, just check the values of the config
// options
// The method `get` will check the values of config options.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't have to add the comments here, it's not necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@@ -45,8 +45,7 @@ private static JdbcCatalog loadJdbcCatalog(Map<String, String> properties) {
IcebergConfig icebergConfig = new IcebergConfig(properties);
String driverClassName = icebergConfig.getJdbcDriver();

// The method `get` will call the method `checkValue`, just check the values of the config
// options
// The method `get` will check the values of config options.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neither here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@jerryshao jerryshao merged commit f2a4708 into apache:main Dec 22, 2023
@qqqttt123 qqqttt123 deleted the ISSUE-1221 branch December 22, 2023 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Improvement] Config should support checkValue.
3 participants