Skip to content

Conversation

jerryshao
Copy link
Contributor

What changes were proposed in this pull request?

This PR proposes to implement the schema metadata and type system in Java and Protobuf.

Why are the changes needed?

This PR defines a core metadata system for unified catalog.

Does this PR introduce any user-facing change?

NA

How was this patch tested?

This PR adds UT to cover the schema definitions.

@jerryshao jerryshao requested review from JunpingDu and xunliu May 16, 2023 10:05
@jerryshao jerryshao self-assigned this May 16, 2023
@jerryshao jerryshao linked an issue May 16, 2023 that may be closed by this pull request
Field.required("audit_info", AuditInfo.class, "The audit info of the lakehouse");

@JsonProperty("id")
private Long id;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this Long type, whether need to be unified to Int(or Long) like other schema types?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have strong feelings about using the same type or not. My original thinking is that there might not be so many entities for some types like "tenant" in one database, using integer should be enough.

public class Lakehouse implements Entity, Auditable {

public static final Field ID =
Field.required("id", Long.class, "The unique identifier of the lakehouse");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this Long.class type, whether need to be unified to Int.class like other schema ID? For example Tenant.ID.

xunliu
xunliu previously approved these changes May 17, 2023
@jerryshao
Copy link
Contributor Author

@xunliu PTAL, thanks.

Copy link
Member

@xunliu xunliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

./gradlew build
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 62076  100 62076    0     0  83773      0 --:--:-- --:--:-- --:--:-- 83660
Starting a Gradle Daemon, 1 incompatible Daemon could not be reused, use --status for details

BUILD SUCCESSFUL in 39s
22 actionable tasks: 20 executed, 1 from cache, 1 up-to-date

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Subtask] Metadata and Type Spec Implementation
2 participants