Skip to content

[#554] improvement(CI): improve Hadoop access permission #620

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[#554] improvement(CI): improve Hadoop access permission #620

wants to merge 1 commit into from

Conversation

mchades
Copy link
Contributor

@mchades mchades commented Oct 27, 2023

What changes were proposed in this pull request?

  • set hdfs as HDFS superuser group in container
  • use datastrato as the Integration Test user instead of root

Why are the changes needed?

we should not use the root user directly to access HDFS

Fix: #554

Does this PR introduce any user-facing change?

no

How was this patch tested?

existing ITs

@github-actions
Copy link

github-actions bot commented Oct 27, 2023 •
edited
Loading

Code Coverage Report

Overall Project 65.89% 🟢

There is no coverage information present for the Files changed

@mchades
Copy link
Contributor Author

mchades commented Oct 27, 2023

This PR needs to rebase and publish new IT image version after #616 is merged

@jerryshao
Copy link
Contributor

@mchades #616 is merged, can you please continue your PR?

@mchades mchades self-assigned this Nov 6, 2023
@mchades
Copy link
Contributor Author

mchades commented Nov 6, 2023

@mchades #616 is merged, can you please continue your PR?

The PR #616 updated the Hive IT image to version 0.1.5 and modified the container network mode. However, this change has not been validated by Hive IT(still using 0.1.4). Therefore, this PR should be addressed after @xunliu resolving PR #639.

@mchades mchades mentioned this pull request Nov 8, 2023
Merged
@mchades mchades requested review from jerryshao and xunliu December 5, 2023 06:45
@mchades mchades marked this pull request as ready for review December 5, 2023 06:46
@mchades
Copy link
Contributor Author

mchades commented Dec 5, 2023

@xunliu Please help to review, thanks!

xunliu
xunliu reviewed Dec 5, 2023
View reviewed changes
xunliu
xunliu reviewed Dec 6, 2023
View reviewed changes
Copy link
Member

@xunliu xunliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @mchades Can you add test case for this PR, maybe like is.,

  1. Create table and insert data via Hive Client.
  2. List hdfs:/hive/db/table/xxx status, check file owner information via Hdfs Client.

@jerryshao
Copy link
Contributor

I will leave this to 0.4.0 if it is not blocked. @mchades you can still work on this if you have time, but the priority is lower than other documentation works.

@jerryshao jerryshao closed this Dec 13, 2023
@mchades mchades mentioned this pull request Dec 18, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xunliu xunliu xunliu left review comments

@jerryshao jerryshao Awaiting requested review from jerryshao

Assignees

@mchades mchades

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Improvement] User hive cannot access /tmp directory in IT container
3 participants
@mchades @jerryshao @xunliu