[#554] improvement(CI-Hive): improve Hadoop access permission #1194
Conversation
|
This PR, based on #620, adds owner validation for #620 (review). @xunliu Please review it again, thanks! |
|
|
||
| HADOOP_PACKAGE_NAME="hadoop-${HADOOP_VERSION}.tar.gz" # Must export this variable for Dockerfile | ||
| HADOOP_DOWNLOAD_URL="http://archive.apache.org/dist/hadoop/core/hadoop-${HADOOP_VERSION}/${HADOOP_PACKAGE_NAME}" | ||
|
|
||
| HIVE_PACKAGE_NAME="apache-hive-${HIVE_VERSION}-bin.tar.gz" # Must export this variable for Dockerfile | ||
| HIVE_DOWNLOAD_URL="https://archive.apache.org/dist/hive/hive-${HIVE_VERSION}/${HIVE_PACKAGE_NAME}" | ||
|
|
||
| JDBC_DIVER_PACKAGE_NAME="mysql-connector-java-${MYSQL_JDBC_DRIVER_VERSION}.tar.gz" # Must export this variable for Dockerfile | ||
| JDBC_DIVER_DOWNLOAD_URL="https://downloads.mysql.com/archives/get/p/3/file/${JDBC_DIVER_PACKAGE_NAME}" |
There was a problem hiding this comment.
Why do you use this version? We used another version in JDBC integration tests and playground.
https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.27/mysql-connector-java-8.0.27.jar
There was a problem hiding this comment.
We have been using this version from the beginning(see following ref) in Hive-CI-Docker, but the JDBC integration tests and the playground introduced later are inconsistent with the previous version.
There was a problem hiding this comment.
OK, we don't need to keep consistent in this pr. I guess we can change them to a consistent version in later pr.
|
Will you raise a follow-up pr to modify the Docker image which integration-test uses and the document? |
The Docker image The document also modified in this PR(see file |
|
Do you also need to ship a new hive image for users? cc @qqqttt123 |
No. This image is used by developers. |
What changes were proposed in this pull request?
hdfsas HDFS superuser group in containerdatastratoas the Hive catalog Integration Test user instead ofrootWhy are the changes needed?
we should not use the
rootuser directly to access HDFSFix: #554
Does this PR introduce any user-facing change?
no
How was this patch tested?
existing ITs