[SPARK-22487][SQL][Hive]Remove the unused HIVE_EXECUTION_VERSION property #19712
Conversation
|
ok to test |
|
Test build #83663 has finished for PR 19712 at commit
|
|
cc again @gatorsmile and would you mind adding me to the jenkins' white list? thanks, hoping not bother you. |
| conf += resultSet.getString(1) -> resultSet.getString(2) | ||
| } | ||
|
|
||
| assert(conf.get("spark.sql.hive.version") === Some("1.2.1")) |
There was a problem hiding this comment.
change it to spark.sql.hive.metastore.version?
There was a problem hiding this comment.
this might need to set spark.sql.hive.metastore.version explicitly
There was a problem hiding this comment.
the first commit fails this ut checking spark.sql.hive.metastore.version. set cmd only shows the changed variables, if more unit tests are needed, i can add some.
|
add to whitelist |
|
Test build #83668 has finished for PR 19712 at commit
|
|
Test build #83667 has finished for PR 19712 at commit
|
|
@yaooqinn it would be good if you can include why this config was introduced at the beginning, in the PR description, thanks |
|
retest this please |
|
@cloud-fan referenced |
|
Test build #83675 has finished for PR 19712 at commit
|
|
LGTM, merging to master! |
|
After talking with @liancheng offline, this is still being used for ODBC connection. @cloud-fan Could you revert it back? |
## What changes were proposed in this pull request? a followup of #19712 , adds back the `spark.sql.hive.version`, so that if users try to read this config, they can still get a default value instead of null. ## How was this patch tested? N/A Author: Wenchen Fan <wenchen@databricks.com> Closes #19719 from cloud-fan/minor.
What changes were proposed in this pull request?
At the beginning #2843 added
spark.sql.hive.versionto reveal underlying hive version for jdbc connections. For some time afterwards, it was used as a version identifier for the execution hive client.Actually there is no hive client for executions in spark now and there are no usages of HIVE_EXECUTION_VERSION found in whole spark project. HIVE_EXECUTION_VERSION is set by
spark.sql.hive.version, which is still set internally in some places or by users, this may confuse developers and users with HIVE_METASTORE_VERSION(spark.sql.hive.metastore.version).It might better to be removed.
How was this patch tested?
modify some existing ut
cc @cloud-fan @gatorsmile