Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: graphframes/graphframes
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.8.2
Choose a base ref
...
head repository: graphframes/graphframes
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.8.3
Choose a head ref
  • 9 commits
  • 33 files changed
  • 5 contributors

Commits on Oct 16, 2021

  1. Setting version to 0.8.3-SNAPSHOT

    @mengxr
    mengxr committed Oct 16, 2021
    Configuration menu
    Copy the full SHA
    375547a View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2021

  1. Use official Spark 3.2 PyPI release in CI (#404) 

    * add sbt creds to simplify release

* use official pypi 3.2 release
    @mengxr
    mengxr authored Oct 18, 2021
    Configuration menu
    Copy the full SHA
    4a5b3d9 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2022

  1. Upgrade to latest spark versions 3.3.0, 3.2.2 and 3.1.3 (#415) 

    * Upgrade to latest spark versions

Add CI build for spark 3.3.0

Upgrade spark 3.2.0 to 3.2.2

Upgrade spark 3.1.2 to 3.1.3

* Share single SparkContext for all tests

The previous setup would cause test to end up using a stopped spark
context causing them to fail. This seems to be due to some change in
spark 3.3.0, by making the whole suite using a single SparkContext this
issue goes away.
    Emil Ejbyfeldt authored Aug 25, 2022
    1 Configuration menu
    Copy the full SHA
    100fb01 View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2023

  1. Python: Replace SQLContext with SparkSession (#424) 

    * GraphFrame: Use SparkSession instead of SQLContext

* ci: Remove spark 2.4.8

Spark 2.4.x does not support SparkSession.getActiveSession().  According to https://spark.apache.org/versioning-policy.html, spark 2.4.x is EOL and the last version was released 5 years ago.  There's little meaning to support it.
    William Tang authored Feb 15, 2023
    Configuration menu
    Copy the full SHA
    1e1dbd3 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2023

  1. Update CI actions and dependencies (#425) 

    * ci: Update spark versions

* ci: Update ubuntu to 22.04

* ci: Update actions

* ci: Use openjdk 11

* ci: Use python 3.9

* python: Update nose to 1.3.7
    William Tang authored Feb 16, 2023
    Configuration menu
    Copy the full SHA
    82f7979 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2023

  1. Add support for scala 2.13 (#417) 

    Only adds scala CI builds as I don't belive there is any pre-built
pyspark release using scala 2.13.
    Emil Ejbyfeldt authored Feb 22, 2023
    Configuration menu
    Copy the full SHA
    3d76998 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2023

  1. Remove deprecated sqlContext in graphframe scala code (#432) 

    * update

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update tests

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update guide

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

* update example

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>

---------

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 authored Jul 18, 2023
    Configuration menu
    Copy the full SHA
    a2c1dbc View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2023

  1. Spark 3.4.1 and drop EOL Spark versions 3.0 and 3.1 (#430) 

    * Spark 3.4.0

* Drop EOL spark versions

* Upgrade 3.2 to 3.2.4

* Align scala version with what is used in spark

* Used 2.12.12 in scala ci for sbt-scoverage compatibility

* rebuild

* Use Spark 3.4.1

* Trigger CI

* Apply suggestions from code review

Co-authored-by: Enrico Minack <github@enrico.minack.dev>

---------

Co-authored-by: Enrico Minack <github@enrico.minack.dev>
    @EnricoMi
    Emil Ejbyfeldt and EnricoMi authored Sep 12, 2023
    Configuration menu
    Copy the full SHA
    7e2cd1c View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2023

  1. Setting version to 0.8.3

    @WeichenXu123
    WeichenXu123 committed Sep 25, 2023
    Configuration menu
    Copy the full SHA
    72187f4 View commit details
    Browse the repository at this point in the history
Loading