5.3.2

What's Changed

• fix: perform retries in a way that is compatible with driver retry logic by @ali-ince in #662
• feat: support checkpointing on structured streams by @ali-ince in #660

Full Changelog: 5.3.1...5.3.2

Neo4j Connector for Apache Spark 5.3.1

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!

What's Changed

• build: add package for spark-packages by @venikkin in #614
• fix: add a fix for a spark package base dir and the upload script by @venikkin in #616
• Exclude node properties from relationship's by @fbiville in #622
• Fix how resolvers are handled during driver construction by @ali-ince in #626
• fix: validate schema options before initializing schema by @fbiville in #630

New Contributors

• @venikkin made their first contribution in #610

Full Changelog: 5.3.0...5.3.1

Neo4j Connector for Apache Spark 5.3.0

What's Changed

⭐ Release Highlights

• Inferred schema type constraints + improved constraints management by @conker84 in #581

This introduces support for type/existence constraints based on the DataFrame definition and improves the handling of existing constraints.

Check out the documentation to learn more!

📜 Change Log

• [Snyk] Security upgrade org.testcontainers:testcontainers from 1.19.0 to 1.19.1 by @fbiville in #554
• Skip POM backups when setting new versions by @fbiville in #552
• Bump com.fasterxml.jackson.core:jackson-annotations from 2.15.2 to 2.15.3 by @dependabot in #557
• Bump com.fasterxml.jackson.core:jackson-annotations from 2.15.3 to 2.16.0 by @dependabot in #562
• Bump testcontainers.version from 1.19.1 to 1.19.2 by @dependabot in #561
• Bump org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.1 by @dependabot in #559
• Bump org.neo4j:neo4j-cypher-dsl from 2022.9.0 to 2022.9.1 by @dependabot in #558
• Bump testcontainers.version from 1.19.2 to 1.19.3 by @dependabot in #565
• Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.1 to 3.2.2 by @dependabot in #564
• Bump org.codehaus.mojo:versions-maven-plugin from 2.16.1 to 2.16.2 by @dependabot in #563
• Remove outdated version checks and rename tests by @fbiville in #567
• Bump actions/setup-java from 3 to 4 by @dependabot in #568
• Bump actions/setup-python from 4 to 5 by @dependabot in #570
• Fix README formatting by @fbiville in #569
• feat: add metrics support to the sink path by @ali-ince in #566
• Remove unneeded action by @fbiville in #571
• Remove unneeded workflow by @fbiville in #574
• Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.2 to 3.2.3 by @dependabot in #578
• Bump org.neo4j.driver:neo4j-java-driver from 4.4.12 to 4.4.13 by @dependabot in #585
• Add owners by @fbiville in #580
• Bump com.fasterxml.jackson.core:jackson-annotations from 2.16.0 to 2.16.1 by @dependabot in #586
• Bump org.apache.maven.plugins:maven-compiler-plugin from 3.11.0 to 3.12.1 by @dependabot in #587
• Improved user-agent management by @conker84 in #583
• Restrict reviews of CI-related changes by @fbiville in #588
• Improved query error management by @conker84 in #584
• Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.3 to 3.2.5 by @dependabot in #589
• Followup user agent by @conker84 in #590
• Remove unused bookmarks parameter by @fbiville in #591
• Bump actions/cache from 3 to 4 by @dependabot in #592
• Bump testcontainers.version from 1.19.3 to 1.19.4 by @dependabot in #593
• updated spark version and made it configurable by @conker84 in #595
• Bump testcontainers.version from 1.19.4 to 1.19.5 by @dependabot in #596

Full Changelog: 5.2.0...5.3.0

Neo4j Connector for Apache Spark 5.2.0

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!

Spark baseline version

This new release bumps the required Spark version to 3.4, in order to support the new push down Top N optimization.

What's Changed

• Implement Top N pushdown by @fbiville in #526
• Update Maven wrapper by @fbiville in #551
• Fix typo of close utility method name by @fbiville
• Upgrade scala versions by @fbiville in #542
• Bump actions/checkout from 3 to 4 by @dependabot
• Extract documentation away from repository by @ali-ince
  The documentation sources now live in https://github.com/neo4j/docs-spark/
• fixes #527: Add example notebooks by @conker84 in #528
  Interact with the notebooks here: https://neo4j.com/docs/spark/current/playground/

Full Changelog: 5.1.0...5.2.0

Neo4j Connector for Apache Spark 5.1.0

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!

Spark baseline version

This new release bumps the required Spark version to 3.3, in order to support the new push down v2 filters and limit optimizations.

What's Changed

• Support push down v2 filters by @fbiville in #492
• Bump neo4j-cypher-dsl from 2020.1.4 to 2022.8.5 by @dependabot in #510
• fixes #505: First-class support to GDS by @conker84 in #506
• Bump org.neo4j:neo4j-cypher-dsl from 2022.8.5 to 2022.9.0 by @dependabot in #533
• [Snyk] Upgrade org.neo4j.driver:neo4j-java-driver from 4.4.11 to 4.4.12 by @fbiville in #532
• Bump testcontainers.version from 1.18.3 to 1.19.0 by @dependabot in #538
• Add support for bearer authentication by @fbiville in #537
• Implement pushdown limit support by @fbiville in #520
• Make build script switch artifact IDs by @fbiville in #517
• fixes #531: Problem with datetime properties with null values by @conker84 in #536
• added fixed bug in version validation by @conker84 in #540
• Update Scala 2.13 version by @fbiville in #539

Full Changelog: 5.0.3...5.1.0

Neo4j Connector for Apache Spark 5.0.3

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!
There are two files:

• neo4j-connector-apache-spark_2.12-5.0.3_for_spark_3.jar, md5 46775288407098da460c5668db6fedea
• neo4j-connector-apache-spark_2.13-5.0.3_for_spark_3.jar, md5 72727cc4b399d0fe17926d8a43d91c86

What's Changed

• Bump testcontainers.version from 1.18.0 to 1.18.1 by @dependabot in #495
• Bump testcontainers.version from 1.18.0 to 1.18.2 by @dependabot in #508
• Bump testcontainers.version from 1.18.2 to 1.18.3 by @dependabot in #511
• Bump maven-source-plugin from 3.2.1 to 3.3.0 by @dependabot in #500
• Broken links fix by @lidiazuin in #513
• Bump jackson-annotations from 2.15.0 to 2.15.2 by @dependabot in #509
• Bump maven-surefire-plugin from 3.1.0 to 3.1.2 by @dependabot in #515
• Remove obsolete Spark version condition by @fbiville in #518
• fixes #503: Not able to Insert Neo4j Map Data type using the neo4j-spark connector by @conker84 in #507
• [Snyk] Upgrade @neo4j-antora/antora-add-notes from 0.1.6 to 0.3.0 by @fbiville in #522

Full Changelog: 5.0.2...5.0.3

Neo4j Connector for Apache Spark 5.0.2

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!
There are two files:

• neo4j-connector-apache-spark_2.12-5.0.2_for_spark_3.jar, md5 6033893da4ef48be27a9fa2055b681b0
• neo4j-connector-apache-spark_2.13-5.0.2_for_spark_3.jar, md5 c94d6247bb5a6862fefde7d98533c0f0

What's Changed

• Update Neo4j versions used by integration tests by @fbiville in #461
• Remove checks for Scala 2.11 from CI since 2.11 is not supported by @fbiville in #462
• Make the build script optional - use sensible Maven defaults by @fbiville in #463
• Enable Dependabot by @fbiville in #464
• Build with LTS versions of JDK by @fbiville in #465
• Bump testcontainers.version from 1.17.3 to 1.18.0 by @dependabot in #476
• Bump maven-assembly-plugin from 2.5.3 to 3.5.0 by @dependabot in #474
• Bump actions/setup-node from 1 to 3 by @dependabot in #473
• Bump actions/setup-python from 2 to 4 by @dependabot in #467
• Bump actions/setup-java from 1 to 3 by @dependabot in #468
• Bump actions/cache from 1 to 3 by @dependabot in #470
• Bump actions/checkout from 2 to 3 by @dependabot in #471
• Bump avides/actions-project-version-check from 1.2.0 to 1.4.0 by @dependabot in #481
• Bump build-helper-maven-plugin from 3.0.0 to 3.3.0 by @dependabot in #480
• Bump jackson-annotations from 2.12.0 to 2.15.0 by @dependabot in #479
• Bump junit from 4.13.1 to 4.13.2 by @dependabot in #477
• Simplify documentation structure & review quickstart guide by @fbiville in #460
• Bump scala-maven-plugin from 4.5.4 to 4.8.1 by @dependabot in #487
• Remove always-false comparison in Spark version checker by @fbiville in #466
• Bump maven-surefire-plugin from 3.0.0 to 3.1.0 by @dependabot in #488
• Bump maven-source-plugin from 2.2.1 to 3.2.1 by @dependabot in #483
• Bump maven-compiler-plugin from 3.8.1 to 3.11.0 by @dependabot in #485
• Bump hamcrest-library from 1.3 to 2.2 by @dependabot in #486
• Bump maven-assembly-plugin from 3.5.0 to 3.6.0 by @dependabot in #494
• fixes #498: spark version validation fails on EMR / EMR Serverless by @conker84 in #501
• bump version to 5.0.2 by @conker84 in #502

New Contributors

• @fbiville made their first contribution in #461 🎉🎉🎉

Full Changelog: 5.0.1...5.0.2

Neo4j Connector for Apache Spark 5.0.1

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!
There are two files:

• neo4j-connector-apache-spark_2.12-5.0.1_for_spark_3.jar, md5 08f1fa9e936eab63aa380ac50f30f2e0
• neo4j-connector-apache-spark_2.13-5.0.1_for_spark_3.jar, md5 692af135dfdf0d188e9c9b2d944996ef

Below are the changes.

What's Changed

• Minor Improvements by @conker84 in #453
• fixes #444: Fixing issue where LocalTime and Time are being written as individual fields instead of single Struct Field (#450) by @conker84 in #454

Full Changelog: 5.0.0...5.0.1

Neo4j Connector for Apache Spark 5.0.0

Hello Community!

We are pleased to announce the 5.0.0 of the Spark Connector which is the first official release that fully supports Neo4j 5.0 and Neo4j 4.4. For earlier versions of <= Neo4j 4.3, please use Neo4j Apache Spark 4.2.

N.B Because of a bug in Spark, the Aggregate Pushdown feature can only be activated with Spark >3.2.2 (please see apache/spark#36727)

There are three files:

• neo4j-connector-apache-spark_2.12-5.0.0_for_spark_3.jar, md5 ce814cd8d0a34f14599302895be13956
• neo4j-connector-apache-spark_2.13-5.0.0_for_spark_3.jar, md5 39e5ebb2b71a3f93d3230db91f25e2db

What's Changed

• Code clean by @conker84 in #445
• fixes #391: Support Spark 3.2 DatasourceV2 Aggregate pushdown by @conker84 in #431
• Support Neo4j 5.0 indexes and constraints syntax changes by @conker84 in #449

Full Changelog: 4.1.5...5.0.0

Neo4j Connector for Apache Spark 4.2.0

Hello Community!
We are pleased to announce the 4.2.0 of the Spark Connector with this version we deprecated the support for Spark 2.4 and support only Spark versions from 3.2.
With this release we added the support for Spark's Aggregate Pushdown which allows doing aggregations directly in Neo4j.

Nota bene Given a Spark bug the Aggregate Pushdown feature can be activated only from Spark 3.2.2 (please see apache/spark#36727)

There are three files:

• neo4j-connector-apache-spark_2.12-4.2.0_for_spark_3.jar, md5 6de1f93ece9f45aa51486317a62620f0
• neo4j-connector-apache-spark_2.13-4.2.0_for_spark_3.jar, md5 a79cf2b9d8d6725587760fa149932be2

What's Changed

• fixes #391: Support Spark 3.2 DatasourceV2 Aggregate pushdown by @conker84 in 1585e97
• Code clean by @conker84 in #445
• Improved Spark version validation by @conker84 in #448

Full Changelog: 4.1.5...4.2.0