Releases: snowflakedb/snowflake-ingest-java
v2.2.2
What's Changed
- SNOW-1619347: upgrade bc-fips by @sfc-gh-tzhang in #810
- SNOW-1483230 Parameter support & disable blob encryption for new table format by @sfc-gh-alhuang in #801
- NO_SNOW Upgrade dependency versions by @sfc-gh-xhuang in #800
- SNOW-1497358 Support multiple stage for new table format by @sfc-gh-alhuang in #812
- SNOW-1654124: Write file name to metadata at the place when we create the file by @sfc-gh-tzhang in #824
- V2.2.2 Release by @sfc-gh-tzhang in #826
Full Changelog: v2.2.0...v2.2.2
v2.2.1
[Improvement] This release fixes a performance regression in upload
What's Changed
- SNOW-1619347: upgrade bc-fips by @sfc-gh-tzhang in #810
- SNOW-1483230 Parameter support & disable blob encryption for new table format by @sfc-gh-alhuang in #801
- NO_SNOW Upgrade dependency versions by @sfc-gh-xhuang in #800
- SNOW-1497358 Support multiple stage for new table format by @sfc-gh-alhuang in #812
Full Changelog: v2.2.0...v2.2.1
v2.2.0
[Bug Fix] Fix a critical bug that could potentially cause corruption with change_tracking enabled.
It is strongly recommend to upgrade to 2.2.0+ as the minimum version for all Snowpipe Streaming usage
[Improvement] Refactor some of the code logic to support different storage volume
What's Changed
- Code refactor for different storage volume support by @sfc-gh-alhuang in #792
- SNOW-1512047 Introduce independent per-table flushes when interleaving is disabled by @sfc-gh-alhuang in #788
- SNOW-1618257 Fix PRIMARY_FILE_ID_KEY by @sfc-gh-kkloudas in #807
- V2.2.0 Release by @sfc-gh-tzhang in #808
Full Changelog: v2.1.2...v2.2.0
v2.1.2
This release contains a few improvements and bug fixes for Snowpipe Streaming:
[Bug Fix] Flaky tests are improved
[Bug Fix] Fixed an issue with failover across deployments
[Improvement] InsertRows Performance improvements
[Improvement] Added or Improved various logs for better observability
[Improvement] Fine tune channel and chunk sizes
What's Changed
- Add error log for flush task scheduling by @sfc-gh-alhuang in #760
- SNOW-1435061 Upgrade JDBC driver for domains fix by @sfc-gh-lsembera in #766
- no-snow Parallelize e2e jar tests by @sfc-gh-lsembera in #767
- SNOW-1357377 Add request Id in all streaming ingest APIs by @sfc-gh-japatel in #759
- no-snow Fix flaky test by @sfc-gh-lsembera in #772
- SNOW-1457523: Fix CVE for snowflake-ingest-java io.airlift:aircompressor 0.21 by @sfc-gh-tzhang in #774
- Various performance improvements in the path by @sfc-gh-psaha in #782
- SNOW-1457523: Upgrade Parquet dependencies version to fix CVE by @sfc-gh-tzhang in #779
- SNOW-1465503 Check row count in Parquet footer before committing by @sfc-gh-lsembera in #784
- SNOW-1512935 Retry SocketTimeoutException by @sfc-gh-lsembera in #789
- NO-SNOW Fix flaky test using multiple threads by @sfc-gh-lsembera in #790
- SNOW-1373151: Proactively refresh token to avoid token expiration exception from JDBC by @sfc-gh-tzhang in #780
- Reject new stage metadata if the deployment id does not match what the client was created with by @sfc-gh-psaha in #794
- SNOW-1512935 Reduce client socket timeout from 5 minutes to 1 minute by @sfc-gh-lsembera in #795
- SNOW-1545879 Reduce the max channel/chunk sizes by @sfc-gh-lsembera in #796
- NO-SNOW Upgrade protobuf package by @sfc-gh-xhuang in #793
- V2.1.2 Release by @sfc-gh-asen in #797
v2.1.1
What's Changed
- NO-SNOW updated version by @sfc-gh-dshah in #686
- SNOW-1165817 SNOW-1165815 Update vulnerable dependencies by @sfc-gh-lsembera in #693
- SNOW-1230527 Disable linkage checker by @sfc-gh-lsembera in #717
- SNOW-1196967 SNOW-1196966 Update vulnerable dependencies by @sfc-gh-lsembera in #720
- SNOW-1255447: Fix leaked connection due to error response by @sfc-gh-tzhang in #719
- SNOW-1000828 Expose specific error messages as part of the channel invalidation exception by @sfc-gh-alhuang in #721
- SNOW-993600 External OAuth2.0 Support by @sfc-gh-alhuang in #718
- NO-SNOW Reduce OAuth test time by @sfc-gh-alhuang in #732
- SNOW-1258064 Remove/Relax the limitations in order to generate bigger files by @sfc-gh-alhuang in #730
- [Snyk] SNOW-1327598: Fix for 3 vulnerabilities by @sfc-gh-snowflakedb-snyk-sa in #734
- SNOW-1167007: Validate no null values for not null cols by @sfc-gh-wtrefon in #739
- [Snyk] Security upgrade org.bouncycastle:bcpkix-jdk18on from 1.78 to 1.78.1 by @sfc-gh-snowflakedb-snyk-sa in #752
- SNOW-1361785 Run e2e jar tests with multiple Java versions by @sfc-gh-lsembera in #756
- No-SNOW Update fasterxml by @sfc-gh-xhuang in #740
- SNOW-1262359 Upgrade commons-configuration2 by @sfc-gh-lsembera in #761
- V2.1.1 Release by @sfc-gh-tzhang in #758
v2.1.0
What's Changed
- NO-SNOW Correct invalid use of javadoc by @sfc-gh-xhuang in #668
- SNOW-983635 Allow ZSTD compression algorithm by @sfc-gh-lthiede in #654
- SNOW-1032874: Fix an Overflow Issue for Integer-store Timestamp by @sfc-gh-tzhang in #673
- Snowpipe Streaming: send column ordinal back to GS with blob EPs to register by @sfc-gh-azagrebin in #674
- SNOW-979849: Add start offset token at batch level in the Client SDK by @sfc-gh-tzhang in #676
- SNOW-967235 Sends start and end offsets from the Client SDK by @sfc-gh-tjones in #675
- NO-SNOW: Two minor changes that address PR comments and fix code format by @sfc-gh-tzhang in #679
- SNOW-987122 Upgrade JDBC to 3.14.5 and Catch new exception type for r… by @sfc-gh-japatel in #677
- Allow clients to drop channel on close or blindly by @sfc-gh-psaha in #657
- Add more detailed javadoc for the drop related api to warn about the … by @sfc-gh-psaha in #681
- SNOW-979849: Support user to pass a user defined offset token verification logic as part of channel creation by @sfc-gh-tzhang in #680
- V2.1.0 Release by @sfc-gh-tzhang in #682
New Contributors
- @sfc-gh-psaha made their first contribution in #657
Full Changelog: v2.0.5...v2.1.0
v2.0.5
This release contains a few improvements and bug fixes for Snowpipe Streaming:
[Improvement] Added an optional offset token parameter for openChannel
.
[Improvement] Added support for specifying compression algorithm to be used for BDEC Parquet files.
[Improvement] Updated to support customized URL and added Snowflake account name in request header.
[Improvement] Implemented a change to send spansMixedTables
flag in blob registration requests.
[Improvement] Deprecated BUFFER_FLUSH_INTERVAL_IN_MILLIS
parameter, instead use the MAX_CLIENT_LAG
parameter.
[Improvement] Implemented the refresh of downscoped GCS tokens.
[Bug Fix] Reverted one change that updated public API for internal use case.
[Bug Fix] Fixed the end-to-end JAR test so it can run on all cloud platforms.
What's Changed
- Add an optional offset token parameter for openChannel by @sfc-gh-asen in #645
- support specifying compression algorithm to be used for BDEC Parquet files by @sfc-gh-gdoci in #579
- Disable Snowpipe flaky tests by @sfc-gh-tzhang in #647
- changed snapshot version from 2.0.4 to 2.0.5-SNAPSHOT by @sfc-gh-dshah in #650
- benchmark setup by @sfc-gh-lthiede in #655
- Update to support customized url and add customer name in request header by @sfc-gh-dwang in #656
- send spansMixedTables flag in blob registration requests by @sfc-gh-gdoci in #651
- Fix name of database the dataset is created in by @sfc-gh-lthiede in #658
- NO-SNOW: Deprecate BUFFER_FLUSH_INTERVAL_IN_MILLIS parameter by @sfc-gh-tzhang in #659
- NO-SNOW Run e2e jar tests on all clouds by @sfc-gh-lsembera in #666
- NO-SNOW: Revert one change that updates public API for internal use case by @sfc-gh-tzhang in #662
- GCS token refresh by @sfc-gh-lsembera in #665
- V2.0.5 release by @sfc-gh-tzhang in #664
Full Changelog: v2.0.4...v2.0.5
v2.0.4
This release contains a few improvements and bug fixes for Snowpipe Streaming:
- [Improvement] Support a new ON_ERROR option SKIP_BATCH, which will skip the entire batch if there is any issue and return all errors as part of the response
- [Improvement] Add row index information to all exceptions
- [Improvement] Upgrade snappy-java dependency
- [Improvement] Add a new interface to return the table schema information for a channel
- [Improvement] Add a new configuration option MAX_CLIENT_LAG which can contain the flush frequency, by default we flush every second
- [Bug Fix] Fix an issue when using snowflake-jdbc-fips
- [Bug Fix] Fix a rare ConcurrentModificationException issue
- [Bug Fix] Fix two issues in insertRows API that might cause wrong results in a very rare case
- [Bug Fix] Limit the max allowed number of chunks in blob to avoid the case when the request is too large
What's Changed
- Remove Public Preview note by @sfc-gh-lsembera in #582
- @no-snow refactor a few variables into ClientBufferParameters by @sfc-gh-gdoci in #581
- SNOW-914666 Adds MAX_CLIENT_LAG configuration option by @sfc-gh-tjones in #586
- SNOW-919423 Surfaces Table Schema for a Channel by @sfc-gh-tjones in #589
- NO-SNOW Remove wrong documentation by @sfc-gh-lthiede in #591
- updated SDK version to 2.0.4-SNAPSHOT by @sfc-gh-dshah in #588
- SNOW-924864 Upgrade snappy-java by @sfc-gh-lsembera in #592
- SNOW-918949: Add row index information to all exceptions by @sfc-gh-tzhang in #590
- SNOW-928802 Use concurrent hashmap to prevent ConcurrentModificationE… by @sfc-gh-japatel in #594
- SNOW-926149 Fix issues while using snowflake-jdbc-fips by @sfc-gh-lsembera in #596
- Update readme for JDBC version issues #599 by @sfc-gh-xhuang in #600
- SNOW-906224: Add new ON_ERROR=SKIP_BATCH option by @sfc-gh-tzhang in #597
- NO-SNOW: Fix two issues in insertRows API by @sfc-gh-tzhang in #602
- SNOW-902709 Limit the max allowed number of chunks in blob by @sfc-gh-lsembera in #580
- PRODSEC-3611 fix GHA parsing by @sfc-gh-jfan in #603
- no-snow Sort pom.xml in format.sh by @sfc-gh-lsembera in #604
- SNOW-936900 Use upgraded version of snappy-java by @sfc-gh-lsembera in #605
- SNOW-945927 SNOW-945928 Fix Snyk vulnerabilities by @sfc-gh-lsembera in #641
- NO-SNOW: fix row index issue for ON_ERROR=SKIP_BATCH by @sfc-gh-tzhang in #606
- V2.0.4 release by @sfc-gh-tzhang in #643
New Contributors
- @sfc-gh-lthiede made their first contribution in #591
Full Changelog: v2.0.3...v2.0.4
v2.0.3
This release contains a few improvements and bug fixes for Snowpipe Streaming:
- [Improvement] Support OAuth Authentication
- [Improvement] Remove exactly once related code for Snowpipe
- [Improvement] Support publishing unshaded snapshot release to Nexus repo
- [Improvement] Add retry logic for invalid JWT token
- [Improvement] Add warning for large batches in insertRows
- [Bug Fix] Fix a NPE issue caused by race condition
What's Changed
- SNOW-352846 OAuth Authentication: #1 Polymorph auth credential by @sfc-gh-alhuang in #532
- SNOW-693813 disable client info snowpipe tests, remove later by @sfc-gh-japatel in #536
- [SNOW-693813] Remove exactly once methods calling into server by @sfc-gh-rcheng in #539
- SNOW-855191 Race condition causing NPE by @sfc-gh-lsembera in #544
- SNOW-352846 OAuth Authentication: #2 OAuth Support by @sfc-gh-alhuang in #537
- SNOW-870386 Add default scheme parameters for test utils when profile is not specified by @sfc-gh-alhuang in #552
- SNOW-352846 OAuth Authentication: #3 Add error message by @sfc-gh-alhuang in #553
- NO-SNOW: Support publishing unshaded snapshot release to Nexus repo by @sfc-gh-tzhang in #561
- Update AWS test profile by @sfc-gh-alhuang in #564
- changed version from 2.0.2 to 2.0.3-SNAPSHOT by @sfc-gh-dshah in #565
- SNOW-877039 Ingest SDK retry for INVALID JWT Response by @sfc-gh-alhuang in #562
- Move testJWTRetries test to IT by @sfc-gh-alhuang in #572
- SNOW-861190 Add warning for large batches in insertRows by @sfc-gh-lsembera in #551
- Using jenkins sonatype username and password instead of my own by @sfc-gh-dshah in #571
- NO-SNOW: Add a few requested changes by @sfc-gh-tzhang in #576
- V2.0.3 release by @sfc-gh-tzhang in #577
New Contributors
- @sfc-gh-alhuang made their first contribution in #532
Full Changelog: v2.0.2...v2.0.3
v2.0.2
This release contains a few improvements for Snowpipe Streaming:
- [Improvement] Update dependencies based on Wiz and Snyk vulnerability scan result
- [Improvement] Improve retry logic on exceptions like SSLException
- [Improvement] Make the role as an optional input and support using of default role associated with the user
- [Improvement] Send uncompressed chunk length to server side for tracking purpose
What's Changed
- Remove CLA bot and update README.md to include cla and formatter requirements by @sfc-gh-rcheng in #510
- SNOW-841883 Upgrade snappy-java to 1.1.10.1 by @sfc-gh-lsembera in #533
- [Snyk] Security upgrade com.google.guava:guava from 31.1-jre to 32.0.0-jre by @sfc-gh-snyk-sca-sa in #534
- NO-SNOW Bump version to 2.0.2-SNAPSHOT by @sfc-gh-lsembera in #529
- Use later version of google guava, as recommended by CVE by @sfc-gh-japatel in #535
- [SNOW-843760] Upgrade netty-common in response to CVE vulnerability by @sfc-gh-rcheng in #538
- NO-SNOW Add note about dependencies by @sfc-gh-lsembera in #540
- SNOW-851351 Retry javax.net.ssl.SSLException by @sfc-gh-lsembera in #543
- [SNOW-843760] Update pom.xml corresponding to Wiz vulnerability scan by @sfc-gh-rcheng in #546
- SNOW-532834 Using default role when snowflake.role.name is not specified by @sfc-gh-alhuang in #507
- NO-SNOW Retry failed requests more aggressively by @sfc-gh-lsembera in #550
- v2.0.2 version upgrade by @sfc-gh-tzhang in #557
- @snow SNOW-835618 Snowpipe Streaming: send uncompressed chunk length from SDK to GS by @sfc-gh-azagrebin in #521
- reduce number of rows in BigFilesIT by @sfc-gh-gdoci in #559
Full Changelog: v2.0.1...v2.0.2