Prepare Unified codebase for Spark ( Iceberg ) Support

[ilias1111]

Description

1. Universal Changes
   • Replaced the QUALIFY clause with PostgreSQL-compatible logic to ensure consistent behavior across different target types.
   • Modified the casting syntax to use CAST(x AS data_type) for Spark compatibility.
   • Adapted integration tests by creating Spark-specific source files to accommodate Spark's syntax requirements and limitations.
   • Configured the incremental strategy based on the target type, using 'delete+insert' for Postgres and Redshift, and 'merge' for Spark.
   • Adjusted the handling of ROW_NUMBER() in tests to account for Spark's non-deterministic behavior.
2. Snowplow Utils Integration
   • Snowplow Unified now leverages the updated utility functions, macros, and configurations provided by Snowplow Utils to ensure compatibility and optimal performance when running on Spark with Iceberg.

What type of PR is this? (check all applicable)

• 🍕 Feature
• 🐛 Bug Fix
• 📝 Documentation Update
• 🎨 Style
• 🧑‍💻 Code Refactor
• 🔥 Performance Improvements
• ✅ Test
• 🤖 Build
• 🔁 CI
• 📦 Chore (Release)
• ⏩ Revert

Related Tickets & Documents

Checklist

• 💣 Is your change a breaking change?
• 📖 I have updated the CHANGELOG.md

Added tests?

• 👍 yes
• 🙅 no, because they aren't needed
• 🙋 no, because I need help

Added to documentation?

• 📓 internal package docs (ymls, macros, readme, if applicable)
• 📕 I have raised a Snowplow documentation PR if applicable (Link here)
• 🙅 no documentation needed

[optional] What gif best describes this PR or how it makes you feel?

[agnessnowplow]

Just like for utils, leaving some comments for clarification, nothing else sticks out for now.

[agnessnowplow]

Tentative approval, same as with utils, let's agree on the incremental_stategy (be explicit or permissive) for the release and also investigate why bigquery / redshift started failing. Most likely not related to the spark prep (utils was not linked to the branch for redshift at least) hence the ok for now.