Skip to content

Latest commit

 

History

History
481 lines (252 loc) · 20.9 KB

CHANGELOG.md

File metadata and controls

481 lines (252 loc) · 20.9 KB
Raw

Changelog

7.21.0 (2024-09-13)

🎁 New Features

  • async run task and evaluator functions (23f5a21)
  • get experiment data back as a dataframe. (23f5a21)
  • run experiment with default traces (#34566) (23f5a21)

❔ Miscellaneous Chores

7.20.1 (2024-08-19)

🐛 Bug Fixes

📚 Documentation

7.20.0 (2024-08-16)

🎁 New Features

7.19.0 (2024-08-07)

🎁 New Features

  • Add client to provide public APIs to perform CRUD operations on datasets (#32096) (512070b)
  • Allow dataset client to create experiments on datasets (512070b)

7.18.1 (2024-05-22)

🐛 Bug Fixes

  • Correctly insert default prediction id column using df.insert() (f938d29)
  • Improve error message for type errors for raw data character count (#31272) (f938d29)
  • Include SHAP value invalid rows if full of nulls in error message (#31264) (24f079f)
  • Remove f-strings from docstrings and place them above arguments (#31117) (5c812d6)
  • Update similarity timestamp validation (#31125) (1c32c37)

7.18.0 (2024-05-15)

🎁 New Features

  • python exporter embedding similarity search support (#30921) (3a58f8f)

💫 Code Refactoring

  • add preprocessing step for similarity search param in flightserver (#30984) (34e2a96)

7.17.1 (2024-05-10)

🐛 Bug Fixes

  • Serialization of nested dictionaries (#30931) (ece89dc)
  • Avoid side effects in dictionary fields (ece89dc)
  • Value validation should require dictionaries, not JSON (ece89dc)

7.17.0 (2024-05-07)

🎁 New Features

  • Add session and user ids to spans batch logging (#30588) (90a0416)
  • Send arize schema as part of the request body (#30841) (b2f8e67)

🐛 Bug Fixes

  • improve evaluation column naming error message (267d23d)
  • relax opentelemetry-semantic-conventions dependency (#30840) (267d23d)
  • update URL to model page in logger (#30591) (5f0ee5a)

7.16.1 (2024-04-29)

🐛 Bug Fixes

  • Add missing __init__.py file to tracing validation module (#30539) (d18f108)

7.16.0 (2024-04-24)

🎁 New Features

  • Add log_evaluations method for delayed evaluation logging (#30179) (0f52763)
  • Fileimporter evaluations updates records (#30301) (7e1cbf6)

📚 Documentation

  • Add docstring to log_evaluations (0f52763)
  • Add docstring to log_spans (0f52763)

💫 Code Refactoring

7.15.0 (2024-04-17)

🎁 New Features

7.14.1 (2024-04-03)

🐛 Bug Fixes

7.14.0 (2024-03-29)

🎁 New Features

  • Support export of spans from Arize platform (#29350) (e248248)
  • Increase span field validation string length limits (#29501)(c32f464)

7.13.0 (2024-03-28)

🎁 New Features

🐛 Bug Fixes

  • use pandas items() vs deprecated iteritems() (#29384) (0eb377e)

7.12.1 (2024-03-26)

💫 Code Refactoring

❔ Miscellaneous Chores

7.12.0 (2024-03-23)

🎁 New Features

  • evals: Add evals log spans sdk
  • Add certificate file reading to sdk client

🐛 Bug Fixes

  • Avoid side-effects and correct null validation
  • tracing: Improve log spans from phoenix
  • tracing: correct missing value check for explanations
  • tracing: Import tracing modules dynamically

🧪 Tests

  • Avoid tracing tests for old python

🔀 CI

  • Add tracing dependencies to release workflow

7.11.1 (2024-03-05)

🐛 Bug Fixes

  • Fix ImportError when importing Client from arize.api

7.11.0 (2024-02-23)

❗ Dependency Changes

  • Add optional extra dependencies if the Arize package is installed as pip install arize[NLP_Metrics]:
    • nltk>=3.0.0, <4
    • sacrebleu>=2.3.1, <3
    • rouge-score>=0.1.2, <1
    • evaluate>=0.3, <1

🎁 New Features

  • Add optional strict typing in pandas logger Schema
  • Add 0ptional strict typing in record-at-a-time logger

7.10.2 (2024-02-14)

🐛 Bug Fixes

  • Address backward compatibility issue for batch logging via Pandas for on-prem customers
  • Validate that space and API keys are of string type

7.10.1 (2024-02-6)

❗Dependency Changes:

  • Add deprecated to our Tracing extra requirements. The deprecated dependency comes from opentelemetry-semantic-conventions, which absence produced an ImportError

7.10.0 (2024-02-1)

❗Dependency Updates:

  • Relax MimicExplainer extra requirements: require only interpret-community[mimic]>=0.22.0,<1

🎁 New Features:

  • Add batch ingestion via Pandas DataFrames for MULTICLASS model type
  • New TRACING environment. You can now log spans & traces for your LLM applications into Arize using batch ingestion via Pandas DataFrames
  • Removed size limitation on the Schema. You can now log wider models (more columns in your DataFrame)
  • Prediction ID and Ranking Group ID have an increased character limit from 128 to 512

7.9.0 (2023-12-28)

🎁 New Features:

  • New MULTICLASS model type available for record-at-a-time ingestion

7.8.1 (2023-12-18)

🐛 Bug Fixes:

  • Fix missing columns validation feedback to have repeated columns in the message
  • Fix KeyError when llm_params is not found in the dataframe. Improved feedback to the user was included.

7.8.0 (2023-12-13)

❗ Dependency Changes

  • Updated pandas requirement. We now accept pandas 2.x

🎁 New Features

  • Enable latent actuals for GENERATIVE_LLM models
  • Enable feedback when files are too large for better user experience and troubleshooting

7.7.2 (2023-11-09)

🐛 Bug Fixes:

  • Default prediction sent as string for GENERATIVE_LLM single-record-logger (before it was incorrectly set as an integer, resulting in it being categorized as prediction score instead of prediction label)

7.7.1 (2023-11-08)

🐛 Bug Fixes:

  • Only check the value of prompt/response if not None

7.7.0 (2023-11-02)

🎁 New Features

  • Add CORPUS support
  • Accept strings for prompt and response
  • Make prompt and response optional
  • Add support for a list of strings features in single-record-logger

🐛 Bug Fixes:

  • Avoid creating a view of a Pandas dataframe

7.6.1 (2023-10-24)

🐛 Bug Fixes:

  • Add validation on embedding raw data for batch and record-at-a-time loggers
  • Raise validation string limits for string fields
  • Add truncation warnings for long string fields

7.6.0 (2023-10-12)

🎁 New Features

  • Add ability to send features with type list[str]
  • Add new fields available to send token usage to Arize, both using our pandas batch logger and the single record logger

7.5.1 (2023-10-05)

❗Dependency Changes

  • Require python>=3.6 (as opposed to python>=3.8) for our core SDK. Our extras still require python>=3.8.
  • Require pyarrow>=0.15.0 (as opposed to pyarrow>=5.0.0)

🐛 Bug Fixes:

  • Increase time interval validation from 2 years to 5 years

7.5.0 (2023-09-02)

🎁 New Features

  • Add prompt templates and LLM config fields to the single log and pandas batch ingestion. These fields are used in the Arize Prompt Template Playground

🐛 Bug Fixes:

  • Add a validation check that fails if there are more than 30 embedding features sent

7.4.0 (2023-08-15)

🎁 New Features

  • Add filtering via the keyword where to the Exporter client

7.3.0 (2023-08-01)

🎁 New Features

  • AutoEmbeddings support for any model in the HuggingFace Hub, public or private.
  • Add AutoEmbeddings UseCase for Object Detection
  • Add EmbeddingGenerator.list_default_models() method

Bug Fixes

  • Computer Vision AutoEmbeddings switched from using FeatureExtractor(deprecated from HuggingFace) to ImageProcessor class

7.2.0 (2023-07-22)

🎁 New Features

  • Authenticating Arize Client using environment variables

🐛 Bug Fixes

  • Fix permission errors for pandas logging on Windows machines
  • Fix enforcement of tags into being strings

7.1.0 (2023-06-26)

🎁 New Features

  • Add Generative_LLM model-type support for single-record logging

7.0.6 (2023-06-24)

❗Dependency Changes

  • Removed dependency on interpret for the MimicExplainer

7.0.5 (2023-06-23)

❗ Dependency Changes

  • Add missing dependency for Exporter: tqdm>=4.60.0,<5

🐛 Bug Fixes

  • Update reserved headers
  • exporter: Fix progress bar in the Exporter client
  • exporter: Sort exported dataframe by time
  • exporter: Add validation check to Exporter client that will fail if start_time > end_time
  • exporter: Return empty response when an export query returns no data instead of an error.
  • exporter: Fix the Exporter client returning empty columns in the dataframe if there was no data in them
  • Fix incorrect parsing of GENERATIVE_LLM model prompt & response fields

7.0.4 (2023-06-13)

❗ Dependency Changes

  • Relax protobuf requirements from protobuf~=3.12 to protobuf>=3.12, <5

7.0.3 (2023-06-02)

🎁 New Features

  • Add new ExportClient, you can now export data from Arize using the Python SDK

🐛 Bug Fixes

  • Allow REGRESSION models to use the MimicExplainer
  • Remove null value validation for prediction_label and actual_label from single-record logging
  • Add model mapping rules validation for OBJECT_DETECTION models

7.0.2 (2023-05-12)

❗ Dependency Changes

  • Change optional dependency for MimicExplainer, raise the version ceiling of lightgbm from 3.3.4 to 4

🐛 Bug Fixes

  • Improve error messages around prediction ID, prediction labels, and tags
  • Fix predictions sent as scores instead of labels for NUMERIC model types
  • Add a validation check that will fail if the character limit on tags (1000 max) is exceeded
  • Add a validation check that will fail if actuals are sent without prediction ID information (for single-record logging). This would result in a delayed record being sent without a prediction ID, which is necessary for the latent join
  • Add a validation check that will fail if the Schema, without prediction columns, does not contain a prediction ID column (for pandas logging). This would result in a delayed record being sent without a prediction ID, which is necessary for the latent join
  • Add a validation check that will fail if the Schema points to an empty string as a column name
  • Add check for invalid index in AutoEmbeddings: DataFrames must have a sorted, continuous index starting at 0
  • Remove label requirements & accept null values on SCORE_CATEGORICAL, NUMERIC, and RANKING models
  • Allow feature and tag columns to contain null values for pandas logging
  • Allow to send delayed actuals for RANKING models, it is no longer enforced the presence of rank and prediction_group_id columns in the Schema. However, if the columns are sent, they must not have nulls, since we cannot construct predictions with either value null

7.0.1 (2023-04-25)

🐛 Bug Fixes

  • Fix GENERATIVE_LLM models being sent as SCORE_CATEGORICAL models

7.0.0 (2023-04-13)

⚠ BREAKING CHANGES

  • Require Python >= 3.8 for all extra functionality
  • Remove numeric_sequence support

❗ Dependency Changes

  • Add optional extra dependencies if the Arize package is installed as pip install arize[LLM_Evaluation]:

    • nltk>=3.0.0, <4
    • sacrebleu>=2.3.1, <3
    • rouge-score>=0.1.2, <1
    • evaluate>=0.3, <1

🎁 New Features

  • Add Object Detection model-type support
  • Add Generative LLM model-type support for pandas logging
  • Add evaluation metrics generation for Generative LLM models
  • Make prediction IDs optional
  • Add summarization UseCase to AutoEmbeddings
  • Add optional, additional custom headers to Client instantiation
  • Add a warning message when only actuals are sent
  • Add a descriptive error message when embedding features are sent without a vector
  • Add warning when prediction label or prediction ID will be defaulted

🐛 Bug Fixes

  • A bug causing skipped validation checks when the new REGRESSION and CATEGORICAL model types are selected
  • Add a validation check that will fail if the character limit on prediction ID (128 max) is exceeded
  • Add a validation check that will fail if there are duplicated columns in the dataframe
  • Changed time range requirements to -2/+1 (two years in the past, and 1 future year)