You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When ingesting an Elasticsearch source into DataHub, if the index mapping includes a nested property with multiple sub-properties, DataHub fails to detect the sub-properties of the nested field after ingestion. For example, with the following mapping:
Go to 'Elasticsearch Source Configuration' in DataHub.
Click on 'Ingest Elasticsearch Index'.
Scroll down to the section with the nested properties in the index mapping.
See error: Sub-properties of the nested field are not detected.
Expected behavior
The sub-properties of the nested field should be detected and ingested properly by DataHub, allowing them to be visible and usable within the DataHub interface.
Desktop (please complete the following information):
…mapping
Addresses the issue with Elasticsearch ingestion where sub-properties within the 'nested' type are not being detected during the mapping process, leading to incomplete ingestion results.
Fixes: datahub-project#10911
Bumyu
pushed a commit
to Bumyu/datahub
that referenced
this issue
Sep 9, 2024
…mapping
Fixes an issue where sub-properties in 'nested' type mappings are not detected
during Elasticsearch ingestion, leading to incomplete ingestion of data.
Fixes: datahub-project#10911
Bumyu
pushed a commit
to Bumyu/datahub
that referenced
this issue
Sep 10, 2024
…mapping
fix issue where sub-properties in 'nested' type mappings are not detected during
Elasticsearch ingestion, resulting in incomplete ingestion of data. Prior to this
fix, only top-level properties were recognized, missing critical sub-fields.
Fixes: datahub-project#10911
…mapping
fix issue where sub-properties in 'nested' type mappings are not detected during
Elasticsearch ingestion, resulting in incomplete ingestion of data. Prior to this
fix, only top-level properties were recognized, missing critical sub-fields.
Fixes: datahub-project#10911
Describe the bug
When ingesting an Elasticsearch source into DataHub, if the index mapping includes a nested property with multiple sub-properties, DataHub fails to detect the sub-properties of the nested field after ingestion. For example, with the following mapping:
In this case, DataHub does not detect the address and type properties within the addresses nested field.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The sub-properties of the nested field should be detected and ingested properly by DataHub, allowing them to be visible and usable within the DataHub interface.
Desktop (please complete the following information):
Additional context
The source of this issue resides here: https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/elastic_search.py
I tried to find a possible solution that can be used and that worked for my situation:
The text was updated successfully, but these errors were encountered: