Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] [Search Comparison Tool] Display Images #286

Open
sejli opened this issue Sep 12, 2023 · 2 comments
Open

[FEATURE] [Search Comparison Tool] Display Images #286

sejli opened this issue Sep 12, 2023 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@sejli
Copy link
Member

sejli commented Sep 12, 2023

Building upon #195

Is your feature request related to a problem?

As described in the previous issue, the current search comparison tool only displays textual document source. In the case there is an image URL in the document, displaying the image within the rows of the search result would be useful for comparing indices containing images.

What solution would you like?

Simply put, we want images to be displayed in the search comparison tool if a valid URL exists in the document source. If not, a placeholder image will be displayed within the row. We want the user to be able to customize this experience, with options not limited to:

  1. Which field in the document source contains the image
  2. Whether a base URL should be appended to the value stored in the document field

What alternatives have you considered?

There are no other ways to search and display images in OpenSearch.

Do you have any additional context?

The ML playground currently has a working implementation of displaying image, without much customization. As shown in this example images are displayed within the search result rows, and may be clicked on to be enlarged.
image

@sejli sejli added enhancement New feature or request untriaged labels Sep 12, 2023
@sejli sejli self-assigned this Sep 12, 2023
@msfroh msfroh removed the untriaged label Sep 20, 2023
@macohen macohen removed the v2.12.0 label Oct 5, 2023
@dtaivpp
Copy link

dtaivpp commented Dec 22, 2023

@macohen @sejli would it make sense to create a new field type within OpenSearch that is an "image" type that is always a URI? By formalizing it a bit to make it a proper type we could do some interesting things like extracting the extension to it has a meta image type. This could also enable users to have a better experience with ingestion pipelines as they wouldn't need to pre-process their images to base64 to get them embedded.

@macohen
Copy link
Collaborator

macohen commented Dec 22, 2023

It could make sense. I think we would want to separate that as a dependency here and have an issue in core, though. Would this also have an impact on kNN ingestion/multi-modal search? What do you think, @msfroh?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Later (6 months plus)
Development

No branches or pull requests

4 participants