Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Update search query builder to support int and date range queries #195

Merged
merged 7 commits into from
Sep 9, 2024

Conversation

chouinar
Copy link
Collaborator

@chouinar chouinar commented Sep 9, 2024

Summary

Fixes #164

Time to review: 10 mins

Changes proposed

Adds support to our search query builder layer to handle building queries for filtering on ints and dates between specific date ranges.

Context for reviewers

Added ints and dates in the same ticket as the queries are essentially the same, just ints vs dates. While it throws an exception if both start and end values are None, I think the API request schema should also do that so that the error is more relevant/accurate to the API, but I can add that later (likely a lot more niche edge cases to handle for requests).

Nothing too exciting with these queries, they work as expected and are just simple ranges.

Additional information

The test dataset is roughly accurate (turns out books didn't always have exact release dates until the last ~20 years).

I also have tested these queries manually with the opportunity endpoints fields, the following two queries would work (can be tested locally at http://localhost:5601/app/dev_tools#/console ):

GET opportunity-index-alias/_search
{
  "size": 5,
  "query": {
    "bool": {
      "filter": [
          {
            "range": {
              "summary.post_date": {
                "gte": "2020-01-01",
                "lte": "2025-01-01"
              }
            }
          }
        ]
    }
  }
}

GET opportunity-index-alias/_search
{
  "size": 12,
  "query": {
    "bool": {
      "filter": [
          {
            "range": {
              "summary.award_floor": {
                "gte": 1234,
                "lte": 100000
              }
            }
          }
        ]
    }
  }
}

@chouinar chouinar merged commit 6b485e4 into main Sep 9, 2024
8 checks passed
@chouinar chouinar deleted the chouinar/search-builder-more branch September 9, 2024 19:50
acouch pushed a commit that referenced this pull request Sep 18, 2024
Fixes #164

Adds support to our search query builder layer to handle building
queries for filtering on ints and dates between specific date ranges.

Added ints and dates in the same ticket as the queries are essentially
the same, just ints vs dates. While it throws an exception if both start
and end values are None, I think the API request schema should also do
that so that the error is more relevant/accurate to the API, but I can
add that later (likely a lot more niche edge cases to handle for
requests).

Nothing too exciting with these queries, they work as expected and are
just simple ranges.

The test dataset is roughly accurate (turns out books didn't always have
exact release dates until the last ~20 years).

I also have tested these queries manually with the opportunity endpoints
fields, the following two queries would work (can be tested locally at
http://localhost:5601/app/dev_tools#/console ):
```
GET opportunity-index-alias/_search
{
  "size": 5,
  "query": {
    "bool": {
      "filter": [
          {
            "range": {
              "summary.post_date": {
                "gte": "2020-01-01",
                "lte": "2025-01-01"
              }
            }
          }
        ]
    }
  }
}

GET opportunity-index-alias/_search
{
  "size": 12,
  "query": {
    "bool": {
      "filter": [
          {
            "range": {
              "summary.award_floor": {
                "gte": 1234,
                "lte": 100000
              }
            }
          }
        ]
    }
  }
}
```
acouch pushed a commit that referenced this pull request Sep 18, 2024
Fixes #164

Adds support to our search query builder layer to handle building
queries for filtering on ints and dates between specific date ranges.

Added ints and dates in the same ticket as the queries are essentially
the same, just ints vs dates. While it throws an exception if both start
and end values are None, I think the API request schema should also do
that so that the error is more relevant/accurate to the API, but I can
add that later (likely a lot more niche edge cases to handle for
requests).

Nothing too exciting with these queries, they work as expected and are
just simple ranges.

The test dataset is roughly accurate (turns out books didn't always have
exact release dates until the last ~20 years).

I also have tested these queries manually with the opportunity endpoints
fields, the following two queries would work (can be tested locally at
http://localhost:5601/app/dev_tools#/console ):
```
GET opportunity-index-alias/_search
{
  "size": 5,
  "query": {
    "bool": {
      "filter": [
          {
            "range": {
              "summary.post_date": {
                "gte": "2020-01-01",
                "lte": "2025-01-01"
              }
            }
          }
        ]
    }
  }
}

GET opportunity-index-alias/_search
{
  "size": 12,
  "query": {
    "bool": {
      "filter": [
          {
            "range": {
              "summary.award_floor": {
                "gte": 1234,
                "lte": 100000
              }
            }
          }
        ]
    }
  }
}
```
acouch pushed a commit to HHS/simpler-grants-gov that referenced this pull request Sep 18, 2024
…vapbc#195)

Fixes #164

Adds support to our search query builder layer to handle building
queries for filtering on ints and dates between specific date ranges.

Added ints and dates in the same ticket as the queries are essentially
the same, just ints vs dates. While it throws an exception if both start
and end values are None, I think the API request schema should also do
that so that the error is more relevant/accurate to the API, but I can
add that later (likely a lot more niche edge cases to handle for
requests).

Nothing too exciting with these queries, they work as expected and are
just simple ranges.

The test dataset is roughly accurate (turns out books didn't always have
exact release dates until the last ~20 years).

I also have tested these queries manually with the opportunity endpoints
fields, the following two queries would work (can be tested locally at
http://localhost:5601/app/dev_tools#/console ):
```
GET opportunity-index-alias/_search
{
  "size": 5,
  "query": {
    "bool": {
      "filter": [
          {
            "range": {
              "summary.post_date": {
                "gte": "2020-01-01",
                "lte": "2025-01-01"
              }
            }
          }
        ]
    }
  }
}

GET opportunity-index-alias/_search
{
  "size": 12,
  "query": {
    "bool": {
      "filter": [
          {
            "range": {
              "summary.award_floor": {
                "gte": 1234,
                "lte": 100000
              }
            }
          }
        ]
    }
  }
}
```
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Task]: Modify the opensearch query builder to handle date ranges
2 participants