Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Paginating Relationships #318

Open
kyleboe opened this issue May 23, 2024 · 1 comment
Open

Paginating Relationships #318

kyleboe opened this issue May 23, 2024 · 1 comment

Comments

@kyleboe
Copy link

kyleboe commented May 23, 2024

When including related resources in a request, all associated records are returned, which can lead to performance issues and excessive data transfer when dealing with larger datasets.

Example

defmodule UserController do
  plug JSONAPI.QueryParser,
    include: ~w(posts),
    view: UserView
  # . . .
end
defmodule UserView do
  def fields do
    [:name]
  end

  def links(data, conn) when not is_list(data) do
    # Some actual pagination logic
    %{
      first: "foo",
      last: "bar",
      next: "baz",
      prev: "bang"
    }
  end

  def relationships do
    [posts: PostView]
  end
end

When a request to /users/1?include=posts is made, ideally the ability to limit the number of posts returned in a given request and to paginate the nested relationship. Currently, this does not exist and because of the special treatment of the links callback, it is merged into the data key as well as the top level document.

{
  "data": {
    "type": "users",
    "id": "1",
    "attributes": {
      "name": "Foo Barton"
    },
    "links": {
        "first": "foo",
        "last": "bar",
        "next": "baz",
        "prev": "bang",
        "self": "http://www.example.com/users/1"
    },
    "relationships": {
      "posts": {
        "data": [
          {
            "id": "1",
            "type": "posts"
          },
          {
            "id": "2",
            "type": "posts"
          }
          // . . .
          // Tens of thousands of records
        ]
      }
    }
  },
  "links": {
    "first": "foo",
    "last": "bar",
    "next": "baz",
    "prev": "bang",
    "self": "http://www.example.com/users/1"
  },
  "included": []
}
    

Proposed Solution

I believe allowing for paginating nested relationships requires being less prescriptive about the usage of links. I acknowledge that pagination is outside the scope of this lib; with that in mind, I believe a change made to resolve this issue would further emphasize the line between pagination and JSONAPI serialization.

While it would be a significant change, I believe links should be removed as a responsibility of this lib. This can be handled by allowing other keys to be merged in when render is called:

defmodule UserController do
  plug JSONAPI.QueryParser,
    include: ~w(posts),
    view: UserView

  def show(conn, %{"id" => id}) do
    user = Users.get_by_id_and_load_posts(id)

    conn
    |> put_view(UserView)
    |> render("show.json",
      %{
        data: user,
        links: Paginator.paginate_posts(conn, user)
      }
    )
end

That way a request like /users/1?include=posts&page[posts][number]=1&page[posts][size]=10 could be made.

There may be a better way to handle this within the context of the current relationships definition as well.

  def relationships do
    [posts: {PostView, :paginate}]
  end

However, everything I've come up with to back the above style of configuration has only furthered the coupling between this lib and pagination, which would seem to move things in the wrong direction.

Thoughts?

Copy link
Contributor

This issue has been automatically marked as "stale:discard". We are sorry that we haven't been able to prioritize it yet.
If this issue still relevant, please leave any comment if you have any new additional information that helps to solve this issue. We encourage you to create a pull request, if you can. We are happy to help you with that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant