Skip to content
This repository has been archived by the owner on Aug 16, 2024. It is now read-only.

Question: How to solve N+1 problem with Rejoiner #68

Open
xiaomeiwen opened this issue Jan 25, 2019 · 2 comments
Open

Question: How to solve N+1 problem with Rejoiner #68

xiaomeiwen opened this issue Jan 25, 2019 · 2 comments

Comments

@xiaomeiwen
Copy link

xiaomeiwen commented Jan 25, 2019

I recently face a performance issue in a multi-level query using Rejoiner.
Here's an example similar to my data structure:

companyEmployees(startDate, endDate) {
  company {
    id
    department {
      id
      team {
        id
        employee {
          id
        }
      }
    }
  }
}

Assume each level is 1-many.
I have defined a set of gRPC APIs to get details at each level, and all the IDs are globally unique (e.g. team 123 only belongs to a specific department, which belongs to a specific company):

getCompanyList()
getDepartmentsByCompanyId()
getTeamsByDepartmentId()
getEmployeesByTeamId()

I then created a query called companyEmployees(companyId) which will use Rejoiner's SchemaModification to handle nested query based on the data structure. It goes like this:

    @Query("companyDepartments")
    ImmutableList<Department> getDepartmentsByCompany(DepartmentByCompanyRequest request,
                                                  DepartmentConfigServiceBlockingStub client) {
        List<Department> departments = client.getDepartmentsByCompany(request).getDepartmentsList();
        return ImmutableList.copyOf(departments);
    }

    @SchemaModification(addField = "teams", onType = Department.class)
    ImmutableList<Team> departmentTeams(DepartmentConfigServiceBlockingStub client, Department department) {
        // query team details by department ID
        request = TeamByDepartmentRequest.newBuilder().setDepartmentId(department.getId()).build();
        teams = client.getTeamsByDepartment(request).getTeamsList();
        return ImmutableList.copyOf(teams);
    }
    @SchemaModification(addField = "employees", onType = Team.class)
    ImmutableList<Employee> teamEmployees(DepartmentConfigServiceBlockingStub client, Team team) {
        // similar logic
    }

I added logs to trace the companyDepartments query and how it triggers SchemaModification. In my DB, I have 1 company, 2 departments, each department contains 2 teams, and each team contains 5 members. And I found out the following behavior:
It get the company first -> call SchemaModification to get first department -> call SchemaModification to get first team in first department ->...
It looks like a DFS, and the query time becomes linear, based on how much data is there in DB.

Is there any approach I can change my query time in to a constant time, let those queries happen parallel?

Facebook use Dataloader to solve this n+1 prob, I wonder how to solve it with rejoiner.
https://engineering.shopify.com/blogs/engineering/solving-the-n-1-problem-for-graphql-through-batching

@xiaomeiwen xiaomeiwen changed the title Question: SchemaModification Question: SchemaModification time issue Jan 25, 2019
@xiaomeiwen xiaomeiwen changed the title Question: SchemaModification time issue Question: How to solve N+1 problem with Rejoiner Jan 25, 2019
@LIU0472
Copy link

LIU0472 commented Apr 4, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants