Stats and languages results incorrect due to graphql 100 records limit

[rickstaa]

Describe the bug

The current language and stats card results are incorrect for users with more than 100 repositories (see https://docs.github.com/en/graphql/overview/resource-limitations). This is because we are currently only fetching the first 100 repositories of a user account.

github-readme-stats/src/fetchers/stats-fetcher.js

Line 47 in a68abbb

repositories(first: 100, ownerAffiliations: OWNER, orderBy: {direction: DESC, field: STARGAZERS}) {

github-readme-stats/src/fetchers/top-languages-fetcher.js

Line 17 in a68abbb

repositories(ownerAffiliations: OWNER, isFork: false, first: 100) {

Because of this not all languages and stars of a user are represented.

Expected behaviour

I think we can use pagination options (i.e. cursor, after, endCursor objects) to loop through all repositories of a user account (see https://stackoverflow.com/a/70039506/8135687). @anuraghazra Let me know what you think.

TODOs

• Allow fetching of multiple pages for the languages data. fix: Fetch all repos for top languages #2111.
• Improve the stats card such that it fetches more pages when a user has more than 100 repositories with stars. fix: fetch all repos for for the stats card #2100
  • Merge feat: enable multi page star fetching for private vercel instances #2159 to make this available on private Vercel instances while giving us the control to disable it on the public instance because of rate-limiting problems.

[rickstaa]

For the language card, a PR is already open to solve this #1178. For the stats card, there not yet exists a pull request. This is, however, less important since only people with more than 100 repositories with stars are affected.

[lumynou5]

For the language card, a PR is already open to solve this #1178. For the stats card, there not yet exists a pull request. This is, however, less important since only people with more than 100 repositories with stars are affected.

Why it is less important? Although people with more than 100 repositories with stars are less.

[rickstaa]

@Luminous-Coder Thanks for your question. What I meant is that it is probably affecting fewer people. Since the GraphQL call we make returns the repositories listed from most starred to least starred, a user must have 100 repositories with many stars for to score to be significantly affected. I hope that clears things up. Nevertheless, we should solve both cases. 👍🏻

[MatteoPierro]

Hi @rickstaa, is it ok if I work on this for the stats card?

[rickstaa]

@MatteoPierro of course! That would be amazing. An example of how this can be done using pagination can be found here. I think for the stats card, to be resource efficient, I think we can stop the pagination when the GraphQL API does not return repositories that have any stars.

[MatteoPierro]

@rickstaa here it's my PR #2100

[MatteoPierro]

@rickstaa Can I work on fixing the fetch all repos for top languages part as well? I see there is already a PR but it looks stale.

[rickstaa]

@MatteoPierro, yea no problem. That would be great! 🚀 The pull request was created a long time ago, and I just reviewed it last week. The best thing to do is to work on this pull request so that @tc-imba gets co-authored. I think that #1178 should also be good if the syntax and tests are improved (see #1178 (comment)).

[MatteoPierro]

ok. how can I work on @tc-imba PR in co-authored mode?

[rickstaa]

The easiest way is to let git handle the co-authoring. This can be done as follows:

1. Add @tc-imba's branch as a remote git remote add tcimba [email protected]:tc-imba/github-readme-stats.git.
2. Fetch all new branches from the remotes git fetch -all.
3. Checkout @tc-imba's master branch git checkout tcimba/master.
4. Create a new branch based on @tc-imba's master branch git checkout -b name_this_however_you_want.
5. Add the upstream master git remote add upstream [email protected]:anuraghazra/github-readme-stats.git.
6. Fetch the upstream master git fetch upstream.
7. Merge the upstream master into your new branch git merge upstream/master.
8. Fix merge conflicts, and you are good to go! 🚀

Let me know if you run into any problems.

[MatteoPierro]

Alright, probably I made it. #2111

[rickstaa]

Just a heads-up, I filled a feature request to make the total stars a specific user has received on his/her repositories available under the user object of the GraphQL API. Adding this feature would also enable #2111 on the public instance.community/community#35677. Please add your support there.

[milkshakegum]

Hi! I'm experiencing the same issue too!

This is the link of my stats card:

https://github-readme-stats.vercel.app/api?username=milkshakegum&show_icons=true&title_color=fff&icon_color=f9f9f9&text_color=9f9f9f&bg_color=141617&count_private=true&include_all_commits=true&hide_border=true&hide_title=true

But it shows:

It's inconsistent

[rickstaa]

@milkshakegum Thanks for your report. Merging:

• fix: Fetch all repos for top languages #2111
• feat: enable multi page star fetching for private vercel instances #2159
• fix: prevent totalCommitsFetch error result from being cached #2177

Will fix the issues you experience. You can already use these on your own fork. They are not yet merged into the master because we have to ensure the GraphQL and Vercel limits are not hit. We will likely implement this as an opt-in for Vercel instance and Github Action users (see #2179 and #2159).