Code search results repeated for monorepos

[ef4]

This search matches two addons that appear in the cardstack monorepo. But the results are repeated once for every other package in the monorepo, which spams the search results.

[kategengler]

Unfortunately, we have no good way to handle monorepos. The code search relies upon an index built from on-disk files, so we literally clone the repo pointed to by every addon http://codeallday.com/blog/how-ember-observer-searches-addon-source-code/

Monorepos also have incorrect stats for stats from GitHub such as issues and latest commit date.

[ef4]

Seems like a reasonably self-contained issue for a motivated new contributor: after cloning, go find the package.json that has the right name in it, and use that as the root directory for the code indexing.

Most of the time you'll find the right package.json immediately at the root, but in a monorepo you'll search around a bit until you find it.

[kategengler]

It's a little more complicated than that: It would require a bit of an architecture change for the code search feature. We clone all the addons into a directory and remove any files we don't want to index (no option for excludes for cindex from google/codesearch. Nightly, new addons are cloned but existing addons are updated through git pull.

cindex, in theory, takes many args of all the directories to index, but I recall this not working as expected, so we'd have to experiment with that to go that way.

To do this without that feature of cindex, I think we could keep one directory of all addons on disk for cloning/updating and find + move the correct directories into a directory that is indexed by cindex

[ef4]

Ah, I see. Indexing runs all at once across all addons. So this would require changes to how the indexing process itself maps files to packages.