Use a different package manager

[joshwcomeau]

Currently, to keep things simple, Guppy just uses the system NPM to install dependencies.

It would be great to find a more efficient solution, though. I have my eye on pnpm (https://github.com/pnpm/pnpm), a manager that helps avoid the bloat of new projects having several hundred MB in node_modules. Especially for aspiring developers who may be on lower-end machines with less total disk space, being able to significantly shrink the size of having multiple projects seems like a huge win, and I imagine the installation speed would go way up as well!

[joshwcomeau]

This is actually super important since NPM 5.6.0 has a bug that will cause dependencies to become corrupted when npm install-ing. This is a huge issue with Guppy because when running the eject task, the dependencies become corrupted, and none of the scripts work (they won't work in the terminal either, so it's not a Guppy bug, but in the terminal you can just remove all of node_modules and reinstall, which isn't an option in Guppy).

Maybe using yarn is a safer bet; pnpm is exciting but we'd need to make sure to test it extensively, it's not as battle-tested.

[cudba]

I think the user needs control over the used package manager. You can run into problems (e.g CI) if you add dependencies with yarn while others on the project use npm (package-lock.json). Checking if theres a npm / yarn -lock.json and pick this as default could make sense. And have an 'App Default package manager' for new apps maybe (with some info on which to pick for what kind of setups)?

[Leodau]

Kindda like the 'App Package Manager' idea, good oportunity to teach people which to use depending on the wanted stack, NPM & Yarn could be a good start.

#59 Think is not a bad idea tho.

[karlsander]

I don't know how much overhead adding support for multiple package managers would add, but I would assume quite a lot, since guppy becomes more valuable the more closely it interacts with the package manager. I would just pick either yarn or npm and build on that. Picking between two package managers, that tbh sound identically to a developer that hasn't experienced the issues of using npm before, seems confusing without adding any value.

I really like what pnpm is doing but I don't know if its really aligns well with the goal of this project? This is probably a question of belief, but I feel like the "all your deps are right there in your project folder" model is much better for beginners:

• It makes it clear that dependencies are not magic that somehow get injected into your project but code, just like yours, that you can look at (even tinker with), right in the project you have open in VS Code (though with pnpm's hardlinks you can look at it as well)
• Having everything contained in the guppy executable and the project folder exposes users to less debugging of issues that aren't part of their code. Is a macOS update safe? Can I fully restore from a Time Machine backup? Can I use Migration Assistant to replace my computer? These are things I think about with various developer tools, but if its just the Guppy.app and ~/Documents/myapp, the answer is clear.
• It would be awesome to support running in sandboxed environments in the far future. I'm thinking of Mac App Store apps, dockerified linux apps on Chrome OS or Windows 10S, which is sold to schools.

About file size: Is it common at all to have more than 20 CRA or Gatsby apps lying around? Does anyone make a computer with less than 128GB of storage that can comfortably run an Electron app all the time while working?

The increase in speed would be valuable though, since it would also help with the annoyance of locking while install actions are running.

I've never run into blocking issues with npm personally, and I think npm audit (fix) would be a very neat module to add to guppy, but If there's a bug affecting guppy right now, and with yarn being generally more stable (?) it might be the better choice.

[cudba]

you re right, by default the user shouldnt be forced to deal with the package manager. And going with 1 keeps things simpler.
Still for deeper integration into guppy, you re probably dependant on the installed version of npm/yarn? maybe abstracting away the actual npm/yarn scripts from the integrated package manager actions might be a good way. you could provide the scripts according to the installed version, and stay open to the addition of additional package managers.

I feel like the world of package managers / dependencies, linting, bundling and so on can be really frustrating. But eventually, ppl have to deal with it. I feel like there's where guppy could be really educational and help just do things, but also teach how its done and maybe even why ;)

keeping it somewhat close to the actual implementation is really valuable in my opinion

[joshwcomeau]

Hey folks, thanks for the discussion!

I really like what pnpm is doing but I don't know if its really aligns well with the goal of this project?

Totally agree @karlsander. Giving it some thought, it is really valuable to have the deps right there in the project.

If storage space becomes an issue for folks with many many projects, we could add a "Delete local dependencies" button in the UI so they can quickly delete deps on old projects. But yeah, I'd rather wait and see if this is an issue.

I feel like the world of package managers / dependencies, linting, bundling and so on can be really frustrating. But eventually, ppl have to deal with it. I feel like there's where guppy could be really educational and help just do things, but also teach how its done and maybe even why ;)

Yeah, so there's a short-term vision and a long-term vision.

Short term is this: "Guppy should make it as simple as possible to jump into React web development by distilling all of the chores into simple actions. Flexibility and configuration aren't as important as clarity."

Longer term is: "Guppy should be a stepping stone that helps aspiring developers build the skills they need to have a career as a professional front-end engineer"

I think the brand-new experience should always hide package managers from the user. In the future, I can imagine "unlocking" additional features, so that after you've been using Guppy for a while, these ideas are introduced and new controls are added to the UI.

But yeah, thats well into the future, and who knows if that'll ever happen. So for now, I think we shouldn't provide any choices for package managers.

---

As for actually selecting one, here's what I'm thinking:

• NPM has some bugs that Yarn doesn't have. This has come up a few times now, most recently in [WIP] Windows Support #63, where NPM throws a weird error but Yarn doesn't, when running tasks in Windows.

• Yarn's original purpose was to ensure that every dependency installation is the same, by using a lockfile to track exactly what's installed. While NPM does this now too, with package-lock, it's a core part of Yarn's philosophy.

• NPM has an audit feature which is super helpful! Looks like yarn might get it too eventually. Let's assume it'll come to Yarn and we can use it once we get to that point in the priorities.

• I really like pnpm's dependency installation speed, but maybe we can implement a simpler version ourselves. If a new project is created with create-react-app version X, we can maybe copy those fresh dependencies to a Guppy directory. The next time a project is created, we can start by copying over the dependencies from that Guppy directory.

For that last point, it's not clear to me that it's a good idea, and is certainly not a priority for now. But I feel OK skipping pnpm because its killer feature can maybe be implemented ourselves.

Does anyone want to take on converting NPM to Yarn? I think this is still a top-priority issue, especially if it's blocking Windows support!

[superhawk610]

I'll take the initial work, but I would love some help on testing. I'll have the WIP up here later this evening on the npm-to-yarn branch. The main changes I foresee are:

• updating all instances of npm to yarn
• updating all npm-specific commands to their yarn equivalent (eg - install to add)
• installing yarn globally after creating a new project for the first time if it is not already
• check for existing package manager usage with has-yarn and display a compatibility warning for Windows users not using yarn
• updating the readme/documentation to reference yarn instead of npm

[j-f1]

Yarn is typically installed globally rather than per-project, so I’m not sure why it would be added as a dependency.

[superhawk610]

That's true, but there would still be other conflicts (package-lock.json vs yarn.lock), and I can't confirm off the top of my head but I think I've had issues in the past mixing and matching between npm and yarn in the same project.

[Haroenv]

There’s a node module “has-yarn” by @sindresorhus to detect if a project is already using yarn, could be useful? Or maybe it’s contentious enough to be an option?

[superhawk610]

@Haroenv that looks perfect, I'll update the to-do list.

[joshwcomeau]

Supporting NPM projects

Hm, so yeah, there is the usecase where someone imports a project that uses the NPM cli for dependencies, and then adding dependencies with Guppy uses yarn, and that's not ideal...

I'm not super worried about that case, though; or, more accurately, if Yarn can solve our short-term problems, I'm happy leaving this issue for the future (maybe later we can fix the issues with NPM and then use has-yarn to be smarter about which package manager Guppy should use, but that feels lower-priority to me).

Checklist - local vs. global Yarn installation

@superhawk610 for the most part, the checklist looks pretty good to me!

@j-f1 mentioned that Yarn is typically installed globally, and that's true... but I kinda feel like we should break with the norm and install it locally (not locally in the user's project, but local to Guppy's bundled node_modules).

If the user already has yarn installed globally, it feels wrong to install our own (and potentially change its version). We could do a check to only install it if it isn't already installed, but then subtle bugs might crop up, since their global installation might be very old. I can already imagine the non-reproducible issues folks will open because they're on a year-old Yarn.

I like the idea of Guppy being self-contained (originally I wanted to use the Node that comes with Electron rather than have the user need to download Node, but there were problems with that idea that I no longer remember...). If it owns the local installation, it has much more control over the experience, and has less potential to disrupt the user's other, non-guppy projects.

Unanswered questions:

• Are there any caveats to using local Yarn?
• How do we specify when running commands to use the local installation, in a cross-platform way? On MacOS I imagine we could just do ./node_modules/yarn instead of yarn for commands, but yeah, needs to be tested on other platforms.

@j-f1 let me know if you know of any caveats to local Yarn (or if you disagree with something I've said)

[AWolf81]

@superhawk610 I've pushed a first version of npm to yarn at my fork here.

I've tested:

• Create new project
• Add dependency
• Remove dependency

Todo:

• Should constant PACKAGE_MANAGER inside package-manager.service in a config file at a different location?
• Check local usage of yarn
• Look through all files for npm usage.

[superhawk610]

Oh awesome! My home server decided to have a meltdown last night so I didn't get a chance to do much. I'll take a look and see if I can help.

[joshwcomeau]

This was completed!