Cache pytorch

[thequicksort]

Pytorch takes a while to compile (> 30 minutes) and is wildly computationally intensive. For developers, this is a huge pain-point.

While the nix philosophy is all about reproducibility, maybe it's okay if we ease up on that promise for developers by using cached versions of pytorch. It's a fairly popular library, so perhaps justifiable?

Here are a few (non-exhaustive) options:

1. Use PytorchWorld

One approach is to use PyTorchWorld: Nix scripts for pytorch related libraries

Advantages:

• Bespoke for Nix

Disadvantages:

• Python 3.6 only right now (Python 3.7 is on the way)
• Managed through Git Submodules (this introduces operational burden for developers)

2. Download Pre-builts

It's also possible for us to define the torch derivations locally in nix/pkgs and just pull pre-built versions from urls. This is described in post /#24 in Improving NixOS data science infrastructure

• generic.nix
• default.nix

Advantages:

• Independent of Python version
• No extra package dependencies

Disadvantages

• Need to specify build configurations for each OS, architecture (macos-cpu, linux-cpu, linux-gpu). Though based on the above posts, this isn't so bad.
• Not as straight-forward as having a single dependency handle the configuration.
• Requires maintenance for us (keeping versions in check, up-to-date, etc)

I'm actually leaning towards option 2.

• Introduce cached Pytorch

[thequicksort]

Potentially related: #41