NuSLERP

[cg123]

Adds a new merge method nuslerp. This method allows for a superset of the functionality of slerp. If provided with a base model nuslerp will perform spherical interpolation of the task vectors. While the original slerp always flattens weight tensors into a single dimension nuslerp can also do row-wise and column-wise interpolation of tensors.

This method remedies one of my long-standing gripes with how I implemented slerp. Instead of taking a t parameter and using base_model to specify which is the "first" model, nuslerp simply takes a weight parameter for each model and computes the interpolation factor t internally. This makes it fit the conventions of the other merge methods much better. The weight parameter behaves in the same fashion as it does for merge_method: linear with normalize: true.

The idea to add task vector SLERP is inspired by DeepMind's great use of it in their WARP paper.

[jpeek34556]

I sincerely think that the spherical visualization of SLERP and the 3D axis within could be the forerunner to future model merging GUI. A kind of DAW from music producing but now we're mixing models. Visualization is the psychological ignition. A picture is worth 1000 words. Kudos on nuslerp. ref.: https://en.wikipedia.org/wiki/Phase_space