Robust javascript dependency handling in the browser

[hafen]

This may be more of a question for @jcheng5 and equally relevant to shiny, but as I've been following some recent issues (#83, #58, etc.), and as I've been dealing with an app that tries to allow dynamic specification of different htmlwidgets to view, I've been thinking a lot about dependency injection for htmlwidgets / shiny in general.

As the number of widgets grows and apps or docs with multiple widgets that potentially interact with each other become more commonplace, there are bound to be issues where simply handling dependencies by sticking a script tag in head for each dependency will eventually run into problems, such as collisions (different widgets want different versions of the same dependency, different widgets declare global variables with the same name but different functionality). This can be skirted by using iframes but then you can't have interactivity between the widgets. Have there been any discussions or are there any thoughts on using something like requirejs or doing some kind of per-widget namespacing?

[ramnathv]

@jcheng5 will have more advice to offer, but I don't think a universal solution to this problem exists even in the javascript world. There are so many competing standards, and I have not found anything that works out of the box.

Having said that, more robust dependency resolution is something I have been thinking about as well. Case in point is a widget I wrote, which uses select components from bootstrap. As a hack, I name the dependency bootstrap, but specify the version as 0.1 so that if a full version of bootstrap is available on the page, this will get overruled. So some concept of priority in addition to version would be useful.

[jcheng5]

The problem is that the requirejs patterns would need to be followed not only in the widget binding JS, but in the JS libraries themselves. And at the end of the day I'm not sure the npm-style solution of everyone dragging along their own copies of dependencies, is much better than the traditional-style "dependency hell" (I mean that as a technical term, not a pejorative 😄) that we have today.

Are you actually running into problems with conflicting versions of the same JS libraries being required, where the newer one isn't backwards compatible enough with the older? (Granted this is a big problem with mainly-CSS libraries like Bootstrap, but requirejs is going to do nothing for you there...)

[hafen]

Thanks @jcheng5 and @ramnathv.

I may have an edge use case here that motivated the question. In my Trelliscope shiny app, my goal is to fully support any htmlwidget to be embedded in the application. So a user may specify several different plots, all using different htmlwidgets with different dependencies. In one instance of an app, the user could potentially switch from one display to another, requiring the widget's dependencies to be loaded on the fly in the browser. I'm sure at some point this will lead to a namespace issue.

A related issue that actually I guess is more about dynamic loading of assets is that I've been working on making this happen with Shiny.renderDependencies and HTMLWidgets.staticRender (see here). This works for some widgets (like rbokeh). But, for example, with threejs, there is a global THREEJS that does not get loaded through this process. When I use something like .$getScript it does load properly.

I'm starting to think the answer to both of these is to just use iframes for each widget. @ramnathv, do you know when the plan is to merge that feature?

[ramnathv]

@hafen I hope to have the iframe branch ready for review in a couple of weeks. It just needs some minor tweaks and documentation on the limitations. Does this fit with your schedule for trelliscope?

[hafen]

Sure - that would be great!

[jcheng5]

@hafen There was an issue with rthreejs that caused the THREEJS object not to be visible--it was fixed maybe 3 weeks ago. If you haven't tried since then, maybe give it another shot.

We've been smashing plenty of widgets together in rmarkdown documents and for the most part, things work OK. Of course since everything is being loaded into the same namespace it's possible there will be conflicts--this is no different than doing any other work in the browser though. You can go iframe if you want to but to me that feels like a solution of last resort, especially if you're using Shiny since it makes communication with R more complicated (I think--I haven't thought about it much).

[hafen]

Thanks @jcheng5. I agree it would be nice to avoid iframes. Thanks for the pointer on rthreejs - it's probably been that long or longer since I updated it. I wonder how many widgets the eval() thing applies to. It seems like I had some trouble with some other widgets, but it's been a while so I'll re-check. I guess any widgets I might have trouble with will also be troublesome for renderUI, so hopefully that will make any issues I have more mainstream.