Support for other languages

[filips123]

There should also be support for other languages:

• PHP (already some project)
• Java (already some project)
• C/C++
• Node.js

This is because some of them are very used in web development and some of them provide research tools. There are already some projects to compile them to WebAssembly.

[hamilton]

@filips123 we'd love to see an example with any of these that have been compiled to WebAssembly! In some cases, adding a language is quite easy. In others, it is often much more difficult. The details matter a lot. Our team doesn't have a lot of bandwidth to add other languages, so we're hoping the community can contribute here. Once you have a language compiled, making a plugin for Iodide is pretty straightforward. Take a look at our docs here: https://iodide-project.github.io/docs/language_plugins/

[filips123]

@hamilton I didn't test any of them so I can't provide you any examples.

[filips123]

@hamilton

I will try to do something for PHP.

There is already project to compile PHP to WASM, but it needs to be added as plugin. I created oraoto/pib#23 for this.
I currently prototype plugin in https://gist.github.com/filips123/8d6135debb763374fac5078bcee28542 but it currently doesn't work.

Do you know how to fix those errors? Also, how can communication (level 2+) be implemented between language and JS?

[hamilton]

@filips123 fantastic - I'll take a look at this once my work week has gotten under way. I think @mdboom may be able to give you some pointers re: level 2 and beyond (or point you to the source where he handles things in Pyodide, for inspiration). Worth noting that this kind of wasm use case is relatively new, so there aren't really that many domain experts here. You may have to make your own fire!

[mdboom]

@filips123: Thanks for working on this. As to the specific error -- The gist is loading php.js from https://oraoto.github.io/pib/php.js, but that .js file is loading the .data (which contains the tree of files for PHP) from a relative URL which is just php.data. Since the webpage is at alpha.iodide.io, it tries to load https://alpha.iodide.io/php.data and it fails. Fortunately, you can change how emscripten looks up the .data files at runtime using the locateFile option. I've forked and updated your gist here to show you what I mean.

With that, I was able to get the php plugin to load. Behind that there seem to be some other issues:

• evaluate doesn't return anything. That's where the result of whatever was run on the PHP side would need to be converted to something Javascript and returned.
• The renderer sets shouldHandle to always true. This means that this handler will always handle all output for Iodide, which is probably not what you want. It should only return true when the value is something is PHP-specific. You may not need it at all, if, for example, the PHP evaluate function always returns Javascript strings.

To answer your question about how Level 2 can be acheived: that's very specific on the implementation of the language itself. In Python, for example, there is a whole C API to handle Python data types, and this can be used (and even called directly from Javascript) to convert Python data types to Javascript data types. I don't know anything about the implementation of PHP, but if you're lucky, it has something similar. You can look here for how that's done in Python, but it would mostly have to be converted to PHP-specific calls.

[filips123]

Some problem with PHP plugin is that it sometimes needs some time to initialize. In the original project, the module will call some function after it is initialized (source - see init function and phpModuleOptions). How can this be done in Iodide?

---

evaluate doesn't return anything. That's where the result of whatever was run on the PHP side would need to be converted to something Javascript and returned.

What should evaluate return? Should it be the output (stdout and stderr) of the script (echo) or return value?

The problem is that execution is async. When execution is done, it will call window.php.export that will add any output (stdout and stderr) to report preview in <div>. Is that OK?

---

The renderer sets shouldHandle to always true. This means that this handler will always handle all output for Iodide, which is probably not what you want. It should only return true when the value is something is PHP-specific. You may not need it at all, if, for example, the PHP evaluate function always returns Javascript strings.

I know... I just add it until output will have better handling.

---

I will try to do level 2 later, because first level 1 needs to be done :)

[mdboom]

Some problem with PHP plugin is that it sometimes needs some time to initialize. In the original project, the module will call some function after it is initialized (source - see init function and phpModuleOptions). How can this be done in Iodide?

I think you could just follow the pattern you linked to -- set the postRun configuration option to the function you want to call when emscripten has finished loading the module. But maybe I'm missing something?

What should evaluate return? Should it be the output (stdout and stderr) of the script (echo) or return value?

It should be the return value. stdout and stderr are by default sent to console.log by emscripten. Right now, that means they go to the developer console, but a future version of Iodide will display those in the Iodide UI.

[filips123]

I think you could just follow the pattern you linked to -- set the postRun configuration option to the function you want to call when emscripten has finished loading the module. But maybe I'm missing something?

Yes, but what should postRun function do? How to inform Iodide that module has finished loading?

It should be the return value. stdout and stderr are by default sent to console.log by emscripten. Right now, that means they go to the developer console, but a future version of Iodide will display those in the Iodide UI.

If I am correct, the return value can't be changed by the script. It is just "compiler status", so when the code is correct it is "0".

[mdboom]

Yes, but what should postRun function do? How to inform Iodide that module has finished loading?

I'm sorry, I misunderstood the question. Iodide expects the language plugin code to define a Promise called languagePluginLoader which resolves when done.

var languagePluginLoader = new Promise((resolve, reject) => {
   // do all your initialization here, and call `resolve()` when done...
}

[mdboom]

If I am correct, the return value can't be changed by the script. It is just "compiler status", so when the code is correct it is "0".

The evaluate function should return the return value of whatever is run, not just a success/fail. If that is what the PHP wasm library you are using does, that would have to be changed in order to the get the return value out.

[hamilton]

@filips123 any progress on this front?

[filips123]

@hamilton I currently don't have much time, but I will continue working. I contacted author of PHP In Browser about questions for return value and level 2.

[sdwfrost]

Has anyone made a plugin for Typescript? Happy to give it a go, but dont want to duplicate effort

[hamilton]

No one to my knowledge has done so! A typescript plug-in would be welcome.

…

On Sat, Mar 23, 2019 at 11:08 AM Simon Frost ***@***.***> wrote: Has anyone made a plugin for Typescript? Happy to give it a go, but dont want to duplicate effort — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1637 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAF1945LlzDH5R_ZqHvmesNpOwlVh9LIks5vZm2WgaJpZM4b2-lQ> .

[bcolloran]

i'm going to close this issue because it's a bit broad (and more than we can tackle in general), but please do feel free to file issues regarding difficulties you encounter during specific language implementations!

[filips123]

@bcolloran Maybe create some tracking system for language suggestions and implementations.

[bcolloran]

@filips123 i'm open to suggestions about how to track that, but to expand on this a little: we are a very a small team, and unfortunately we don't have the resources to respond to suggestions that we implement language X. What we can do is offer support when someone from the community undertakes an implementation of language X -- if there are specific bugs or questions that come up in the process of such an implementation, then we can try to help with specific issues. but as much as we would love to do so, we just don't have the resources to help with issue like "please implement language X", so they just end up as noise in the tracker and dilute the signal. I hope that makes sense

I suppose we could do something like create a repo of language plugins or language plugin definitions so that there is a place where people can go (in addition to our docs https://iodide-project.github.io/docs/language_plugins/ ) to see what language plugins exist and discuss implementations etc. I'm not sure something like that would really be helpful in the end, but we can discuss ideas like that if you wish.

[filips123]

I suppose we could do something like create a repo of language plugins or language plugin definitions

Something like this would probably be good.