[Web] Error when using Web Workers on Next.js

[illbexyz]

Describe the issue

I get the following error when using onnxruntime-web on Next.js with web workers:

• Error: no available backend found. ERR: [wasm] TypeError: Failed to fetch, [cpu] Error: multiple calls to 'initWasm()' detected.

I had no issues with [email protected].

I believe the issue is that Next.js is trying to load the file from the filesystem:

• See Request URL in the following screenshot file:///Users/alberto.nicoletti/node_modules/onnxruntime-web/dist/ort.bundle.min.mjs

Additional notes:

• I have no issues when not using web workers
• I obtain a slightly different error when disabling the proxy but setting onnx.env.wasm.numThreads = 2
  • Error: no available backend found. ERR: [wasm] SecurityError: Failed to construct 'Worker': Script at 'file:///Users/alberto.nicoletti/node_modules/onnxruntime-web/dist/ort.bundle.min.mjs' cannot be accessed from origin 'http://localhost:3000'., [cpu] Error: previous call to 'initWasm()' failed..
  • In this case there are no network requests being made but I believe these issues are likely related.

Network requests screenshot:

To reproduce

The following reproductions are a bare Next.js 14 starter repository with [email protected].

• Proxy error reproduction here: https://github.com/illbexyz/nextjs-onnxruntime/tree/wasm-proxy
  • See https://github.com/illbexyz/nextjs-onnxruntime/blob/wasm-proxy/src/pages/components/OnnxButton.tsx where I use a web worker by enabling onnx.env.wasm.proxy = true.
• Multi-thread reproduction here: https://github.com/illbexyz/nextjs-onnxruntime/tree/wasm-threads
  • See https://github.com/illbexyz/nextjs-onnxruntime/blob/wasm-threads/src/pages/components/OnnxButton.tsx where I use onnx.env.wasm.numThreads = 2.
  • In this reproduction I also added crossOriginIsolation headers here: https://github.com/illbexyz/nextjs-onnxruntime/blob/wasm-threads/next.config.mjs

Urgency

Kinda urgent: every Next.js project is impacted.

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.19.2

Execution Provider

'wasm'/'cpu' (WebAssembly CPU)

[satyajandhyala]

@illbexyz Can you try using onnxruntime 1.19 in a clean environment?

[illbexyz]

@satyajandhyala This is already the most minimal reproduction I could give. It's just onnxruntime-web added to a bare Next.js project initialized with npx create-next-app@latest.

[fs-eire]

Script at 'file:///Users/alberto.nicoletti/node_modules/onnxruntime-web/dist/ort.bundle.min.mjs' cannot be accessed from origin 'http://localhost:3000'.

This is a known issue. If you use webpack loading onnxruntime-web as ESM, you need this line in your webpack config:

module: { parser: { javascript: { importMeta: false } } },

(see https://github.com/Microsoft/onnxruntime/blob/abdc31de401262bcb03f538423389c2eb264a0ce/js/web/test/e2e/webpack.config.esm-js.js)

Explaination: onnxruntime-web use import.meta.url at runtime to determine the script path so that it is able to create worker with correct path. However the default behavior of Webpack rewrites import.meta.url into a static local file path ( in your case file:///Users/alberto.nicoletti/node_modules/onnxruntime-web/dist/ort.bundle.min.mjs) and it won't work at runtime. the highlighted line in the config disabled this specific behavior of webpack.

May be a good idea to update our document/examples.

[illbexyz]

Hey @fs-eire, thanks for your time!

I tried adding the following to next.config.mjs:

webpack: (config) => {
  config.module.parser.javascript.importMeta = false;

  return config;
},

But now I get a different error:

SyntaxError: Cannot use 'import.meta' outside a module

I updated the reproductions if you wanna have a look.