mupdf not working with Next.js RouteHandler #90

arwtyxouymz · 2024-06-07T00:39:56Z

Hi there,

I also posted this on Discord and the member suggested me to create the issue here :)
I'm currently trying to use mupdf with Next.js RouteHandler, but I'm facing difficulties making it work. I would appreciate any help.

Overview

I am attempting to use mupdf within Next.js RouteHandler.
The test is very simple: fetch a PDF and convert it to PNG for the response.
I've tried both runtime = edge and runtime = nodejs, but both result in errors.
(I prefer to use edge runtime because my project is related to LLM work).

Next.js: 14.2.3(I tried 13.5.6, but the result was same)
mupdf: 0.2.2

Target file

The api is very simple. Just getting pdf file from /public directory, and load and convert it to png by using mupdf, and return it to client.

import { NextRequest, NextResponse } from "next/server";
import * as mupdf from "mupdf";

export const runtime = "edge";

export async function GET(req: NextRequest) {
  const res = await fetch("http://localhost:3000/dummy.pdf");
  const document = mupdf.Document.openDocument(
    await res.arrayBuffer(),
    "application/pdf"
  );
  const page = document.loadPage(0);
  const pixmap = page.toPixmap(
    mupdf.Matrix.scale(2, 2),
    mupdf.ColorSpace.DeviceRGB,
    false,
    true
  );
  const pngImage = pixmap.asPNG();

  return new NextResponse(pngImage, {
    headers: { "Content-Type": "image/png" },
  });
}

Errors

runtime=edge

  
pnpm dev

> [email protected] dev /Users/arwtyxouymz/work/sandbox/mupdf-edge-test
> next dev

  ▲ Next.js 14.2.3
  - Local:        http://localhost:3000

 ✓ Starting...
 ✓ Ready in 2.7s
 ○ Compiling /_not-found ...
 ✓ Compiled /_not-found in 2.9s (464 modules)
 GET /sample.pdf 404 in 3007ms
 ✓ Compiled in 107ms (236 modules)
 ✓ Compiled in 133ms (236 modules)
 ⨯ ./node_modules/.pnpm/[email protected]/node_modules/mupdf/dist/mupdf-wasm.js:100:27
Module not found: Can't resolve 'module'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
./node_modules/.pnpm/[email protected]/node_modules/mupdf/dist/mupdf.js
./app/api/mupdf/route.ts
./node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/next/dist/build/webpack/loaders/next-edge-app-route-loader/index.js?absolutePagePath=%2FUsers%2Farwtyxouymz%2Fwork%2Fsandbox%2Fmupdf-edge-test%2Fapp%2Fapi%2Fmupdf%2Froute.ts&page=%2Fapi%2Fmupdf%2Froute&appDirLoader=bmV4dC1hcHAtbG9hZGVyP25hbWU9YXBwJTJGYXBpJTJGbXVwZGYlMkZyb3V0ZSZwYWdlPSUyRmFwaSUyRm11cGRmJTJGcm91dGUmYXBwUGF0aHM9JnBhZ2VQYXRoPXByaXZhdGUtbmV4dC1hcHAtZGlyJTJGYXBpJTJGbXVwZGYlMkZyb3V0ZS50cyZhcHBEaXI9JTJGVXNlcnMlMkZhcnd0eXhvdXlteiUyRndvcmslMkZzYW5kYm94JTJGbXVwZGYtZWRnZS10ZXN0JTJGYXBwJnBhZ2VFeHRlbnNpb25zPXRzeCZwYWdlRXh0ZW5zaW9ucz10cyZwYWdlRXh0ZW5zaW9ucz1qc3gmcGFnZUV4dGVuc2lvbnM9anMmcm9vdERpcj0lMkZVc2VycyUyRmFyd3R5eG91eW16JTJGd29yayUyRnNhbmRib3glMkZtdXBkZi1lZGdlLXRlc3QmaXNEZXY9dHJ1ZSZ0c2NvbmZpZ1BhdGg9dHNjb25maWcuanNvbiZiYXNlUGF0aD0mYXNzZXRQcmVmaXg9Jm5leHRDb25maWdPdXRwdXQ9JnByZWZlcnJlZFJlZ2lvbj0mbWlkZGxld2FyZUNvbmZpZz1lMzAlM0Qh&nextConfigOutput=&preferredRegion=&middlewareConfig=e30%3D!
 ○ Compiling /api/mupdf ...
 ⚠ ./node_modules/.pnpm/[email protected]/node_modules/mupdf/dist/mupdf.js
The generated code contains 'async/await' because this module is using "topLevelAwait".
However, your target environment does not appear to support 'async/await'.
As a result, the code may not run as expected or may cause runtime errors.

Import trace for requested module:
./node_modules/.pnpm/[email protected]/node_modules/mupdf/dist/mupdf.js
./app/api/mupdf/route.ts
./node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/next/dist/build/webpack/loaders/next-app-loader.js?name=app%2Fapi%2Fmupdf%2Froute&page=%2Fapi%2Fmupdf%2Froute&appPaths=&pagePath=private-next-app-dir%2Fapi%2Fmupdf%2Froute.ts&appDir=%2FUsers%2Farwtyxouymz%2Fwork%2Fsandbox%2Fmupdf-edge-test%2Fapp&pageExtensions=tsx&pageExtensions=ts&pageExtensions=jsx&pageExtensions=js&rootDir=%2FUsers%2Farwtyxouymz%2Fwork%2Fsandbox%2Fmupdf-edge-test&isDev=true&tsconfigPath=tsconfig.json&basePath=&assetPrefix=&nextConfigOutput=&preferredRegion=&middlewareConfig=e30%3D!./app/api/mupdf/route.ts?__next_edge_ssr_entry__
 GET /api/mupdf 500 in 1835ms
 ✓ Compiled /_error in 148ms (419 modules)

runtime=nodejs

pnpm dev

> [email protected] dev /Users/arwtyxouymz/work/sandbox/mupdf-edge-test
> next dev

  ▲ Next.js 14.2.3
  - Local:        http://localhost:3000

 ✓ Starting...
 ✓ Ready in 1546ms
 ⨯ ./node_modules/.pnpm/[email protected]/node_modules/mupdf/dist/mupdf-wasm.js:114:1
Module not found: Can't resolve './'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
./node_modules/.pnpm/[email protected]/node_modules/mupdf/dist/mupdf.js
./app/api/mupdf/route.ts
 ○ Compiling /_not-found ...
 ⨯ ./node_modules/.pnpm/[email protected]/node_modules/mupdf/dist/mupdf-wasm.js:114:1
Module not found: Can't resolve './'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
./node_modules/.pnpm/[email protected]/node_modules/mupdf/dist/mupdf.js
./app/api/mupdf/route.ts
 ✓ Compiled /_not-found in 100ms (236 modules)
 ○ Compiling /_error ...
 ⨯ ./node_modules/.pnpm/[email protected]/node_modules/mupdf/dist/mupdf-wasm.js:114:1
Module not found: Can't resolve './'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
./node_modules/.pnpm/[email protected]/node_modules/mupdf/dist/mupdf.js
./app/api/mupdf/route.ts
 GET /api/mupdf 500 in 2651m

How to reproduce

I create the minimum reproduce repo.
https://github.com/arwtyxouymz/mupdf-edge-test
You can try it just like this.

Updated: 06/11
I updated my repo to be able to test easily.

$ pnpm dev

# In another session
# Route Handler
## => Not work
$ curl http://localhost:3000/api/mupdf/edge
## => Not work
$ curl http://localhost:3000/api/mupdf/nodejs

# API Routes
## => Not work
$ curl http://localhost:3000/api/pages/mupdf/edge
## => WORK!!
$ curl http://localhost:3000/api/pages/mupdf/nodejs

Thank you for your help and for maintaining this great project!

The text was updated successfully, but these errors were encountered:

jamie-lemon · 2024-06-07T13:05:47Z

Okay - this is not related to using Next.js or the RouteHandler - we had a previous issue here: #12 - you can also checkout the react-example branch to validate the same errors you are seeing ( see examples/mupdf-react after checking out that branch ) .
We have mupdf.js working in the browser with the node modules, but more with vanilla JS - see examples/annotations-test for how that is done. I don't know if this helps you with your environment though!

We need to figure out how to resolve this "module not found" issue, to be honest I don't comprehend the complexities around it. I will assign some engineers to this Issue who might understand this better.

arwtyxouymz · 2024-06-08T01:37:44Z

@jamie-lemon Thank you for the response and providing examples.
I found it works with next.js API Routes with nodejs runtime.
I don't understand why API routes with nodejs runtime works even though it doesn't work Route Handler with nodejs runtime.

I updated the https://github.com/arwtyxouymz/mupdf-edge-test and you can try it by:

$ pnpm dev
# This causes error (using Route Handler)
$ curl http://localhost:3000/api/mupdf
# This works (using API Routes with nodejs runtime)
$ curl http://localhost:3000/api/pages/mupdf

I hope this helps your investigation. Thanks.

mfts · 2024-06-08T07:45:03Z

@arwtyxouymz @jamie-lemon I solved it by explicitly adding the mupdf wasm files in Next.js config

https://github.com/mfts/papermark/blob/main/next.config.mjs#L14

Maybe that solves your issue

arwtyxouymz · 2024-06-08T15:44:59Z

@mfts

Actually, I read your article and tried your solution before creating the issue.
But it didn't work with app router. (You use pages router)
And your mupdf version is v0.1.* but i use v0.2.*.
I tried your repo code with mupdf v0.2 and app router, it didn't work.

mfts · 2024-06-09T08:20:12Z

That's a good point. I'll try it with 0.2 and work with it

jamie-lemon · 2024-06-10T12:07:32Z

@arwtyxouymz Thanks for the feedback - I can confirm that I can also see it working with your API Routes with nodejs runtime solution.
"I don't understand why API routes with nodejs runtime works even though it doesn't work Route Handler with nodejs runtime."

I also don't understand this :) . All I know is that the library works fine under the node js environment, which explains why the "API routes with nodejs runtime" would work in this case. I have asked @mipo1357 to look into this issue further and hopefully he might have some ideas ( he is more of a React programmer than I am ;) )

mipo1357 · 2024-06-24T00:24:20Z

Thanks for pointing out the problem. I am attempting to investigate here but am unable to provide a solution. Please give me some more time.

arwtyxouymz · 2024-07-05T11:37:08Z

@mipo1357 @jamie-lemon
I could make mupdf work with runtime=nodejs in Route Handler by just adding mupdf to serverComponentsExternalPackages

/** @type {import('next').NextConfig} */
const nextConfig = {
  experimental: {
    serverComponentsExternalPackages: ['mupdf'],
  },
};

export default nextConfig;

mfts · 2024-07-05T14:33:49Z

Thanks @arwtyxouymz for your example repo

I successfully updated mupdf from v0.1.3 to v0.2.2.

I made the following changes:

next.config.js: change to /dist/ from /lib/

 experimental: {
  outputFileTracingIncludes: {
    "/api/mupdf/*": ["./node_modules/mupdf/dist/*.wasm"],
  },

pages/api/mupdf.ts: import statement
```
import * as mupdf from "mupdf";
```
tsconfig.json: change to bundler
```
"moduleResolution": "bundler",
```

next.js 14.2.4
node 20.8.0
typescript 5.5.3

mipo1357 · 2024-07-09T08:01:09Z

Thanks for the info. I will check these actions!

Southclaws · 2024-07-17T10:59:48Z

I'm running into this now, same settings as what @mfts has shared but still getting the following error on build:

Failed to compile.

./node_modules/mupdf/dist/mupdf-wasm.js
Module not found: Can't resolve 'module'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
./node_modules/mupdf/dist/mupdf.js
./src/screens/pitch/usePitchScreen.ts
./src/screens/pitch/PitchScreen.tsx
./src/screens/pitch/PitchScreenModal.tsx
./src/screens/profile/ProfileScreen.tsx

mfts · 2024-07-20T10:36:13Z

@Southclaws bundler, next.config is set like mine above?
And you're on mupdf@^0.2.2 ?

Southclaws · 2024-08-14T17:32:46Z

I did get this working eventually on 0.2.2 inside a route handler (not a page handler) - though now we've switched to doing this in our Go backend using https://github.com/klippa-app/go-pdfium

jamie-lemon assigned ccxvii and mipo1357 Jun 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mupdf not working with Next.js RouteHandler #90

mupdf not working with Next.js RouteHandler #90

arwtyxouymz commented Jun 7, 2024 •

edited

Loading

jamie-lemon commented Jun 7, 2024

arwtyxouymz commented Jun 8, 2024 •

edited

Loading

mfts commented Jun 8, 2024

arwtyxouymz commented Jun 8, 2024

mfts commented Jun 9, 2024

jamie-lemon commented Jun 10, 2024

mipo1357 commented Jun 24, 2024

arwtyxouymz commented Jul 5, 2024

mfts commented Jul 5, 2024 •

edited

Loading

mipo1357 commented Jul 9, 2024

Southclaws commented Jul 17, 2024

mfts commented Jul 20, 2024

Southclaws commented Aug 14, 2024

mupdf not working with Next.js RouteHandler #90

mupdf not working with Next.js RouteHandler #90

Comments

arwtyxouymz commented Jun 7, 2024 • edited Loading

Overview

Target file

Errors

How to reproduce

jamie-lemon commented Jun 7, 2024

arwtyxouymz commented Jun 8, 2024 • edited Loading

mfts commented Jun 8, 2024

arwtyxouymz commented Jun 8, 2024

mfts commented Jun 9, 2024

jamie-lemon commented Jun 10, 2024

mipo1357 commented Jun 24, 2024

arwtyxouymz commented Jul 5, 2024

mfts commented Jul 5, 2024 • edited Loading

mipo1357 commented Jul 9, 2024

Southclaws commented Jul 17, 2024

mfts commented Jul 20, 2024

Southclaws commented Aug 14, 2024

arwtyxouymz commented Jun 7, 2024 •

edited

Loading

arwtyxouymz commented Jun 8, 2024 •

edited

Loading

mfts commented Jul 5, 2024 •

edited

Loading