🐛 v3 crashes on iOS and macOS devices due to increasing memory usage

[GaLambert]

System Info

• transformers.js version: 3.2.2
• iOS: 18.3.2
• macOS: 15.3.1
• Framework: Next.js (14.2.22) and Vite (6.0.5)

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

We've encountered issues with the v3 of transformers.js (specifically v3.2.2, potentially others) on iOS devices and macOS. The application crashes on iOS (both Safari and Chrome), and causes extremely high and growing memory usage (10+GB) on macOS, eventually also crashing.

Tested with the following examples:

• https://github.com/huggingface/transformers.js-examples/tree/main/next-client
• https://github.com/huggingface/transformers.js-examples/tree/main/video-object-detection

Temporary solution: As a workaround, downgrading to @xenova/transformers@2.15.1 resolves the iOS crashing issue. However, this is not an ideal long-term solution.

We would greatly appreciate any insights or assistance in resolving these issues with the latest version. Thank you! 🙏

Reproduction

1. Use transformers.js version 3.2.2
2. Run a model (e.g., YoloV9) on an iOS device (iPhone)
3. Observe that the application crashes
4. Run the same model on macOS
5. Check Activity Monitor and observe memory usage exceeding 10GB+ for Safari

[xenova]

Thanks for the report. Could you try again with the latest version to see if this is still an issue? Currently v3.4.0

cc @fs-eire @guschmue too

[Jerboas86]

Thanks for the report. Could you try again with the latest version to see if this is still an issue? Currently v3.4.0

the bug is still there in v3.4.0

I also tried to use models with single threaded onnx runtime, the application still crashes.

[GaLambert]

After testing, I confirm that the bug persists in transformers.js v3.4.0.

[fs-eire]

This may be caused by JSC not doing well with multi-threaded WASM. Could you help to check when you use @xenova/transformers@2.15.1 and set numThreads to 4, what will the memory usage be like?

[Jerboas86]

With @huggingface/transformers v3.4 on ipadOs 18.3.2, using:

env.backends.onnx.wasm.numThreads = N;

• N = 1 -> Crash / 3.5Go memory usage
• N = 4 -> Crash / 3.5Go memory usage

[fs-eire]

I did a local WASM build for disabling multi-thread support. Could you please try it with transformers.js v3.4.0 with the following setup:

import { env } from "@huggingface/transformers";
env.backends.onnx.wasm.numThreads = 1;
env.backends.onnx.wasm.wasmPaths = {
  mjs: "<path-to>/ort-wasm-simd.mjs",
  wasm: "<path-to>/ort-wasm-simd.wasm",
}

ort-wasm-simd_20250319.zip

EDIT: it would be great helpful if you could help to test the memory usage data for using A, B and C.

env.backends.onnx.wasm.numThreads = 1;
env.backends.onnx.wasm.wasmPaths = {

  // A
  //mjs: '<path-to>/ort-wasm-simd.mjs',
  //wasm: '<path-to>/ort-wasm-simd.wasm',

  // B
  mjs: 'https://cdn.jsdelivr.net/npm/onnxruntime-web@1.21.0/dist/ort-wasm-simd-threaded.mjs',
  wasm: 'https://cdn.jsdelivr.net/npm/onnxruntime-web@1.21.0/dist/ort-wasm-simd-threaded.wasm'

  // C
  //mjs: 'https://cdn.jsdelivr.net/npm/onnxruntime-web@1.21.0/dist/ort-wasm-simd-threaded.jsep.mjs',
  //wasm: 'https://cdn.jsdelivr.net/npm/onnxruntime-web@1.21.0/dist/ort-wasm-simd-threaded.jsep.wasm'
};

[GaLambert]

In my setup:
On installing v3.4.0, and debugging the env, the env.backends is undefined on starting my worker that initialises the model and makes inferences.
I can add env.backends.onnx.numThreads = 1, and indicate the local .mjs and .wasm files though, but it doesn't influence the application

https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.4.0/dist/ort-wasm-simd-threaded.jsep.wasm is automatically installed on starting the worker.

Where the memory goes up again to 10+GB on macOS Safari

[fs-eire]

Could you please try this branch:

https://github.com/fs-eire/transformers.js-examples/tree/investigate-issue-1242

It works on both of my windows and mac but I didn't reproduce the memory issue on both.

[GaLambert]

That indeed fixes the issue @fs-eire . Thanks!

[fs-eire]

That indeed fixes the issue @fs-eire . Thanks!

Thank you for the test. Which option fixes the issue, A or B?

[GaLambert]

I tested A (with the 2 files from the example repo) and B. Both solve the issue.