Session requires model path?

[lilhoser]

I want to load a model from a byte array per the InferenceSession(Byte[]) prototype as specified in the docs. For example:

using (var session = new InferenceSession(m_ModelData))
{

This throws an exception:

Microsoft.ML.OnnxRuntime.OnnxRuntimeException: '[ErrorCode:RuntimeException] Exception during initialization: D:\a\_work\1\s\onnxruntime\core\optimizer\initializer.cc:31 onnxruntime::Initializer::Initializer !model_path.IsEmpty() was false. model_path must not be empty. Ensure that a path is provided when the model is created or loaded.
'

What am I missing here?

[wschin]

Do you store the initializer as external data? If so, you can convert external data to normal data by following this example:

import onnx
from onnx.external_data_helper import load_external_data_for_model

onnx_model = onnx.load("path/to/the/model.onnx", load_external_data=False)
load_external_data_for_model(onnx_model, "data/directory/path/")
# Then the onnx_model has loaded the external data from the specific directory
onnx.save(onnx_model, "path/to/model")

[lilhoser]

I'm not sure. I got this model from hf: https://huggingface.co/nenkoru/alpaca-lora-7b-onnx-fp32-no-past

It loads fine if I use new InferenceSession("path/to/model.onnx"), it fails if I try new InferenceSession(binary_data) where binary_data is the binary contents of the file read using File.ReadAllBytes.

by the way, I'm using ONNX in c# not python.

[wschin]

I check that model. It indeed uses external data. Per ONNX spec, a path is required in such case.

The current workaround is to use the above Python script to inliune external data to the model file.

More specifically,

import onnx
from onnx.external_data_helper import load_external_data_for_model

onnx_model = onnx.load("decoder_model.onnx", load_external_data=False)
# Not sure if it's `decoder_model.onnx_data` or the folder containing `decoder_model.onnx_data`.
# Please try both.
load_external_data_for_model(onnx_model, "decoder_model.onnx_data")
# Then the onnx_model has loaded the external data from the specific directory
onnx.save(onnx_model, "desired_model.onnx")

Then, in your C# code, use desired_model.onnx instead of the original one.

[lilhoser]

Thanks for the code snippet. I ran and got this exception:

  File "C:\Downloads\hf_env\lib\site-packages\onnx\__init__.py", line 231, in save_model
    s = _serialize(proto)
  File "C:\Downloads\hf_env\lib\site-packages\onnx\__init__.py", line 72, in _serialize
    result = proto.SerializeToString()
ValueError: Message onnx.ModelProto exceeds maximum protobuf size of 2GB: 26956774946

[wschin]

Humm, it's protobuf limitation. There are several potential solutions/workarounds.

• Use PrePackedWeightsContainer.
• Move all onnx_model.graph.initializer to onnx_model.graph.input and feed those initializers as inputs when launching InferenceSession.
• Implement new API which takes bytes and a folder path containing external data (seems the same to just load from model directly).

Not sure which one fits best to your scenario.

[brahim7]

Hi guys Hi @wschin I have same issue :
[E:onnxruntime:, inference_session.cc:1533 onnxruntime::InferenceSession::Initialize::<lambda_9a5ue43270b854edk3er320c0a5c4y9a>::operator ()] Exception during initialization: D:\a_work\1\s\onnxruntime\core\optimizer\initializer.cc:31 onnxruntime::Initializer::Initializer !model_path.IsEmpty() was false. model_path must not be empty. Ensure that a path is provided when the model
is created or loaded.

Is there a way to call ORTModelForCausalLM from optimum.onnxruntime inside pipeline of transformers.js (this is in python, )does it exit in transformers.js?

example :

import { pipeline, ORTModelForCausalLM } from ‘@xenova/transformers’; ??

let pipe = await pipeline(‘text-generation’,repo_id from huggingface);
let output = await pipe(text, {
temperature: 2,
max_new_tokens: 100,
repetition_penalty: 1.5,
no_repeat_ngram_size: 2,
num_beams: 2,
num_return_sequences: 2,
});
??
I called same files in python and all works good, but not with transformers.js (see error above)

i have onnx files Is the concept of a "PrePackedWeightsContainer" exist in standard ONNX?

some of my files : decoder_model.onnx_data. decoder_model.onnx (and samee with past model) , tokenizer.model, tokenizer.json, config.json...etc

Thank you