Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mistral int4 error #1330

Open
eddan168 opened this issue Aug 21, 2024 · 1 comment
Open

Mistral int4 error #1330

eddan168 opened this issue Aug 21, 2024 · 1 comment

Comments

@eddan168
Copy link

Describe the bug
A clear and concise description of what the bug is.
An update to the mistral_int4_optimize.json added an entry to the 'data_configs':

{'name': 'inc_static_quant_data_config', 'user_script': 'user_script.py', 'load_dataset_config': {'type': 'simple_dataset'}, 'dataloader_config': {'type': 'mistralai_calib_dataloader', 'batch_size': 1}}

It is missing 'type' key and the config.py access 'type' key directly and runs into error:

To Reproduce
Steps to reproduce the behavior.
python mistral.py --optimize --config mistral_int4_optimize.json

Expected behavior
A clear and concise description of what you expected to happen.
Should not have key error

Olive config
Add Olive configurations here.

Olive logs
Add logs here.

Traceback (most recent call last):
File "mistral.py", line 136, in
main()
File "mistral.py", line 125, in main
optimize(args.model_id, config)
File "mistral.py", line 62, in optimize
olive_run(olive_config)
File "/home/yongfan/Olive/olive/workflows/run/run.py", line 296, in run
run_config: RunConfig = RunConfig.parse_file_or_obj(run_config)
File "/home/yongfan/Olive/olive/common/config_utils.py", line 119, in parse_file_or_obj
return cls.parse_obj(file_or_obj)
File "/home/yongfan/miniconda3/envs/olive/lib/python3.8/site-packages/pydantic/v1/main.py", line 526, in parse_obj
return cls(**obj)
File "/home/yongfan/miniconda3/envs/olive/lib/python3.8/site-packages/pydantic/v1/main.py", line 339, in init
values, fields_set, validation_error = validate_model(pydantic_self.class, data)
File "/home/yongfan/miniconda3/envs/olive/lib/python3.8/site-packages/pydantic/v1/main.py", line 1074, in validate_model
v_, errors_ = field.validate(value, values, loc=field.alias, cls=cls_)
File "/home/yongfan/miniconda3/envs/olive/lib/python3.8/site-packages/pydantic/v1/fields.py", line 892, in validate
v, errors = self._validate_sequence_like(v, values, loc, cls)
File "/home/yongfan/miniconda3/envs/olive/lib/python3.8/site-packages/pydantic/v1/fields.py", line 925, in _validate_sequence_like
r, ee = self.validate_singleton(v, values, v_loc, cls)
File "/home/yongfan/miniconda3/envs/olive/lib/python3.8/site-packages/pydantic/v1/fields.py", line 1091, in _validate_singleton
value, error = field.validate(v, values, loc=loc, cls=cls)
File "/home/yongfan/miniconda3/envs/olive/lib/python3.8/site-packages/pydantic/v1/fields.py", line 864, in validate
v, errors = self._apply_validators(v, values, loc, cls, self.pre_validators)
File "/home/yongfan/miniconda3/envs/olive/lib/python3.8/site-packages/pydantic/v1/fields.py", line 1154, in _apply_validators
v = validator(cls, v, values, self, self.model_config)
File "/home/yongfan/miniconda3/envs/olive/lib/python3.8/site-packages/pydantic/v1/class_validators.py", line 306, in
return lambda cls, v, values, field, config: validator(cls, v, values=values)
File "/home/yongfan/Olive/olive/workflows/run/config.py", line 213, in validate_data_configs_with_hf_model
if v["type"] in TRANSFORMER_DUMMY_DATA_CONTAINER:
KeyError: 'type'

Other information

  • OS: [e.g. Windows, Linux]
  • Olive version: [e.g. 0.4.0 or main]
  • ONNXRuntime package and version: [e.g. onnxruntime-gpu: 1.16.1]
  • Transformers package version: [e.g. transformers 4.44.1]

Additional context
Add any other context about the problem here.

@xiaoyu-work
Copy link
Contributor

This error had been fixed. Can you try again with main branch?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants