Add ResNet to models exportable with ONNX #16948

chamidullinr · 2022-04-26T14:48:51Z

What does this PR do?

I added OnnxConfig to make ResNet model available for ONNX conversion.

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline,
Pull Request section?
Was this discussed/approved via a Github issue or the forum? Please add a link
to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

HuggingFaceDocBuilderDev · 2022-04-26T15:04:22Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

chainyo · 2022-04-27T10:04:19Z

src/transformers/models/resnet/configuration_resnet.py

+
+    @property
+    def atol_for_validation(self) -> float:
+        return 1e-4


Hey @chamidullinr nice PR, did you try the conversion config with the default --atol=1e-5 before putting it to 1e-4 ?

If you are okay you could push a ResNet ONNX converted model to the ONNXConfig for all organisation, it would be awesome! 🤗

I pushed a ResNet-50 ONNX converted model. And I set --atol=1e-3 because some tests were failing.

lewtun

Thanks for adding the first non-Transformer model to our ONNX exporter @chamidullinr !

Apart from @chainyo's comment, this all looks good to me, so can you please check that the slow tests pass by running:

RUN_SLOW=1 pytest tests/onnx/test_onnx_v2.py -s -k "resnet"

Gently pinging @sgugger for final approval 🤗

sgugger

Thanks for the PR. LGTM module existing comments :-)

chamidullinr · 2022-04-27T14:53:01Z

Thanks for the suggestions. One test fails when running slow tests. The error is:

ValueError: Outputs values doesn't match between reference model and ONNX exported model: Got max absolute difference of: 0.000244140625

Should I set --atol=1e-3 or is there some way to fix this?

lewtun · 2022-04-27T16:58:34Z

Should I set --atol=1e-3 or is there some way to fix this?

Ah yes, some models require a lower tolerance due to their architectures. Setting --atol=1e-3 as the default is fine!

* add roformer onnx config

* Avoid repeated per-lang filtering * Language groups and logits preprocessing * Style

* Add first draft * Improve script and README * Improve README * Apply suggestions from code review * Improve script, add link to resulting model * Add corresponding test * Adjust learning rate

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

1. Fixes evaluation errors popping up when you train/eval on squad v2 (one was newly encountered and one that was previously reported Running SQuAD 1.0 sample command raises IndexError huggingface#15401 but not completely fixed). 2. Removes boolean arguments that don't use store_true. Please, don't use these: *ANY non-empty string is being converted to True in this case and this clearly is not the desired behavior (and it creates a LOT of confusion). 3. All no-trainer test scripts are now saving metric values in the same way (with the right prefix eval_), which is consistent with the trainer-based versions. 4. Adds forgotten model.eval() in the no-trainer versions. This improved some results, but not everything (see the discussion in the end). Please, see the F1 scores and the discussion below.

…#16946) * Add fix * Apply suggestion from code review Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

* Fix `distributed_concat` with scalar tensor * Update trainer_pt_utils.py

BertModelForSequenceClassification -> BertForSequenceClassification

…gingface#16966)

…ace#16947) * Fix multiple deletions of the same files in save_pretrained * Add is_main_process argument

* Fixup all examples

* Fix doc notebooks links * Remove missing section

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Add -e flag * add check * create new keys * run python setup.py build install * add comments * change to develop Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

The emoji version must be either 0.5.4 or 0.6.0. Newer emoji versions have been updated to newer versions of the Emoji Charts, thus not consistent with the one used for pre-processing the pre-training Tweet corpus (i.e. not consistent with the vocab).

github-actions · 2022-05-26T15:02:07Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

chainyo · 2022-06-03T12:17:11Z

Can't we rebase this branch and add the code for ResNet ?

regisss · 2022-06-07T15:50:19Z

I'm taking care of this in #17585, I didn't see this PR before opening mine. @chainyo

It should be done by the end of the week!

chainyo · 2022-06-07T16:24:30Z

It should be done by the end of the week!

Pretty cool!

Add ResNet to models exportable with ONNX

e8da6c1

LysandreJik requested review from lewtun and michaelbenayoun April 26, 2022 20:58

chainyo reviewed Apr 27, 2022

View reviewed changes

lewtun approved these changes Apr 27, 2022

View reviewed changes

sgugger approved these changes Apr 27, 2022

View reviewed changes

Krishna Sirumalla and others added 19 commits April 28, 2022 08:18

Add onnx config for RoFormer (huggingface#16861)

6c70ba7

* add roformer onnx config

documentation: some minor clean up (huggingface#16850)

98b7f23

Fix RuntimeError message format (huggingface#16906)

197c466

use original loaded keys to find mismatched keys (huggingface#16920)

4407c88

[Research] Speed up evaluation for XTREME-S (huggingface#16785)

0d0a862

* Avoid repeated per-lang filtering * Language groups and logits preprocessing * Style

Add semantic script, trainer (huggingface#16834)

4f181f5

* Add first draft * Improve script and README * Improve README * Apply suggestions from code review * Improve script, add link to resulting model * Add corresponding test * Adjust learning rate

Fix HubertRobustTest PT/TF equivalence test on GPU (huggingface#16943)

7a7275f

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

[HF Argparser] Fix parsing of optional boolean arguments (huggingface…

26470f8

…#16946) * Add fix * Apply suggestion from code review Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

Fix distributed_concat with scalar tensor (huggingface#16963)

6fed2b4

* Fix `distributed_concat` with scalar tensor * Update trainer_pt_utils.py

Update custom_models.mdx (huggingface#16964)

5cfc020

BertModelForSequenceClassification -> BertForSequenceClassification

Fix add-new-model-like when model doesn't support all frameworks (hug…

c20f2c6

…gingface#16966)

Fix multiple deletions of the same files in save_pretrained (huggingf…

58db6e4

…ace#16947) * Fix multiple deletions of the same files in save_pretrained * Add is_main_process argument

Fixup no_trainer save logic (huggingface#16968)

4f10667

* Fixup all examples

Fix doc notebooks links (huggingface#16969)

7e5c6d1

* Fix doc notebooks links * Remove missing section

Fix check_all_models_are_tested (huggingface#16970)

0dcd254

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Add -e flag to some GH workflow yml files (huggingface#16959)

a653a3f

* Add -e flag * add check * create new keys * run python setup.py build install * add comments * change to develop Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Changed atol in ResNet ONNX configuration

b8a7d52

chamidullinr closed this Jun 3, 2022

chamidullinr deleted the add-resnet-onnxconfig branch June 3, 2022 10:36

regisss mentioned this pull request Jun 9, 2022

Add ONNX support for ResNet #17585

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ResNet to models exportable with ONNX #16948

Add ResNet to models exportable with ONNX #16948

chamidullinr commented Apr 26, 2022

HuggingFaceDocBuilderDev commented Apr 26, 2022

chainyo Apr 27, 2022 •

edited

Loading

chamidullinr Apr 28, 2022

lewtun left a comment

sgugger left a comment

chamidullinr commented Apr 27, 2022

lewtun commented Apr 27, 2022

github-actions bot commented May 26, 2022

chainyo commented Jun 3, 2022

regisss commented Jun 7, 2022

chainyo commented Jun 7, 2022

Add ResNet to models exportable with ONNX #16948

Add ResNet to models exportable with ONNX #16948

Conversation

chamidullinr commented Apr 26, 2022

What does this PR do?

Before submitting

Who can review?

HuggingFaceDocBuilderDev commented Apr 26, 2022

chainyo Apr 27, 2022 • edited Loading

Choose a reason for hiding this comment

chamidullinr Apr 28, 2022

Choose a reason for hiding this comment

lewtun left a comment

Choose a reason for hiding this comment

sgugger left a comment

Choose a reason for hiding this comment

chamidullinr commented Apr 27, 2022

lewtun commented Apr 27, 2022

github-actions bot commented May 26, 2022

chainyo commented Jun 3, 2022

regisss commented Jun 7, 2022

chainyo commented Jun 7, 2022

chainyo Apr 27, 2022 •

edited

Loading