Update main branch #4

manuelbrack · 2025-02-24T08:12:47Z

What does this PR do?

Fixes # (issue)

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline?
Did you read our philosophy doc (important for complex PRs)?
Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Update overview.md

add 2K related model for Sana

…rained (#10299) Update gguf.md to remove generator from the pipeline from_pretrained

* Flux IP-Adapter * test cfg * make style * temp remove copied from * fix test * fix test * v2 * fix * make style * temp remove copied from * Apply suggestions from code review Co-authored-by: YiYi Xu <yixu310@gmail.com> * Move encoder_hid_proj to inside FluxTransformer2DModel * merge * separate encode_prompt, add copied from, image_encoder offload * make * fix test * fix * Update src/diffusers/pipelines/flux/pipeline_flux.py * test_flux_prompt_embeds change not needed * true_cfg -> true_cfg_scale * fix merge conflict * test_flux_ip_adapter_inference * add fast test * FluxIPAdapterMixin not test mixin * Update pipeline_flux.py Co-authored-by: YiYi Xu <yixu310@gmail.com> --------- Co-authored-by: YiYi Xu <yixu310@gmail.com>

* flux controlnet inpaint config bug * Update src/diffusers/pipelines/flux/pipeline_flux_controlnet_inpainting.py --------- Co-authored-by: yigitozgenc <yigit@quantuslabs.ai> Co-authored-by: hlky <hlky@hlky.ac>

…10344) update docs and example to use community weights

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

add: q

* update * Update src/diffusers/loaders/single_file_utils.py Co-authored-by: Aryan <aryan@huggingface.co> --------- Co-authored-by: Aryan <aryan@huggingface.co>

* fix the Positinoal Embedding bug in 2K model; * Change the default model to the BF16 one for more stable training and output * make style * substract buffer size * add compute_module_persistent_sizes --------- Co-authored-by: yiyixuxu <yixu310@gmail.com>

Add missing `.shape`

* bump min torchao version to 0.7.0 * update

fix: torchao example.

refactor

* sana lora training tests and misc. * remove push to hub * Update examples/dreambooth/train_dreambooth_lora_sana.py Co-authored-by: Aryan <aryan@huggingface.co> --------- Co-authored-by: Aryan <aryan@huggingface.co>

update

* misc lora test improvements. * updates * fixes to tests

fix

* rename blocks and docs * fix docs --------- Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

updates

* fixes to tests * fixture * fixes

* update * make style * update * update * update * make style * single file related changes * update * fix * update single file urls and docs * update * fix

* post release 0.32.0 * stylew

fix

Update community_projects.md

* fix bug for torch.uint1-7 not support in torch<2.6 * up --------- Co-authored-by: baymax591 <cbai@mail.nwpu.edu.cn>

…10346) Make passing the IP Adapter mask to the attention mechanism optional if there is no need to apply it to a given IP Adapter.

* make set_adapters() robust on silent failures. * fixes to tests * flaky decorator. * fix * flaky to sd3. * remove warning. * sort * quality * skip test_simple_inference_with_text_denoiser_multi_adapter_block_lora * skip testing unsupported features. * raise warning instead of error.

* first draft model loading refactor * revert name change * fix bnb * revert name * fix dduf * fix huanyan * style * Update src/diffusers/models/model_loading_utils.py Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * suggestions from reviews * Update src/diffusers/models/modeling_utils.py Co-authored-by: YiYi Xu <yixu310@gmail.com> * remove safetensors check * fix default value * more fix from suggestions * revert logic for single file * style * typing + fix couple of issues * improve speed * Update src/diffusers/models/modeling_utils.py Co-authored-by: Aryan <aryan@huggingface.co> * fp8 dtype * add tests * rename resolved_archive_file to resolved_model_file * format * map_location default cpu * add utility function * switch to smaller model + test inference * Apply suggestions from code review Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * rm comment * add log * Apply suggestions from code review Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * add decorator * cosine sim instead * fix use_keep_in_fp32_modules * comm --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: YiYi Xu <yixu310@gmail.com> Co-authored-by: Aryan <aryan@huggingface.co>

* feat: introduce a style bot. * updates * Apply suggestions from code review Co-authored-by: Guillaume LEGENDRE <glegendre01@gmail.com> * apply suggestion * fixes * updates --------- Co-authored-by: Guillaume LEGENDRE <glegendre01@gmail.com>

remove prints

use proper gemma class and config in lumina2 tests.

* feat: lora support for Lumina2. * fix-copies. * updates * updates * docs. * fix * add: training script. * tests * updates * updates * major updates. * updates * fixes * docs. * updates * updates

@Mishig

…documented (#7763) * add; utility to check if attn_procs,norms,acts are properly documented. * add support listing to the workflows. * change to 2024. * small fixes. * does adding detailed docstrings help? * uncomment image processor check * quality * fix, thanks to @Mishig. * Apply suggestions from code review Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> * style * JointAttnProcessor2_0 * fixes * fixes * fixes * fixes * fixes * fixes * Update docs/source/en/api/normalization.md Co-authored-by: hlky <hlky@hlky.ac> --------- Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> Co-authored-by: hlky <hlky@hlky.ac>

* poc encode_prompt() tests * fix * updates. * fixes * fixes * updates * updates * updates * revert * updates * updates * updates * updates * remove SDXLOptionalComponentsTesterMixin. * remove tests that directly leveraged encode_prompt() in some way or the other. * fix imports. * remove _save_load * fixes * fixes * fixes * fixes

* store cls instead of an obj * style

)

Add 5 Notebooks, improve their example scripts and update the missing links for the example README.

install accelerate transformers from .

* run fast gpu tests conditionally on pull requests. * revert unneeded changes. * simplify PR.

* Added runtime checkpoint conversion * Updated docs * Fix for quantized model

* update * update

* update * make fix-copies * update * tests * update * update * add co-author Co-Authored-By: Langdx <82783347+Langdx@users.noreply.github.com> * add co-author Co-Authored-By: howe <howezhang2018@gmail.com> * update --------- Co-authored-by: Langdx <82783347+Langdx@users.noreply.github.com> Co-authored-by: howe <howezhang2018@gmail.com>

* fix: run tests from a pr workflow. * correct * update * checking.

template for remote vae.

fix

update

* update * update --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* `device_map` in `load_model_dict_into_meta` * _LOW_CPU_MEM_USAGE_DEFAULT * fix is_peft_version is_bitsandbytes_version

Fix docs

remove check

* lora * update * update --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* More robust from_pretrained init_kwargs type checking * Corrected for Python 3.10 * Type checks subclasses and fixed type warnings * More type corrections and skip tokenizer type checking * make style && make quality * Updated docs and types for Lumina pipelines * Fixed check for empty signature * changed location of helper functions * make style --------- Co-authored-by: hlky <hlky@hlky.ac>

…10816) * Fix `torch_dtype` in Kolors text encoder with `transformers` v4.49 * Default torch_dtype and warning

stevhliu and others added 30 commits December 20, 2024 08:30

[docs] Fix quantization links (#10323)

7d4db57

Update overview.md

[Sana]add 2K related model for Sana (#10322)

a6288a5

add 2K related model for Sana

[Docs] Update gguf.md to remove generator from the pipeline from_pret…

d413881

…rained (#10299) Update gguf.md to remove generator from the pipeline from_pretrained

Fix push_tests_mps.yml (#10326)

a756694

Fix EMAModel test_from_pretrained (#10325)

bf9a641

flux controlnet inpaint config bug (#10291)

233dffd

* flux controlnet inpaint config bug * Update src/diffusers/pipelines/flux/pipeline_flux_controlnet_inpainting.py --------- Co-authored-by: yigitozgenc <yigit@quantuslabs.ai> Co-authored-by: hlky <hlky@hlky.ac>

Community hosted weights for diffusers format HunyuanVideo weights (#…

6aaa051

…10344) update docs and example to use community weights

Fix enable_sequential_cpu_offload in test_kandinsky_combined (#10324)

f615f00

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

update get_parameter_dtype (#10342)

7c2f0af

add: q

[Single File] Add Single File support for HunYuan video (#10320)

da21d59

* update * Update src/diffusers/loaders/single_file_utils.py Co-authored-by: Aryan <aryan@huggingface.co> --------- Co-authored-by: Aryan <aryan@huggingface.co>

.from_single_file() - Add missing .shape (#10332)

3c2e2aa

Add missing `.shape`

Bump minimum TorchAO version to 0.7.0 (#10293)

ffc0eaa

* bump min torchao version to 0.7.0 * update

[docs] fix: torchao example. (#10278)

6a970a4

fix: torchao example.

[tests] Refactor TorchAO serialization fast tests (#10271)

02c777c

refactor

[SANA LoRA] sana lora training tests and misc. (#10296)

76e2727

* sana lora training tests and misc. * remove push to hub * Update examples/dreambooth/train_dreambooth_lora_sana.py Co-authored-by: Aryan <aryan@huggingface.co> --------- Co-authored-by: Aryan <aryan@huggingface.co>

[Single File] Fix loading (#10349)

5fcee4a

update

[Tests] QoL improvements to the LoRA test suite (#10304)

c34fc34

* misc lora test improvements. * updates * fixes to tests

Fix FluxIPAdapterTesterMixin (#10354)

71cc201

Fix failing CogVideoX LoRA fuse test (#10352)

055d955

fix

Rename LTX blocks and docs title (#10213)

9d27df8

* rename blocks and docs * fix docs --------- Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

[LoRA] test fix (#10351)

ea1ba0b

updates

[Tests] Fix more tests sayak (#10359)

851dfa3

* fixes to tests * fixture * fixes

[core] LTX Video 0.9.1 (#10330)

4b55713

* update * make style * update * update * update * make style * single file related changes * update * fix * update single file urls and docs * update * fix

[chore] post release 0.32.0 (#10361)

92933ec

* post release 0.32.0 * stylew

fix test pypi installation in the release workflow (#10360)

9d2c8d8

fix

[Docs] Added model search to community_projects.md (#10358)

c1e7fd5

Update community_projects.md

make style for #10368 (#10370)

6dfaec3

* fix bug for torch.uint1-7 not support in torch<2.6 * up --------- Co-authored-by: baymax591 <cbai@mail.nwpu.edu.cn>

Make passing the IP Adapter mask to the attention mechanism optional (#…

c0c1168

…10346) Make passing the IP Adapter mask to the attention mechanism optional if there is no need to apply it to a given IP Adapter.

sayakpaul and others added 28 commits February 19, 2025 14:33

Remove print statements (#10836)

f8b54cf

remove prints

[tests] use proper gemma class and config in lumina2 tests. (#10828)

0fb7068

use proper gemma class and config in lumina2 tests.

[LoRA] add LoRA support to Lumina2 and fine-tuning script (#10818)

f10d3c6

* feat: lora support for Lumina2. * fix-copies. * updates * updates * docs. * fix * add: training script. * tests * updates * updates * major updates. * updates * fixes * docs. * updates * updates

Add missing isinstance for arg checks in GGUFParameter (#10834)

5321712

store activation cls instead of function (#10832)

a4c1aac

* store cls instead of an obj * style

fix: support transformer models' generation_config in pipeline (#10779

c7a8c43

)

Notebooks for Community Scripts-7 (#10846)

5194138

Add 5 Notebooks, improve their example scripts and update the missing links for the example README.

[CI] install accelerate transformers from main (#10289)

1f85350

install accelerate transformers from .

[CI] run fast gpu tests conditionally on pull requests. (#10310)

454f82e

* run fast gpu tests conditionally on pull requests. * revert unneeded changes. * simplify PR.

SD3 IP-Adapter runtime checkpoint conversion (#10718)

d9ee387

* Added runtime checkpoint conversion * Updated docs * Fix for quantized model

Some consistency-related fixes for HunyuanVideo (#10835)

f070775

* update * update

fix: run tests from a pr workflow. (#9696)

1871a69

* fix: run tests from a pr workflow. * correct * update * checking.

[chore] template for remote vae. (#10849)

9055ccb

template for remote vae.

fix remote vae template (#10852)

6cef7d2

fix

[CI] Fix incorrectly named test module for Hunyuan DiT (#10854)

2b2d042

update

[CI] Update always test Pipelines list in Pipeline fetcher (#10856)

b27d4ed

* update * update --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

device_map in load_model_dict_into_meta (#10851)

d75ea3c

* `device_map` in `load_model_dict_into_meta` * _LOW_CPU_MEM_USAGE_DEFAULT * fix is_peft_version is_bitsandbytes_version

[Fix] Docs overview.md (#10858)

85fcbaf

Fix docs

remove format check for safetensors file (#10864)

ffb6777

remove check

[docs] LoRA support (#10844)

64dec70

* lora * update * update --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

Fix torch_dtype in Kolors text encoder with transformers v4.49 (#…

6f74ef5

…10816) * Fix `torch_dtype` in Kolors text encoder with `transformers` v4.49 * Default torch_dtype and warning

manuelbrack self-assigned this Feb 24, 2025

manuelbrack merged commit c229826 into ml-research:main Feb 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update main branch #4

Update main branch #4

manuelbrack commented Feb 24, 2025

Update main branch #4

Update main branch #4

Conversation

manuelbrack commented Feb 24, 2025

What does this PR do?

Before submitting

Who can review?