Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump the pytorch group across 1 directory with 5 updates #413

Closed
wants to merge 2 commits into from

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Sep 25, 2024

Bumps the pytorch group with 5 updates in the /pytorch directory:

Package From To
peft 0.12.0 0.13.0
protobuf 5.28.1 5.28.2
tokenizers 0.19.1 0.20.0
mkl 2024.2.1 2024.2.2
mkl-include 2024.2.1 2024.2.2

Updates peft from 0.12.0 to 0.13.0

Release notes

Sourced from peft's releases.

LoRA+, VB-LoRA, and more

peft-v0 13 0

Highlights

New methods

LoRA+

@​kallewoof added LoRA+ to PEFT (#1915). This is a function that allows to initialize an optimizer with settings that are better suited for training a LoRA adapter.

VB-LoRA

@​leo-yangli added a new method to PEFT called VB-LoRA (#2039). The idea is to have LoRA layers be composed from a single vector bank (hence "VB") that is shared among all layers. This makes VB-LoRA extremely parameter efficient and the checkpoints especially small (comparable to the VeRA method), while still promising good fine-tuning performance. Check the VB-LoRA docs and example.

Enhancements

New Hugging Face team member @​ariG23498 added the helper function rescale_adapter_scale to PEFT (#1951). Use this context manager to temporarily increase or decrease the scaling of the LoRA adapter of a model. It also works for PEFT adapters loaded directly into a transformers or diffusers model.

@​ariG23498 also added DoRA support for embedding layers (#2006). So if you're using the use_dora=True option in the LoraConfig, you can now also target embedding layers.

For some time now, we support inference with batches that are using different adapters for different samples, so e.g. sample 1-5 use "adapter1" and samples 6-10 use "adapter2". However, this only worked for LoRA layers so far. @​saeid93 extended this to also work with layers targeted by modules_to_save (#1990).

When loading a PEFT adapter, you now have the option to pass low_cpu_mem_usage=True (#1961). This will initialize the adapter with empty weights ("meta" device) before loading the weights instead of initializing on CPU or GPU. This can speed up loading PEFT adapters. So use this option especially if you have a lot of adapters to load at the same time or if these adapters are very big. Please let us know if you encounter issues with this option, as we may make this the default in the future.

Changes

Safe loading of PyTorch weights

Unless indicated otherwise, PEFT adapters are saved and loaded using the secure safetensors format. However, we also support the PyTorch format for checkpoints, which relies on the inherently insecure pickle protocol from Python. In the future, PyTorch will be more strict when loading these files to improve security by making the option weights_only=True the default. This is generally recommended and should not cause any trouble with PEFT checkpoints, which is why with this release, PEFT will enable this by default. Please open an issue if this causes trouble.

What's Changed

... (truncated)

Commits

Updates protobuf from 5.28.1 to 5.28.2

Commits
  • 9fff46d Updating version.json and repo version numbers to: 28.2
  • ce60d01 Merge pull request #18385 from protocolbuffers/cp-lp-28
  • ac9fb5b Add recursion check when parsing unknown fields in Java.
  • 9a5f5fe Internal change
  • 50a7745 Internal change
  • 5b0e543 Fix cord handling in DynamicMessage and oneofs. (#18373)
  • 421fc16 Merge pull request #18343 from protocolbuffers/revert-18339-bazel-rules2
  • 607bfdd Revert "Cherry-pick changes related to new Bazel rules"
  • 106f4a6 Merge pull request #18339 from protocolbuffers/bazel-rules2
  • c2f34d6 Automated rollback of commit 76794bf3adceefcd69a2eb5785635a084fbe2e32.
  • Additional commits viewable in compare view

Updates tokenizers from 0.19.1 to 0.20.0

Release notes

Sourced from tokenizers's releases.

Release v0.20.0: faster encode, better python support

Release v0.20.0

This release is focused on performances and user experience.

Performances:

First off, we did a bit of benchmarking, and found some place for improvement for us! With a few minor changes (mostly #1587) here is what we get on Llama3 running on a g6 instances on AWS https://github.com/huggingface/tokenizers/blob/main/bindings/python/benches/test_tiktoken.py : image

Python API

We shipped better deserialization errors in general, and support for __str__ and __repr__ for all the object. This allows for a lot easier debugging see this:

>>> from tokenizers import Tokenizer;
>>> tokenizer = Tokenizer.from_pretrained("bert-base-uncased");
>>> print(tokenizer)
Tokenizer(version="1.0", truncation=None, padding=None, added_tokens=[{"id":0, "content":"[PAD]", "single_word":False, "lstrip":False, "rstrip":False, ...}, {"id":100, "content":"[UNK]", "single_word":False, "lstrip":False, "rstrip":False, ...}, {"id":101, "content":"[CLS]", "single_word":False, "lstrip":False, "rstrip":False, ...}, {"id":102, "content":"[SEP]", "single_word":False, "lstrip":False, "rstrip":False, ...}, {"id":103, "content":"[MASK]", "single_word":False, "lstrip":False, "rstrip":False, ...}], normalizer=BertNormalizer(clean_text=True, handle_chinese_chars=True, strip_accents=None, lowercase=True), pre_tokenizer=BertPreTokenizer(), post_processor=TemplateProcessing(single=[SpecialToken(id="[CLS]", type_id=0), Sequence(id=A, type_id=0), SpecialToken(id="[SEP]", type_id=0)], pair=[SpecialToken(id="[CLS]", type_id=0), Sequence(id=A, type_id=0), SpecialToken(id="[SEP]", type_id=0), Sequence(id=B, type_id=1), SpecialToken(id="[SEP]", type_id=1)], special_tokens={"[CLS]":SpecialToken(id="[CLS]", ids=[101], tokens=["[CLS]"]), "[SEP]":SpecialToken(id="[SEP]", ids=[102], tokens=["[SEP]"])}), decoder=WordPiece(prefix="##", cleanup=True), model=WordPiece(unk_token="[UNK]", continuing_subword_prefix="##", max_input_chars_per_word=100, vocab={"[PAD]":0, "[unused0]":1, "[unused1]":2, "[unused2]":3, "[unused3]":4, ...}))
>>> tokenizer
Tokenizer(version="1.0", truncation=None, padding=None, added_tokens=[{"id":0, "content":"[PAD]", "single_word":False, "lstrip":False, "rstrip":False, "normalized":False, "special":True}, {"id":100, "content":"[UNK]", "single_word":False, "lstrip":False, "rstrip":False, "normalized":False, "special":True}, {"id":101, "content":"[CLS]", "single_word":False, "lstrip":False, "rstrip":False, "normalized":False, "special":True}, {"id":102, "content":"[SEP]", "single_word":False, "lstrip":False, "rstrip":False, "normalized":False, "special":True}, {"id":103, "content":"[MASK]", "single_word":False, "lstrip":False, "rstrip":False, "normalized":False, "special":True}], normalizer=BertNormalizer(clean_text=True, handle_chinese_chars=True, strip_accents=None, lowercase=True), pre_tokenizer=BertPreTokenizer(), post_processor=TemplateProcessing(single=[SpecialToken(id="[CLS]", type_id=0), Sequence(id=A, type_id=0), SpecialToken(id="[SEP]", type_id=0)], pair=[SpecialToken(id="[CLS]", type_id=0), Sequence(id=A, type_id=0), SpecialToken(id="[SEP]", type_id=0), Sequence(id=B, type_id=1), SpecialToken(id="[SEP]", type_id=1)], special_tokens={"[CLS]":SpecialToken(id="[CLS]", ids=[101], tokens=["[CLS]"]), "[SEP]":SpecialToken(id="[SEP]", ids=[102], tokens=["[SEP]"])}), decoder=WordPiece(prefix="##", cleanup=True), model=WordPiece(unk_token="[UNK]", continuing_subword_prefix="##", max_input_chars_per_word=100, vocab={"[PAD]":0, "[unused0]":1, "[unused1]":2, ...}))

The pre_tokenizer.Sequence and normalizer.Sequence are also more accessible now:

from tokenizers import normalizers
norm = normalizers.Sequence([normalizers.Strip(), normalizers.BertNormalizer()])
norm[0]
norm[1].lowercase=False

What's Changed

... (truncated)

Commits
  • a5adaac version 0.20.0
  • a8def07 Merge branch 'fix_release' of github.com:huggingface/tokenizers into branch_v...
  • fe50673 Fix CI
  • b253835 push cargo
  • fc3bb76 update dependencies
  • bfd9cde Perf improvement 16% by removing offsets. (#1587)
  • bd27fa5 add deserialize for pre tokenizers (#1603)
  • 56c9c70 Tests + Deserialization improvement for normalizers. (#1604)
  • 49dafd7 Fix strip python type (#1602)
  • bded212 Support None to reset pre_tokenizers and normalizers, and index sequences (...
  • Additional commits viewable in compare view

Updates mkl from 2024.2.1 to 2024.2.2

Commits

Updates mkl-include from 2024.2.1 to 2024.2.2

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore <dependency name> major version will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)
  • @dependabot ignore <dependency name> minor version will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)
  • @dependabot ignore <dependency name> will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)
  • @dependabot unignore <dependency name> will remove all of the ignore conditions of the specified dependency
  • @dependabot unignore <dependency name> <ignore condition> will remove the ignore condition of the specified dependency and ignore conditions

dependabot bot and others added 2 commits September 23, 2024 15:57
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: tylertitsworth <tyler.titsworth@intel.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyler Titsworth <tyler.titsworth@intel.com>
Bumps the pytorch group with 5 updates in the /pytorch directory:

| Package | From | To |
| --- | --- | --- |
| [peft](https://github.com/huggingface/peft) | `0.12.0` | `0.13.0` |
| [protobuf](https://github.com/protocolbuffers/protobuf) | `5.28.1` | `5.28.2` |
| [tokenizers](https://github.com/huggingface/tokenizers) | `0.19.1` | `0.20.0` |
| [mkl](https://github.com/oneapi-src/oneMKL) | `2024.2.1` | `2024.2.2` |
| [mkl-include](https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl.html) | `2024.2.1` | `2024.2.2` |



Updates `peft` from 0.12.0 to 0.13.0
- [Release notes](https://github.com/huggingface/peft/releases)
- [Commits](huggingface/peft@v0.12.0...v0.13.0)

Updates `protobuf` from 5.28.1 to 5.28.2
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](protocolbuffers/protobuf@v5.28.1...v5.28.2)

Updates `tokenizers` from 0.19.1 to 0.20.0
- [Release notes](https://github.com/huggingface/tokenizers/releases)
- [Changelog](https://github.com/huggingface/tokenizers/blob/main/RELEASE.md)
- [Commits](huggingface/tokenizers@v0.19.1...v0.20.0)

Updates `mkl` from 2024.2.1 to 2024.2.2
- [Release notes](https://github.com/oneapi-src/oneMKL/releases)
- [Commits](https://github.com/oneapi-src/oneMKL/commits)

Updates `mkl-include` from 2024.2.1 to 2024.2.2

---
updated-dependencies:
- dependency-name: peft
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: pytorch
- dependency-name: protobuf
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: pytorch
- dependency-name: tokenizers
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: pytorch
- dependency-name: mkl
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: pytorch
- dependency-name: mkl-include
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: pytorch
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot bot added the dependencies Pull requests that update a dependency file label Sep 25, 2024
@dependabot dependabot bot requested a review from sharvil10 as a code owner September 25, 2024 15:53
@dependabot dependabot bot added the python Pull requests that update Python code label Sep 25, 2024
Copy link

Dependency Review

The following issues were found:
  • ✅ 0 vulnerable package(s)
  • ✅ 0 package(s) with incompatible licenses
  • ✅ 0 package(s) with invalid SPDX license definitions
  • ⚠️ 2 package(s) with unknown licenses.
See the Details below.

License Issues

pytorch/venv-requirements.txt

PackageVersionLicenseIssue Type
mkl-include2024.2.2NullUnknown License
mkl2024.2.2NullUnknown License

OpenSSF Scorecard

PackageVersionScoreDetails
pip/peft 0.13.0 UnknownUnknown
pip/protobuf 5.28.2 🟢 5.7
Details
CheckScoreReason
Binary-Artifacts🟢 10no binaries found in the repo
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
CI-Tests🟢 1023 out of 23 merged PRs checked by a CI test -- score normalized to 10
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Code-Review⚠️ 0found 29 unreviewed changesets out of 30 -- score normalized to 0
Contributors🟢 1012 different organizations found -- score normalized to 10
Dangerous-Workflow⚠️ 0dangerous workflow patterns detected
Dependency-Update-Tool🟢 10update tool detected
Fuzzing🟢 10project is fuzzed
License🟢 9license file detected
Maintained🟢 1030 commit(s) out of 30 and 3 issue activity out of 30 found in the last 90 days -- score normalized to 10
Packaging⚠️ -1no published package detected
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
Security-Policy🟢 10security policy file detected
Signed-Releases⚠️ 00 out of 5 artifacts are signed or have provenance
Token-Permissions🟢 10GitHub workflow tokens follow principle of least privilege
Vulnerabilities🟢 73 existing vulnerabilities detected
pip/tokenizers 0.20.0 🟢 5
Details
CheckScoreReason
Code-Review🟢 8Found 24/27 approved changesets -- score normalized to 8
Maintained🟢 1030 commit(s) and 17 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
License🟢 10license file detected
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Security-Policy⚠️ 0security policy file not detected
Binary-Artifacts🟢 10no binaries found in the repo
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Fuzzing⚠️ 0project is not fuzzed
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Packaging🟢 10packaging workflow detected
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
Vulnerabilities⚠️ 012 existing vulnerabilities detected
pip/mkl 2024.2.2 UnknownUnknown
pip/mkl-include 2024.2.2 UnknownUnknown

Scanned Manifest Files

pytorch/hf-genai-requirements.txt
  • peft@0.13.0
  • protobuf@5.28.2
  • tokenizers@0.20.0
  • peft@0.12.0
  • protobuf@5.28.1
  • tokenizers@0.19.1
pytorch/venv-requirements.txt
  • mkl@2024.2.2
  • mkl-include@2024.2.2
  • mkl@2024.2.1
  • mkl-include@2024.2.1

@tylertitsworth
Copy link
Contributor

@dependabot rebase

Copy link
Contributor Author

dependabot bot commented on behalf of github Sep 26, 2024

Looks like these dependencies are updatable in another way, so this is no longer needed.

@dependabot dependabot bot closed this Sep 26, 2024
@dependabot dependabot bot deleted the dependabot/pip/pytorch/pytorch-7e87a2c005 branch September 26, 2024 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file python Pull requests that update Python code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant