Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Convert Pretrained models to safetensors format #271

Open
1 task done
agm-eratosth opened this issue Feb 7, 2025 · 6 comments
Open
1 task done

[Feature] Convert Pretrained models to safetensors format #271

agm-eratosth opened this issue Feb 7, 2025 · 6 comments
Labels
type: feature New feature or enhancement

Comments

@agm-eratosth
Copy link

agm-eratosth commented Feb 7, 2025

Is there an existing feature already?

  • Yes, I have checked the existing features.

Description

Hi,

Can you please convert all pretrained models to safetensors? I noticed https://github.com/TorchDSP/torchsig/blob/main/examples/13_example_pretrained_models.ipynb contains .pth, .ckpt etc and it would be much better, logistically, if this was in safetensors.

Additionally, is there full list of all pretrained models? Are the models listed in 13_example_pretrained_models.ipynb the extensive list?

Thank you.

@agm-eratosth agm-eratosth added the type: feature New feature or enhancement label Feb 7, 2025
@MattCarrickPL
Copy link
Collaborator

Thanks for reporting, discussing internally now.

@MattCarrickPL
Copy link
Collaborator

Do YOLO models support safetensors? If there's a way to do this please let us know.

@MattCarrickPL
Copy link
Collaborator

Answering this part of your question:

Are the models listed in 13_example_pretrained_models.ipynb the extensive list?

Yes

@MattCarrickPL
Copy link
Collaborator

No response after a week so I'm closing this out. Happy to discuss conversion to safetensors but need some assistance to get there. If you have insights please let us know and we can discuss further.

@agm-eratosth
Copy link
Author

agm-eratosth commented Feb 21, 2025

Hi @MattCarrickPL - apologies. You brought up a valid point with the way the Ultralytics models/library works. It would likely need to be converted back. For now this format works until I can compare what might happen when converting to safetensors.

However this also brings up a couple of follows I have; how do you plan on versioning your pretrained files?

Do you think you could include a files like ubuntu has here: https://releases.ubuntu.com/24.10/ ? Basically:

https://bucket.ltsnet.net/torchsig/models/models.list

then:
https://bucket.ltsnet.net/torchsig/SHA256SUMS

  • would contain each model and a SHA256 or SHA512 checksum, e.g:
  • 123456789ABCDEF *yolo_detect.pt

https://bucket.ltsnet.net/torchsig/models/SHA256SUMS.gpg

  • would contain PGP signature

This would make it easier to verify downloads similar to: https://ubuntu.com/tutorials/how-to-verify-ubuntu

Even just starting with the models.list and checksums without pgp would be good so we can match versions up by hash.

Thanks.

@MattCarrickPL
Copy link
Collaborator

This is a good recommendation. I'm not sure if we will do checksums. Would adding a release number to the model filename get you the versioning you're looking for? We've still got something of a checklist for additional features we want to push into 1.0+ but I'll reopen this issue so we can revisit this later.

@MattCarrickPL MattCarrickPL reopened this Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: feature New feature or enhancement
Projects
None yet
Development

No branches or pull requests

2 participants