Skip to content

Issues: stanford-crfm/helm

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

118 Open 994 Closed
118 Open 994 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Urgent help needed on max_tokens
#3465 opened Mar 23, 2025 by rankofootball
Broken links of scenario and metric data
#3435 opened Mar 14, 2025 by gaduan8
2
Proposal: Integrating Vietnamese LLM evaluation
#3431 opened Mar 12, 2025 by martinakaduc
1
Issue with link to instances beyond first page
#3425 opened Mar 12, 2025 by liamjxu
How to submit the test results on the air-bench leaderboard of HELM by our closed-source LLM
#3398 opened Mar 3, 2025 by carlWM
1
ReadTheDocs failing inside mkdocs-autorefs
#3393 opened Mar 1, 2025 by yifanmai
GPU summarization metrics require sentencepiece
#3350 opened Feb 19, 2025 by yifanmai
Anthropic Client receiving unexpected key word
#3324 opened Feb 6, 2025 by ahmeda14960
2
Adaptive evaluation by customized data loader
#3323 opened Feb 6, 2025 by sangttruong
1
Incorporating Deepseek R1
#3294 opened Jan 26, 2025 by bryanzhou008
1
Verification about support for Granite models in the HELM leaderboard
#3269 opened Jan 14, 2025 by RonalddMatias
1
Switch AnthropicTokenizer to use client.beta.messages.count_tokens()
#3212 opened Dec 17, 2024 by yifanmai
4
Issue with Model Execution on the HumanEval Benchmark user question
#3201 opened Dec 6, 2024 by lucas-s-p
3
Add new model NECTEC (nectec/OpenThaiLLM-Prebuilt-7B, nectec/Pathumma-llm-text-1.0.0)
#3188 opened Nov 28, 2024 by JackJessada
3
At least one of model_deployment and model must be specified user question
#3184 opened Nov 25, 2024 by thallysonjsa
5
VHELM setup user question
#3183 opened Nov 24, 2024 by smsarov
1
Reproducing WMT14 user question
#3175 opened Nov 19, 2024 by MaxHahnbueck
2
Installation Guide: Required Dependencies for Supported Operating Systems and Distributions user question
#3160 opened Nov 14, 2024 by shakatoday
5
Is there any way to load GGUF models? user question
#3141 opened Nov 7, 2024 by InAnYan
3
Support few-shot chain-of-thought in GPQA / MMLU
#3088 opened Oct 22, 2024 by yifanmai
Issue with running HEIM documentation Improvements or additions to documentation HEIM (Text2Image) user question
#3080 opened Oct 22, 2024 by sudhir-mcw
5
flake8 in Python 3.12 produces a large number of errors
#3072 opened Oct 18, 2024 by yifanmai
How to use this package when I have the pompts and images across models HEIM (Text2Image) user question
#3062 opened Oct 14, 2024 by snehith3195
1
Not able to install install-heim-extras.sh for heim leaderboard HEIM (Text2Image) user question
#3060 opened Oct 11, 2024 by snehith3195
1
Proposal: Multi-turn support
#3059 opened Oct 10, 2024 by yifanmai
ProTip! Add no:assignee to see everything that’s not assigned.