Anthropic Client receiving unexpected key word

[ahmeda14960]

Right now, when using helm run, installing helm from main and attempting to use Anthropic models with anthropic==0.38.0 the following error occurs

export RUN_ENTRIES_CONF_PATH=src/helm/benchmark/presentation/run_entries_safety.conf
export SCHEMA_PATH=src/helm/benchmark/static/schema_safety.yaml
export NUM_TRAIN_TRIALS=1
export MAX_EVAL_INSTANCES=100
export PRIORITY=2

export SUITE_NAME=my_suite
export MODELS_TO_RUN=anthropic/claude-3-5-sonnet-20240620


helm-run --conf-paths $RUN_ENTRIES_CONF_PATH --num-train-trials $NUM_TRAIN_TRIALS --max-eval-instances $MAX_EVAL_INSTANCES --priority $PRIORITY --suite $SUITE_NAME --models-to-run $MODELS_TO_RUN


...

 Error when running simple_safety_tests:model=anthropic_claude-3-5-sonnet-20240620:
Traceback (most recent call last):
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/runner.py", line 216, in run_all
    self.run_one(run_spec)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/runner.py", line 283, in run_one
    request_states: List[RequestState] = adapter.adapt(instances, self.executor.execution_spec.parallelism)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/common/hierarchical_logger.py", line 104, in wrapper
    return fn(*args, **kwargs)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/adaptation/adapters/in_context_learning_adapter.py", line 63, in adapt
    self._adapt_trial_index(all_train_instances, train_trial_index, eval_instances, parallelism)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/adaptation/adapters/in_context_learning_adapter.py", line 86, in _adapt_trial_index
    results: List[List[RequestState]] = parallel_map(
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/common/general.py", line 235, in parallel_map
    results = list(tqdm(executor.map(process, items), total=len(items), disable=None))
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/tqdm/std.py", line 1181, in __iter__
    for obj in iterable:
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/concurrent/futures/_base.py", line 609, in result_iterator
    yield fs.pop().result()
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/adaptation/adapters/in_context_learning_adapter.py", line 83, in generate_requests_for_training_trial
    return self.generate_requests(eval_instance, train_trial_index, training_instances)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/adaptation/adapters/generation_adapter.py", line 37, in generate_requests
    prompt: Prompt = self.construct_prompt(
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/adaptation/adapters/in_context_learning_adapter.py", line 242, in construct_prompt
    prompt = self._make_prompt_fit(prompt)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/adaptation/adapters/in_context_learning_adapter.py", line 323, in _make_prompt_fit
    truncated_text = self.window_service.truncate_from_right(text, self.adapter_spec.max_tokens)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/window_services/local_window_service.py", line 108, in truncate_from_right
    result: str = self.decode(self.encode(text, truncation=True, max_length=max_length).tokens)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/window_services/local_window_service.py", line 45, in encode
    response: TokenizationRequestResult = self.service.tokenize(
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/benchmark/window_services/tokenizer_service.py", line 22, in tokenize
    return self._service.tokenize(self._auth, request)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/proxy/services/server_service.py", line 142, in tokenize
    return self.tokenizer.tokenize(request)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/tokenizers/auto_tokenizer.py", line 68, in tokenize
    tokenizer: Tokenizer = self._get_tokenizer(request.tokenizer)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/tokenizers/auto_tokenizer.py", line 51, in _get_tokenizer
    tokenizer = create_object(tokenizer_spec)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/common/object_spec.py", line 86, in create_object
    return cls(**args)
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/helm/tokenizers/anthropic_tokenizer.py", line 26, in __init__
    tokenizer_object=anthropic.Anthropic().get_tokenizer()
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/anthropic/_client.py", line 122, in __init__
    super().__init__(
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/anthropic/_base_client.py", line 856, in __init__
    self._client = http_client or SyncHttpxClientWrapper(
  File "/Users/ahmed/miniconda3/envs/crfm-helm/lib/python3.9/site-packages/anthropic/_base_client.py", line 754, in __init__
    super().__init__(**kwargs)
TypeError: __init__() got an unexpected keyword argument 'proxies'

[whitepaper82]

I have the same issue. Have you found a solution to this issue?

[yifanmai]

You need a version of httpx before the proxies argument was deleted. According to the httpx CHANGELOG version 0.27.2 and before should work. Could you try running:

pip install httpx==0.27.2