Installation of extension(s) is not working due to parsing error of extensionCommandTree.json: json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

[Hi-Fi]

Describe the bug

When trying to install extensions, just getting error

The command failed with an unexpected error. Here is the traceback:
Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

This happens both locally (using e.g. docker run -it --rm mcr.microsoft.com/azure-cli) and at Azure DevOps.

Related command

az account subscription list (even without logging in)

Errors

The command failed with an unexpected error. Here is the traceback:
Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

Issue script & Debug output

aba60738e9d4:/# az account subscription list
The command failed with an unexpected error. Here is the traceback:
Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/__init__.py", line 335, in loads
    raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)",
json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 589, in execute
    parsed_args = self.parser.parse_args(args)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/knack/parser.py", line 261, in parse_args
    return super().parse_args(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/argparse.py", line 1869, in parse_args
    args, argv = self.parse_known_args(args, namespace)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/parser.py", line 281, in parse_known_args
    self._namespace, self._raw_arguments = super().parse_known_args(args=args, namespace=namespace)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/argparse.py", line 1902, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/argparse.py", line 2117, in _parse_known_args
    stop_index = consume_positionals(start_index)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/argparse.py", line 2073, in consume_positionals
    take_action(action, args)
  File "/usr/local/lib/python3.11/argparse.py", line 1978, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/local/lib/python3.11/argparse.py", line 1241, in __call__
    subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/parser.py", line 281, in parse_known_args
    self._namespace, self._raw_arguments = super().parse_known_args(args=args, namespace=namespace)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/argparse.py", line 1902, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/argparse.py", line 2117, in _parse_known_args
    stop_index = consume_positionals(start_index)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/argparse.py", line 2073, in consume_positionals
    take_action(action, args)
  File "/usr/local/lib/python3.11/argparse.py", line 1962, in take_action
    argument_values = self._get_values(action, argument_strings)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/parser.py", line 272, in _get_values
    value = super(AzCliCommandParser, self)._get_values(action, arg_strings)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/argparse.py", line 2506, in _get_values
    self._check_value(action, value[0])
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/parser.py", line 300, in _check_value
    use_dynamic_install = try_install_extension(self, args)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/dynamic_install.py", line 119, in try_install_extension
    _check_value_in_extensions(cli_ctx, parser, args, use_dynamic_install == 'yes_without_prompt')
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/dynamic_install.py", line 150, in _check_value_in_extensions
    ext_name = _search_in_extension_commands(cli_ctx, command_str, allow_prefix_match=allow_prefix_match)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/dynamic_install.py", line 87, in _search_in_extension_commands
    cmd_chain = _get_extension_command_tree(cli_ctx)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/dynamic_install.py", line 38, in _get_extension_command_tree
    EXT_CMD_TREE.data = response.json()
                        ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

Expected behavior

Extension installed fine and command that was asked done.

Environment Summary

{
  "azure-cli": "2.57.0",
  "azure-cli-core": "2.57.0",
  "azure-cli-telemetry": "1.1.0",
  "extensions": {}
}

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[FileJunkie]

az aks pod-identity add is also not working.

[mdrakiburrahman]

We're also hitting this, our automation pipelines are all blocked. This is a new regression caused last night in Azure CLI, things were working fine last night for the same pipeline:

[felazuris]

az devops configure is failing for us

[MarcDenman]

We have run into this as well running az network front-door list in some of our pipelines.

[pbahnl]

Same here for deployment pipelines using az-cli. Everything fails.

[syneex]

az devops configure is also failing for us and we are blocked because of that.

[josh-yates]

It seems to be happening when az doesn't recognise a command and is looking for an extension, as it happens for eg. az abc123. Manually installing extensions using az extension add --name {extension} appears to be a workaround.

[yonzhan]

We are looking into this and will reply here after we figure out.

[balcsida]

We've experienced this issue on our side and running az account help --debug indicates that it indeed fails on https://azurecliextensionsync.blob.core.windows.net/cmd-index/extensionCommandTree.json

extensionCommandTree.json: Unicode text, UTF-8 (with BOM) text, with very long lines (65533), with no line terminators

However, I suspect the CLI code is not prepared for decoding JSON files with BOM.

[Davide-DD]

I'm also hitting this in Github Action, using the following job:

    uses: azure/CLI@v1
    with:
      azcliversion: 2.56.0
      inlineScript: |
        export AZURE_DEVOPS_EXT_PAT="${{ secrets.AZURE_ARTIFACTS_PAT }}"
        az config set extension.use_dynamic_install=yes_without_prompt
        az artifacts universal download --organization "$ORGANIZATION" --project "7ac49839-f0e7-4bcd-a5c3-6b465a2217e9" --scope project --feed "Common" --name "$NAME" --version "${{env.REQUESTED_FRONTEND_VERSION}}" --path $GITHUB_WORKSPACE/wwwroot

The same job was working fine until one hour before, then it just broke. The key "runs-on" is set to "ubuntu-latest".

[robinsmidsrod]

I use the SSH extension in a pipeline, and explicitly doing az extension add --name ssh avoided the above-mentioned issue.

[RenanCarlosPereira]

also happening here:
az monitor app-insights component show --app app-name -g your-resource-group

The command failed with an unexpected error. Here is the traceback:
Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)
Traceback (most recent call last):
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\requests/models.py", line 971, in json
File "json_init_.py", line 335, in loads
json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

[akku16]

We have also started facing the same issue since today morning for all az commands.
The version is azure-cli : 2.50.0

[malsey]

It seems to be happening when az doesn't recognise a command and is looking for an extension, as it happens for eg. az abc123. Manually installing extensions using az extension add --name {extension} appears to be a workaround.

This workaround works fine

[bondido]

I'm also hitting this in Github Action, using the following job:

    uses: azure/CLI@v1
    with:
      azcliversion: 2.56.0
      inlineScript: |
        export AZURE_DEVOPS_EXT_PAT="${{ secrets.AZURE_ARTIFACTS_PAT }}"
        az config set extension.use_dynamic_install=yes_without_prompt
        az artifacts universal download --organization "$ORGANIZATION" --project "7ac49839-f0e7-4bcd-a5c3-6b465a2217e9" --scope project --feed "Common" --name "$NAME" --version "${{env.REQUESTED_FRONTEND_VERSION}}" --path $GITHUB_WORKSPACE/wwwroot

The same job was working fine until one hour before, then it just broke. The key "runs-on" is set to "ubuntu-latest".

In our case, we've also get rid of
az config set extension.use_dynamic_install=yes_without_prompt
as it could try (and fail) to install some extensions that we're not even aware of.

We've changed it to:
az config set extension.use_dynamic_install=no

and installed all required extensions with
az extension add --name ...

[TNEL]

i have some issue running

echo '3....' | az devops login --org https://dev.azure.com/org

ERROR: The command failed with an unexpected error. Here is the traceback: ERROR: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0) Traceback (most recent call last): File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 588, in execute File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/parser.py", line 261, in parse_args File "argparse.py", line 1826, in parse_args File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/parser.py", line 281, in parse_known_args File "argparse.py", line 1859, in parse_known_args File "argparse.py", line 2054, in _parse_known_args File "argparse.py", line 2031, in consume_positionals File "argparse.py", line 1920, in take_action File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/parser.py", line 272, in _get_values File "argparse.py", line 2465, in _get_values File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/parser.py", line 300, in _check_value File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/dynamic_install.py", line 119, in try_install_extension File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/dynamic_install.py", line 143, in _check_value_in_extensions File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/dynamic_install.py", line 87, in _search_in_extension_commands File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/dynamic_install.py", line 38, in _get_extension_command_tree File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\requests/models.py", line 910, in json File "json\__init__.py", line 335, in loads json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0) To check existing issues, please visit: https://github.com/Azure/azure-cli/issues To open a new issue, please run az feedback``

[MichalJes]

Same problem while running az monitor app-insights component create

[jiasli]

ℹ️ Official Announcement

We have identified the root cause to be the incorrect metadata of the Command Tree file:

azure-cli/src/azure-cli-core/azure/cli/core/extension/dynamic_install.py

Line 29 in bad3331

azmirror_endpoint else 'https://aka.ms/azExtCmdTree'

We have already fixed it and dynamic installation should now work as expected. Please retry the commands.

Thanks for reporting this issue and apologize for the inconvenience caused.

[odegroot]

Confirmed, the error is now gone for me, without changing anything on my side.

[Davide-DD]

Everything seems okay now, just ran the same pipeline and it is executing without any problem

[grixxie]

Confirmed this has solved my issues without changing anything on my side, thank you!

[MikeLanglois]

Issue has been resolved thanks!

[jsntcy]

Close this as it has been resolved.