Usage start error using az consumption usage list on Azure CLI version 2.57.0 #28323

raim4033 · 2024-02-07T13:56:59Z

Describe the bug

az consumption usage list --start-date 2021-10-01 --end-date 2021-11-01
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
The command failed with an unexpected error. Here is the traceback:
'usageStart'
Traceback (most recent call last):
File "/home/raimond/.local/lib/python3.8/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/core/commands/init.py", line 664, in execute
raise ex
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/core/commands/init.py", line 729, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/core/commands/init.py", line 711, in _run_job
result = list(result)
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/core/aaz/_paging.py", line 87, in next
return next(self._page_iterator)
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/core/aaz/_paging.py", line 46, in next
curr_page, self._next_link = self._extract_result()
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/command_modules/consumption/custom.py", line 85, in _output
result = list([transform_usage_output(item) for item in result])
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/command_modules/consumption/custom.py", line 85, in
result = list([transform_usage_output(item) for item in result])
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/command_modules/consumption/custom.py", line 95, in transform_usage_output
usageStart = parser.parse(result['usageStart'])
KeyError: 'usageStart'
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues

Related command

az consumption usage list

Errors

The command failed with an unexpected error. Here is the traceback:
'usageStart'
Traceback (most recent call last):
File "/home/raimond/.local/lib/python3.8/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/core/commands/init.py", line 664, in execute
raise ex
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/core/commands/init.py", line 729, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/core/commands/init.py", line 711, in _run_job
result = list(result)
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/core/aaz/_paging.py", line 87, in next
return next(self._page_iterator)
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/core/aaz/_paging.py", line 46, in next
curr_page, self._next_link = self._extract_result()
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/command_modules/consumption/custom.py", line 85, in _output
result = list([transform_usage_output(item) for item in result])
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/command_modules/consumption/custom.py", line 85, in
result = list([transform_usage_output(item) for item in result])
File "/home/raimond/.local/lib/python3.8/site-packages/azure/cli/command_modules/consumption/custom.py", line 95, in transform_usage_output
usageStart = parser.parse(result['usageStart'])
KeyError: 'usageStart'
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues

Issue script & Debug output

debug1.txt

Expected behavior

Result from az consumption list

Environment Summary

azure-cli 2.57.0
core 2.57.0
telemetry 1.1.0
Extensions:
account 0.2.5
aks-preview 1.0.0b5
application-insights 1.2.0
costmanagement 0.3.0
rdbms-connect 1.0.4
resource-graph 2.1.0
storage-preview 1.0.0b1
virtual-wan 0.3.0

Dependencies:
msal 1.26.0
azure-mgmt-resource 23.1.0b2

Python location '/usr/bin/python3'
Extensions directory '/home/raimond/.azure/cliextensions'

Python (Linux) 3.8.10 (default, Nov 22 2023, 10:22:35)
[GCC 9.4.0]

Your CLI is up-to-date

Additional context

No response

yonzhan · 2024-02-07T13:57:12Z

Thank you for opening this issue, we will look into it.

triatic · 2024-02-12T19:22:22Z

Same issue here.

PS C:\> az consumption usage list --start-date 2024-01-01 --end-date 2024-02-01
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
The command failed with an unexpected error. Here is the traceback:
'usageStart'
Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 664, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 729, in _run_jobs_serially
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 711, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/aaz/_paging.py", line 87, in __next__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/aaz/_paging.py", line 46, in __next__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/consumption/custom.py", line 85, in _output
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/consumption/custom.py", line 85, in <listcomp>
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/consumption/custom.py", line 95, in transform_usage_output
KeyError: 'usageStart'
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues

orangesharing · 2024-02-13T07:32:29Z

hi Everyone - I can confirm the same issue.

I am currently running some tests using Azure DevOps Pipelines, and have prepared a pipeline that runs two jobs in parallel, both run the following command against two different subscriptions (service connections) under the same tenant:

        az consumption usage list
        --billing-period-name $(lastMonth)
        --include-additional-properties
        --include-meter-details
        --query "[].[instanceId, consumedService, instanceName, instanceLocation, product, pretaxCost, usageStart, usageEnd]"
        --output tsv

I ran this command 6 times within the span of about 40 minutes:

5 of those runs ended up with one of the job failing and the other one working fine (returning results)
1 run finished with both jobs completing OK
strangely enough, while I reliably get an error indicated above, I have not seen both jobs failing yet - only one of those seems to fail at any given run
this command never returns any errors when I run it from my local (always successful)

Hope that sheds a bit of light and helps your troubleshooting efforts.

aglazer7 · 2024-02-16T09:31:15Z

urllib3.connectionpool: https://management.azure.com:443 "GET /subscriptions/895cda87-06a5-4a91-b04c-d023ccbc6525/providers/Microsoft.Consumption/usageDetails?$filter=properties%2FusageEnd%20ge%20%272024-01-08T00%3A00%3A00Z%27%20and%20properties%2FusageEnd%20le%20%272024-01-09T00%3A00%3A00Z%27&$top=1&api-version=2023-05-01 HTTP/1.1" 200 2976 cli.azure.cli.core.sdk.policies: Response status: 200

Looks like it gives the response, but a closer look at the url shows that there's 2x usageEnd. First one should be of course usageStart.

RahulSrivastav14 · 2024-03-15T06:38:39Z

I've been facing this issue with my workflow that checks for the az consumption usage list :

ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: 'usageStart'
Traceback (most recent call last):
  File "/opt/az/lib/python3.[11](https://github.com/turbotio/ops/actions/runs/8277743509/job/22648667583#step:6:12)/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 664, in execute
    raise ex
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 731, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 7[14](https://github.com/turbotio/ops/actions/runs/8277743509/job/22648667583#step:6:15), in _run_job
    result = list(result)
             ^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/aaz/_paging.py", line 87, in __next__
    return next(self._page_iterator)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/core/aaz/_paging.py", line 46, in __next__
    curr_page, self._next_link = self._extract_result()
                                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/consumption/custom.py", line 85, in _output
    result = list([transform_usage_output(item) for item in result])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/consumption/custom.py", line 85, in <listcomp>
    result = list([transform_usage_output(item) for item in result])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.11/site-packages/azure/cli/command_modules/consumption/custom.py", line 95, in transform_usage_output
    usageStart = parser.parse(result['usageStart'])
                              ~~~~~~^^^^^^^^^^^^^^

rabe1 · 2024-03-25T10:42:46Z

I can confirm that the error still exists also in latest Azure CLI version 2.58.0.
Are there meanwhile any updates on this or a workaround to make this work again?
It's really bad not to be able to retrieve the detailled cost consumption per ressource via this simple CLI query since meanwhile more than a month.

agnaldocarmo · 2024-03-26T19:48:00Z

I'm facing the same issue using Azure CLI version 2.58.0

Hymian7 · 2024-03-27T17:13:43Z

I am also facing the issue in
az version { "azure-cli": "2.58.0", "azure-cli-core": "2.58.0", "azure-cli-telemetry": "1.1.0", "extensions": {} }

tenjaa · 2024-04-02T10:24:47Z

As a workaround you can try an older version

docker run -it mcr.microsoft.com/azure-cli:2.56.0

tworthington · 2024-05-22T09:14:30Z

Still there in 2.61.0

FlorianDp · 2024-07-24T14:20:00Z

Experiencing it on 2.62.0 as well.
az version
{
"azure-cli": "2.62.0",
"azure-cli-core": "2.62.0",
"azure-cli-telemetry": "1.1.0",
"extensions": {
"ai-examples": "0.2.5",
"costmanagement": "0.3.0",
"ml": "2.28.0",
"ssh": "2.0.4"
}
}

ghost · 2024-07-29T20:15:07Z

Same here on...

az version
{
"azure-cli": "2.62.0",
"azure-cli-core": "2.62.0",
"azure-cli-telemetry": "1.1.0",
"extensions": {
"ai-examples": "0.2.5",
"ml": "2.28.0",
"ssh": "2.0.4"
}
}

BenjamenJoubert · 2024-08-01T13:23:51Z

I might have found the issue on the following line:

azure-cli/src/azure-cli/azure/cli/command_modules/consumption/custom.py

Line 78 in 2956483

    
           filter_from = "properties/usageEnd ge \'{}\'".format(start_date.strftime("%Y-%m-%dT%H:%M:%SZ"))

The above line reads as follows:

filter_from = "properties/usageEnd ge \'{}\'".format(start_date.strftime("%Y-%m-%dT%H:%M:%SZ"))

Note that in the above line the filter from is using usageEnd instead of usageStart.

The fixed line should read as follows:

filter_from = "properties/usageStart ge \'{}\'".format(start_date.strftime("%Y-%m-%dT%H:%M:%SZ"))

Is there someone around who can test the above code changes?

petdance · 2024-08-07T14:54:33Z

While the usageEnd/usageStart noted above seems to indeed be a problem, I patched it manually on my install (now 2.63.0) and this error still happens.

Down in the transform_error_output where the error happens, I added a dump of results and it doesn't have the usageStart and usageEnd the code is looking for, or any of the other keys the function wants. The output looks like this:

{
  "id": "...",
  "name": "...",
  "accountName": "...",
  "consumedService": "...",
  "costCenter": "...",
  "meterId": "...",
  "product": "...",
  "subscriptionName": "...",
  "tags": {
    "..."
  },
  "type": "..."
}

blueboxes · 2024-10-20T19:06:57Z

Still there in 2.65.0

teemukj · 2024-12-18T09:03:34Z

Issue still persists in current latest CLI version 2.67.0.

I wish this would be fixed soon, as the cost management UI is quite painful to use at the moment, even if you just want to gather a simple total cost of multiple subscriptions.

madhavi-sudina · 2024-12-18T09:46:13Z

I am also facing the issue with the below cli version. Please try to fix it asap. It's quite painful to get the data manually from the portal for all the resources in each subscription for a tenant and I have multiple tenants to retrieve this data for.
{
"azure-cli": "2.65.0",
"azure-cli-core": "2.65.0",
"azure-cli-telemetry": "1.1.0",
"extensions": {
"ai-examples": "0.2.5",
"ml": "2.30.1",
"ssh": "2.0.5"
}
}

daschaa · 2025-01-06T12:06:57Z

The issue still persists

philippdrebes · 2025-01-14T09:48:34Z

Error still happens in 2.68.0

➜ az version
{
  "azure-cli": "2.68.0",
  "azure-cli-core": "2.68.0",
  "azure-cli-telemetry": "1.1.0",
  "extensions": {
    "application-insights": "1.2.2",
    "bastion": "1.3.1",
    "ssh": "2.0.6",
    "subscription": "1.0.0b2"
  }
}

tazd007 · 2025-01-28T11:52:49Z

Issue still present

PS C:\Git> az version
{
"azure-cli": "2.68.0",
"azure-cli-core": "2.68.0",
"azure-cli-telemetry": "1.1.0",
"extensions": {
"azure-devops": "0.25.0",
"managementpartner": "0.1.3",
"resource-graph": "2.1.0"
}
}

PS C:\Git> az consumption usage list --end-date 2025-01-02 --start-date 2019-01-01
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
The command failed with an unexpected error. Here is the traceback:
'usageStart'
Traceback (most recent call last):
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 666, in execute
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 734, in _run_jobs_serially
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 716, in _run_job
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/aaz/_paging.py", line 87, in next
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/aaz/_paging.py", line 46, in next
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/consumption/custom.py", line 85, in _output
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/consumption/custom.py", line 95, in transform_usage_output
KeyError: 'usageStart'
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues

raim4033 added the bug This issue requires a change to an existing behavior in the product in order to be resolved. label Feb 7, 2024

microsoft-github-policy-service bot added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label Feb 7, 2024

yonzhan added consumption Service Attention This issue is responsible by Azure service team. labels Feb 7, 2024

nick-crain mentioned this issue Feb 5, 2025

error az consumption usage list #30749

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Usage start error using az consumption usage list on Azure CLI version 2.57.0 #28323

Usage start error using az consumption usage list on Azure CLI version 2.57.0 #28323

raim4033 commented Feb 7, 2024

yonzhan commented Feb 7, 2024

triatic commented Feb 12, 2024

orangesharing commented Feb 13, 2024

aglazer7 commented Feb 16, 2024

RahulSrivastav14 commented Mar 15, 2024 •

edited

Loading

rabe1 commented Mar 25, 2024

agnaldocarmo commented Mar 26, 2024 •

edited

Loading

Hymian7 commented Mar 27, 2024

tenjaa commented Apr 2, 2024

tworthington commented May 22, 2024

FlorianDp commented Jul 24, 2024

ghost commented Jul 29, 2024 •

edited by ghost

Loading

BenjamenJoubert commented Aug 1, 2024

petdance commented Aug 7, 2024

blueboxes commented Oct 20, 2024

teemukj commented Dec 18, 2024

madhavi-sudina commented Dec 18, 2024

daschaa commented Jan 6, 2025

philippdrebes commented Jan 14, 2025

tazd007 commented Jan 28, 2025

Usage start error using az consumption usage list on Azure CLI version 2.57.0 #28323

Usage start error using az consumption usage list on Azure CLI version 2.57.0 #28323

Comments

raim4033 commented Feb 7, 2024

Describe the bug

Related command

Errors

Issue script & Debug output

Expected behavior

Environment Summary

Additional context

yonzhan commented Feb 7, 2024

triatic commented Feb 12, 2024

orangesharing commented Feb 13, 2024

aglazer7 commented Feb 16, 2024

RahulSrivastav14 commented Mar 15, 2024 • edited Loading

rabe1 commented Mar 25, 2024

agnaldocarmo commented Mar 26, 2024 • edited Loading

Hymian7 commented Mar 27, 2024

tenjaa commented Apr 2, 2024

tworthington commented May 22, 2024

FlorianDp commented Jul 24, 2024

ghost commented Jul 29, 2024 • edited by ghost Loading

BenjamenJoubert commented Aug 1, 2024

petdance commented Aug 7, 2024

blueboxes commented Oct 20, 2024

teemukj commented Dec 18, 2024

madhavi-sudina commented Dec 18, 2024

daschaa commented Jan 6, 2025

philippdrebes commented Jan 14, 2025

tazd007 commented Jan 28, 2025

RahulSrivastav14 commented Mar 15, 2024 •

edited

Loading

agnaldocarmo commented Mar 26, 2024 •

edited

Loading

ghost commented Jul 29, 2024 •

edited by ghost

Loading