Running Bicep from Docker image

[delarooster]

Bicep version
0.25.53

Describe the bug
I am attempting to run Bicep from a Node app that instantiates a cli instance. When the app runs from the source code on my machine (Apple M1 Pro) I have no issues, however, once in a container the same processes throw any number of errors. Most recently I'm getting:

stderr: ERROR: Unhandled exception. System.ArgumentException: Unable to parse 'Microsoft.Addons/supportProviders/supportPlanTypes@2017-05-15' (Parameter 'resourceType')

prior was:

2024-02-20 16:23:28 Running command: az deployment sub create --name node-cli-deploy --subscription <sub_id> --template-file submodules/main.bicep --location eastus --verbose
2024-02-20 16:23:32 stderr: WARNING: The configuration value of bicep.use_binary_from_path has been set to 'false'.
2024-02-20 16:23:32 
2024-02-20 16:23:32 stderr: INFO: Successfully installed Bicep CLI to /root/.azure/bin/bicep
2024-02-20 16:23:32 
2024-02-20 16:23:32 stderr: ERROR: rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2
2024-02-20 16:23:32  
2024-02-20 16:23:32 
2024-02-20 16:23:32 stderr: INFO: Command ran in 4.025 seconds (init: 0.096, invoke: 3.929)
2024-02-20 16:23:32 
2024-02-20 16:23:32 stderr: INFO: Suppress exception:
2024-02-20 16:23:32 Traceback (most recent call last):
2024-02-20 16:23:32   File "/opt/az/lib/python3.11/site-packages/azure/cli/__main__.py", line 62, in <module>
2024-02-20 16:23:32     raise ex
2024-02-20 16:23:32   File "/opt/az/lib/python3.11/site-packages/azure/cli/__main__.py", line 55, in <module>
2024-02-20 16:23:32     sys.exit(exit_code)
2024-02-20 16:23:32 SystemExit: 1
2024-02-20 16:23:32 
2024-02-20 16:23:32 During handling of the above exception, another exception occurred:
2024-02-20 16:23:32 
2024-02-20 16:23:32 Traceback (most recent call last):
2024-02-20 16:23:32   File "/opt/az/lib/python3.11/site-packages/azure/cli/core/decorators.py", line 79, in _wrapped_func
2024-02-20 16:23:32     return func(*args, **kwargs)
2024-02-20 16:23:32            ^^^^^^^^^^^^^^^^^^^^^
2024-02-20 16:23:32   File "/opt/az/lib/python3.11/site-packages/azure/cli/core/telemetry.py", line 532, in _get_secrets_warning_config
2024-02-20 16:23:32     show_secrets_warning = _get_config().getboolean('clients', 'show_secrets_warning', fallback=None)
2024-02-20 16:23:32                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-20 16:23:32   File "/opt/az/lib/python3.11/site-packages/knack/config.py", line 147, in getboolean
2024-02-20 16:23:32     raise ValueError('Not a boolean: {}'.format(val))
2024-02-20 16:23:32 ValueError: Not a boolean: None

To Reproduce
cli command executed:

az deployment sub create --name node-cli-deploy --subscription <sub_id> --template-file submodules/main.bicep --location eastus --verbose

bicep file:

output helloWorld string = 'Hello, World!'

Docker file:

# configuration in first error above, second error is when set platform target as arm64
FROM --platform=linux/amd64 node:21 AS base

# Installing Python due to previous errors
RUN apt-get update && apt-get install -y python3

# Install Azure CLI
RUN curl -sL https://aka.ms/InstallAzureCLIDeb | bash

WORKDIR /home/node/app

COPY ./src/server/package*.json ./
RUN npm install
COPY ./src/server/ ./src/server/
COPY ./submodules ./submodules

# Debug: List contents of a specific submodule directory to verify presence
RUN ls -la ./submodules/*

EXPOSE 3000
CMD ["node", "./src/server/main.js"]

Additional context
The error above is not the only error I've dealt with, many previously surrounding the well-known issue with arm64 architecture. Been hacking at this for a few days now and curious if there are others who have had this same issue or perhaps I'm working uphill combining Docker and Bicep with known limitations. I've also built this image on specifically amd64 chipsets and run into similar problems.

[delarooster]

All right I've done more digging in the past few days, this looks like it may be a specific CLI issue as when I run the command from my local I'm having the exact same problem output, which the local machine handles gracefully but forces the container runtime to exit.

az deployment sub create --name local-test-123 --subscription <subscriptionId> -f submodules/main.bicep --location eastus --verbose

{
  "id": "/subscriptions/<subscriptionId>/providers/Microsoft.Resources/deployments/local-test-123",
  "location": "eastus",
  "name": "local-test-123",
  "properties": {
    "correlationId": "516ac7d5-507b-4244-a575-a649c9abc4c0",
    "debugSetting": null,
    "dependencies": [],
    "duration": "PT0.6339818S",
    "error": null,
    "mode": "Incremental",
    "onErrorDeployment": null,
    "outputResources": [],
    "outputs": {
      "helloWorld": {
        "type": "String",
        "value": "Hello, World!"
      }
    },
    "parameters": {
    },
    "parametersLink": null,
    "providers": [],
    "provisioningState": "Succeeded",
    "templateHash": "14729165693114952971",
    "templateLink": null,
    "timestamp": "2024-02-23T14:13:36.826246+00:00",
    "validatedResources": null
  },
  "tags": null,
  "type": "Microsoft.Resources/deployments"
}
Command ran in 35.884 seconds (init: 0.061, invoke: 35.823)
Suppress exception:
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/azure-cli/2.57.0/libexec/lib/python3.11/site-packages/azure/cli/__main__.py", line 62, in <module>
    raise ex
  File "/opt/homebrew/Cellar/azure-cli/2.57.0/libexec/lib/python3.11/site-packages/azure/cli/__main__.py", line 55, in <module>
    sys.exit(exit_code)
SystemExit: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/azure-cli/2.57.0/libexec/lib/python3.11/site-packages/azure/cli/core/decorators.py", line 79, in _wrapped_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/azure-cli/2.57.0/libexec/lib/python3.11/site-packages/azure/cli/core/telemetry.py", line 532, in _get_secrets_warning_config
    show_secrets_warning = _get_config().getboolean('clients', 'show_secrets_warning', fallback=None)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/azure-cli/2.57.0/libexec/lib/python3.11/site-packages/knack/config.py", line 147, in getboolean
    raise ValueError('Not a boolean: {}'.format(val))
ValueError: Not a boolean: None

In the meantime if anyone has any thoughts or recommendations I'd be greatly appreciative!

[delarooster]

Closing this issue and opened new issue on Azure Cli repository. Azure/azure-cli#28448

[Jasstkn]

Hey. I am also facing this issue with the exact same error:

ERROR: Unhandled exception. System.ArgumentException: Unable to parse 'Microsoft.Addons/supportProviders/supportPlanTypes@2017-05-15' (Parameter 'resourceType')
   at Bicep.Core.Resources.ResourceTypeReference.Parse(String resourceType)
   at Bicep.Core.TypeSystem.Providers.Az.AzResourceTypeLoader.<>c.<.ctor>b__4_0(KeyValuePair`2 kvp)
   at System.Collections.Immutable.ImmutableDictionary.<>c__DisplayClass9_0`3.<ToImmutableDictionary>b__0(TSource element)
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1, MutationInput, KeyCollisionBehavior )
   at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1, Boolean)
   at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 )
   at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable`1, Func`2, Func`2, IEqualityComparer`1 , IEqualityComparer`1 )
   at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable`1, Func`2, Func`2)
   at Bicep.Core.TypeSystem.Providers.Az.AzResourceTypeLoader..ctor(ITypeLoader typeLoader)
   at Bicep.Core.TypeSystem.Providers.ResourceTypeProviderFactory.<>c.<.cctor>b__7_0()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper, Boolean)
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at Bicep.Core.TypeSystem.Providers.ResourceTypeProviderFactory.GetBuiltInAzResourceTypesProvider()
   at Bicep.Core.Semantics.Namespaces.DefaultNamespaceProvider.<>c__DisplayClass3_0.<.ctor>b__0(ResourceTypesProviderDescriptor descriptor, ResourceScope resourceScope, IFeatureProvider _, BicepSourceFileKind sourceFileKind)
   at Bicep.Core.Semantics.Namespaces.DefaultNamespaceProvider.TryGetNamespace(ResourceTypesProviderDescriptor descriptor, ResourceScope resourceScope, IFeatureProvider features, BicepSourceFileKind sourceFileKind)
   at Bicep.Core.Semantics.Namespaces.NamespaceResolver.<Create>g__TryAddBuiltInNamespace|2_3(String namespace, String version, <>c__DisplayClass2_0&)
   at Bicep.Core.Semantics.Namespaces.NamespaceResolver.Create(IFeatureProvider features, INamespaceProvider namespaceProvider, BicepSourceFile sourceFile, ResourceScope targetScope, ILanguageScope fileScope)
   at Bicep.Core.Semantics.Binder..ctor(INamespaceProvider namespaceProvider, IFeatureProvider features, IArtifactFileLookup sourceFileLookup, ISemanticModelLookup modelLookup, BicepSourceFile sourceFile, ISymbolContext symbolContext, IArtifactReferenceFactory artifactReferenceFactory)
   at Bicep.Core.Semantics.SemanticModel..ctor(Compilation compilation, BicepSourceFile sourceFile)
   at Bicep.Core.Semantics.Compilation.CreateSemanticModel(BicepSourceFile bicepFile)
   at Bicep.Core.Semantics.Compilation.<>c__DisplayClass1_1.<.ctor>b__2()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper, Boolean)
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at Bicep.Core.Semantics.Compilation.GetSemanticModel(ISourceFile sourceFile)
   at Bicep.Core.Semantics.Compilation.GetSemanticModel[T](ISourceFile sourceFile)
   at Bicep.Core.Semantics.Compilation.GetSemanticModel(BicepSourceFile bicepFile)
   at Bicep.Core.Semantics.Compilation.<GetAllDiagnosticsByBicepFile>b__31_1(BicepSourceFile bicepFile)
   at System.Collections.Immutable.ImmutableDictionary.<>c__DisplayClass9_0`3.<ToImmutableDictionary>b__0(TSource element)
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1, MutationInput, KeyCollisionBehavior )
   at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1, Boolean)
   at System.Collections.Immutable.ImmutableDictionary`2.AddRange(IEnumerable`1 )
   at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable`1, Func`2, Func`2, IEqualityComparer`1 , IEqualityComparer`1 )
   at System.Collections.Immutable.ImmutableDictionary.ToImmutableDictionary[TSource,TKey,TValue](IEnumerable`1, Func`2, Func`2)
   at Bicep.Core.Semantics.Compilation.GetAllDiagnosticsByBicepFile()
   at Bicep.Cli.Logging.DiagnosticLogger.LogDiagnostics(DiagnosticOptions options, Compilation compilation)
   at Bicep.Cli.Commands.BuildCommand.RunAsync(BuildArguments args)
   at Bicep.Cli.Program.RunAsync(String[] args, CancellationToken cancellationToken)
   at Bicep.Cli.Program.<>c__DisplayClass3_0.<<Main>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Bicep.Cli.Program.RunWithCancellationAsync(Func`2 runFunc)
   at Bicep.Cli.Program.Main(String[] args)
   at Bicep.Cli.Program.<Main>(String[] args)

[delarooster]

Hello @Jasstkn and thanks for the reminder, meant to post an update on how I ended up solving this riddle.
Here's my final Dockerfile:

FROM amd64/ubuntu:24.04 as base

## Required for disabling performance optimizations for dotnet on arm64 chipsets incompatible with QEMU/Rosetta
## see issue here: https://github.com/dotnet/sdk/issues/37169
ENV DOTNET_EnableWriteXorExecute=0

RUN apt-get update && apt-get install -y \
    ca-certificates \
    curl \
    gnupg \
    lsb-release \
    software-properties-common \
    python3-pip \
    python3-setuptools \
    && rm -rf /var/lib/apt/lists/*

RUN curl -sL https://aka.ms/InstallAzureCLIDeb | bash
RUN curl -sL https://deb.nodesource.com/setup_lts.x
RUN apt-get install -y nodejs npm

# Verify installation
RUN az --version
RUN node --version 
RUN npm --version

# Install Bicep and verify installation
RUN az bicep install
RUN az bicep version

WORKDIR /src/server
COPY ./src/server ./
COPY ./src/server/package*.json ./

WORKDIR /
COPY ./submodules ./submodules/
COPY ./package*.json ./

RUN npm install

EXPOSE 3000

CMD ["node", "src/server/server.js"]

A couple of points:

1. The first (and only) ENV addition is from a dotnet performance optimization that is not compatible with QEMU/Rosetta. That resolved issues of the underlying Bicep dotnet kicking during execution in the container.
2. Prior to the execution of the bicep commands I run this az cli configuration to help suppress certain warnings: az config set clients.show_secrets_warning=False. I found this suggestion after opening up my issue in the Azure Cli repository: Exception raised when executing any Azure CLI command after upgrading to version 2.57. azure-cli#28312 (comment)
3. While not optimal I did end up having to do some custom error handling in the container due to the experimental Bicep features I was managing. Perhaps there is a better way but this was the most straightforward execution I could sort out.

if (
    stderrData.includes("experimental Bicep features have been enabled")
  ) {
    console.log(
      "Known error encountered, but suppressing it as per instructions.",
      "experimental Bicep features have been enabled"
    );
    // Still resolving to not exit the process, but indicating an issue was encountered
    resolve({
      success: true,
      code,
      stdout: stdoutData,
      stderr: stderrData,
      suppressedError: true,
    });
  } 

Hopefully some of this helps you in your resolution!

Best,
Austin