Fail to install databricks extension, cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1.

[jasonwjli]

Describe the bug

I encounter an error when installing az databricks in Windows machine,
I run az extension add --name databricks --debug, and get the error,

I checked I got the environment variable and folder in place
(base) PS C:> python
Python 3.12.2 | packaged by conda-forge | (main, Feb 16 2024, 20:42:31) [MSC v.1937 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import os
print(os.environ.get('CSIDL_COMMON_APPDATA'))
C:\ProgramData
print(os.path.exists('C:\ProgramData'))
True

and the entry is in registry
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders\Common AppData, value = C:\ProgramData

Related command

az extension add --name databricks --debug

Errors

cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.

Issue script & Debug output

cli.knack.cli: Command arguments: ['extension', 'add', '--name', 'databricks', '--debug']
cli.knack.cli: init debug log:
Enable color in terminal.
Enable VT mode.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x000001DC0E59F2E0>, <function OutputProducer.on_global_arguments at 0x000001DC0E93C040>, <function CLIQuery.on_global_arguments at 0x000001DC0E965440>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'extension': ['azure.cli.command_modules.extension']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: extension 0.010 1 7
cli.azure.cli.core: Total (1) 0.010 1 7
cli.azure.cli.core: Loaded 1 groups, 7 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command : extension add
cli.azure.cli.core: Command table: extension add
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x000001DC10BA1C60>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\Jason.azure\commands\2025-01-06.20-08-56.extension_add.19772.log'.
az_command_data_logger: command args: extension add --name {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x000001DC10CD96C0>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x000001DC10CD9760>, <function register_cache_arguments..add_cache_arguments at 0x000001DC10CD98A0>, <function register_upcoming_breaking_change_info..update_breaking_change_info at 0x000001DC10CD9940>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x000001DC0E93C0E0>, <function CLIQuery.handle_query_parameter at 0x000001DC0E9654E0>, <function register_ids_argument..parse_ids_arguments at 0x000001DC10CD9800>]
urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
urllib3.connectionpool: https://aka.ms:443 "GET /azure-cli-extension-index-v1 HTTP/1.1" 301 0
urllib3.connectionpool: Starting new HTTPS connection (1): azcliextensionsync.blob.core.windows.net:443
urllib3.connectionpool: https://azcliextensionsync.blob.core.windows.net:443 "GET /index1/index.json HTTP/1.1" 200 4800148
cli.azure.cli.core.extension._resolve: Candidates ['databricks-0.1.0-py3-none-any.whl', 'databricks-0.2.0-py3-none-any.whl', 'databricks-0.3.0-py3-none-any.whl', 'databricks-0.4.0-py3-none-any.whl', 'databricks-0.5.0-py3-none-any.whl', 'databricks-0.6.0-py3-none-any.whl', 'databricks-0.7.0-py3-none-any.whl', 'databricks-0.7.1-py3-none-any.whl', 'databricks-0.7.2-py3-none-any.whl', 'databricks-0.7.3-py3-none-any.whl', 'databricks-0.8.0-py3-none-any.whl', 'databricks-0.9.0-py3-none-any.whl', 'databricks-0.10.0-py3-none-any.whl', 'databricks-0.10.1-py3-none-any.whl', 'databricks-0.10.2-py3-none-any.whl', 'databricks-1.0.0-py3-none-any.whl', 'databricks-1.0.1-py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['databricks-0.1.0-py3-none-any.whl', 'databricks-0.2.0-py3-none-any.whl', 'databricks-0.3.0-py3-none-any.whl', 'databricks-0.4.0-py3-none-any.whl', 'databricks-0.5.0-py3-none-any.whl', 'databricks-0.6.0-py3-none-any.whl', 'databricks-0.7.0-py3-none-any.whl', 'databricks-0.7.1-py3-none-any.whl', 'databricks-0.7.2-py3-none-any.whl', 'databricks-0.7.3-py3-none-any.whl', 'databricks-0.8.0-py3-none-any.whl', 'databricks-0.9.0-py3-none-any.whl', 'databricks-0.10.0-py3-none-any.whl', 'databricks-0.10.1-py3-none-any.whl', 'databricks-0.10.2-py3-none-any.whl', 'databricks-1.0.0-py3-none-any.whl', 'databricks-1.0.1-py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['databricks-0.1.0-py3-none-any.whl', 'databricks-0.2.0-py3-none-any.whl', 'databricks-0.7.0-py3-none-any.whl', 'databricks-0.7.1-py3-none-any.whl', 'databricks-0.7.2-py3-none-any.whl', 'databricks-0.7.3-py3-none-any.whl', 'databricks-0.8.0-py3-none-any.whl', 'databricks-0.9.0-py3-none-any.whl', 'databricks-0.10.0-py3-none-any.whl', 'databricks-0.10.1-py3-none-any.whl', 'databricks-0.10.2-py3-none-any.whl', 'databricks-1.0.0-py3-none-any.whl', 'databricks-1.0.1-py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['databricks-0.1.0-py3-none-any.whl', 'databricks-0.2.0-py3-none-any.whl', 'databricks-0.7.0-py3-none-any.whl', 'databricks-0.7.1-py3-none-any.whl', 'databricks-0.7.2-py3-none-any.whl', 'databricks-0.7.3-py3-none-any.whl', 'databricks-0.8.0-py3-none-any.whl', 'databricks-0.9.0-py3-none-any.whl', 'databricks-0.10.0-py3-none-any.whl', 'databricks-0.10.1-py3-none-any.whl', 'databricks-0.10.2-py3-none-any.whl', 'databricks-1.0.0-py3-none-any.whl', 'databricks-1.0.1-py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['databricks-0.1.0-py3-none-any.whl', 'databricks-0.2.0-py3-none-any.whl', 'databricks-0.7.0-py3-none-any.whl', 'databricks-0.7.1-py3-none-any.whl', 'databricks-0.7.2-py3-none-any.whl', 'databricks-0.7.3-py3-none-any.whl', 'databricks-0.8.0-py3-none-any.whl', 'databricks-0.9.0-py3-none-any.whl', 'databricks-0.10.0-py3-none-any.whl', 'databricks-0.10.1-py3-none-any.whl', 'databricks-0.10.2-py3-none-any.whl', 'databricks-1.0.0-py3-none-any.whl', 'databricks-1.0.1-py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Chosen {'downloadUrl': 'https://azcliprod.blob.core.windows.net/cli-extensions/databricks-1.0.1-py3-none-any.whl', 'filename': 'databricks-1.0.1-py3-none-any.whl', 'metadata': {'azext.minCliCoreVersion': '2.57.0', 'classifiers': ['Development Status :: 4 - Beta', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'Programming Language :: Python', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.8', 'License :: OSI Approved :: MIT License'], 'extensions': {'python.details': {'contacts': [{'email': '[email protected]', 'name': 'Microsoft Corporation', 'role': 'author'}], 'document_names': {'description': 'DESCRIPTION.rst'}, 'project_urls': {'Home': 'https://github.com/Azure/azure-cli-extensions/tree/main/src/databricks'}}}, 'generator': 'bdist_wheel (0.30.0)', 'license': 'MIT', 'metadata_version': '2.0', 'name': 'databricks', 'summary': 'Microsoft Azure Command-Line Tools DatabricksClient Extension', 'version': '1.0.1'}, 'sha256Digest': '55165d7435cd7a0dc87587e82fc202d792a5f28c9c44398655b0a4770d038c30'}
cli.azure.cli.core.extension.operations: Extension source is url? True
cli.azure.cli.core.extension.operations: Downloading https://azcliprod.blob.core.windows.net/cli-extensions/databricks-1.0.1-py3-none-any.whl to C:\Users\Jason\AppData\Local\Temp\tmpit6m9ctw\databricks-1.0.1-py3-none-any.whl
urllib3.connectionpool: Starting new HTTPS connection (1): azcliprod.blob.core.windows.net:443
urllib3.connectionpool: https://azcliprod.blob.core.windows.net:443 "GET /cli-extensions/databricks-1.0.1-py3-none-any.whl HTTP/1.1" 200 92474
cli.azure.cli.core.extension.operations: Downloaded to C:\Users\Jason\AppData\Local\Temp\tmpit6m9ctw\databricks-1.0.1-py3-none-any.whl
cli.azure.cli.core.extension.operations: Validating the extension C:\Users\Jason\AppData\Local\Temp\tmpit6m9ctw\databricks-1.0.1-py3-none-any.whl
cli.azure.cli.core.extension.operations: Checksum of C:\Users\Jason\AppData\Local\Temp\tmpit6m9ctw\databricks-1.0.1-py3-none-any.whl is OK
cli.azure.cli.core.extension.operations: Validation successful on C:\Users\Jason\AppData\Local\Temp\tmpit6m9ctw\databricks-1.0.1-py3-none-any.whl
cli.azure.cli.core.extension.operations: Executing pip with args: ['install', '--target', 'C:\Users\Jason\.azure\cliextensions\databricks', 'C:\Users\Jason\AppData\Local\Temp\tmpit6m9ctw\databricks-1.0.1-py3-none-any.whl']
cli.azure.cli.core.extension.operations: Running: ['C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe', '-m', 'pip', 'install', '--target', 'C:\Users\Jason\.azure\cliextensions\databricks', 'C:\Users\Jason\AppData\Local\Temp\tmpit6m9ctw\databricks-1.0.1-py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']
cli.azure.cli.core.extension.operations: Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_main.py", line 24, in
sys.exit(_main())
^^^^^^^
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\cli\main.py", line 65, in main
cmd_name, cmd_args = parse_command(args)
^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\cli\main_parser.py", line 79, in parse_command
general_options, args_else = parser.parse_args(args)
^^^^^^^^^^^^^^^^^^^^^^^
File "optparse.py", line 1371, in parse_args
values = self.get_default_values()
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\cli\parser.py", line 279, in get_default_values
self.config.load()
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\configuration.py", line 124, in load
self._load_config_files()
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\configuration.py", line 246, in _load_config_files
config_files = dict(self.iter_config_files())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\configuration.py", line 339, in iter_config_files
config_files = get_configuration_files()
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\configuration.py", line 70, in get_configuration_files
os.path.join(path, CONFIG_BASENAME) for path in appdirs.site_config_dirs("pip")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\utils\appdirs.py", line 47, in site_config_dirs
dirval = appdirs.site_config_dir(appname, appauthor=False, multipath=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_vendor\platformdirs_init.py", line 142, in site_config_dir
).site_config_dir
^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_vendor\platformdirs\windows.py", line 67, in site_config_dir
return self.site_data_dir
^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_vendor\platformdirs\windows.py", line 56, in site_data_dir
path = os.path.normpath(get_win_folder("CSIDL_COMMON_APPDATA"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_vendor\platformdirs\windows.py", line 209, in get_win_folder_from_registry
directory, _ = winreg.QueryValueEx(key, shell_folder_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [WinError 2] The system cannot find the file specified

cli.azure.cli.core.extension.operations: Command '['C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe', '-m', 'pip', 'install', '--target', 'C:\Users\Jason\.azure\cliextensions\databricks', 'C:\Users\Jason\AppData\Local\Temp\tmpit6m9ctw\databricks-1.0.1-py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at C:\Users\Jason.azure\cliextensions\databricks
cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 666, in execute
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 733, in _run_jobs_serially
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 703, in _run_job
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 336, in call
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/operations.py", line 343, in add_extension
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/operations.py", line 172, in _add_whl_ext
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.

cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x000001DC10BA1EE0>]
az_command_data_logger: exit code: 1
cli.main: Command ran in 82.600 seconds (init: 0.818, invoke: 81.782)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3857 in cache file under C:\Users\Jason.azure\telemetry\20250106201018859
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\azure\cli\telemetry_init_.pyc C:\Users\Jason.azure C:\Users\Jason.azure\telemetry\20250106201018859"
telemetry.process: Return from creating process 16048
telemetry.main: Finish creating telemetry upload process.

Expected behavior

the databricks extension installed successfully

Environment Summary

azure-cli 2.67.0

core 2.67.0
telemetry 1.1.0

Dependencies:
msal 1.31.0
azure-mgmt-resource 23.1.1

Python location 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\Jason.azure\cliextensions'

Python (Windows) 3.12.7 (tags/v3.12.7:0b05ead, Oct 1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)]

Legal docs and information: aka.ms/AzureCliLegal

Unable to check if your CLI is up-to-date. Check your internet connection.

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[microsoft-github-policy-service]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @edwinc, @mvvsubbu.