runtime error on Dell/Intel server #25

ezekriSCW · 2024-07-04T07:52:57Z

When trying to use hwbench with a Dell C6420 server (Intel Xeon Silver 4114), an error is raised with following bt.
Note: it seems that CPU vendor hasn't been detected properly.

# python3 -m hwbench.hwbench -j configs/simple.conf -m monitoring.cfg
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/hwbench/hwbench/hwbench.py", line 10, in <module>
    from .bench import benchmarks
  File "/tmp/hwbench/hwbench/bench/benchmarks.py", line 6, in <module>
    from .benchmark import Benchmark
  File "/tmp/hwbench/hwbench/bench/benchmark.py", line 4, in <module>
    from .parameters import BenchmarkParameters
  File "/tmp/hwbench/hwbench/bench/parameters.py", line 3, in <module>
    from .monitoring import Monitoring
  File "/tmp/hwbench/hwbench/bench/monitoring.py", line 4, in <module>
    from ..environment.hardware import BaseHardware
  File "/tmp/hwbench/hwbench/environment/hardware.py", line 7, in <module>
    from .vendors.detect import first_matching_vendor
  File "/tmp/hwbench/hwbench/environment/vendors/detect.py", line 3, in <module>
    from .amd.amd import Amd
  File "/tmp/hwbench/hwbench/environment/vendors/amd/amd.py", line 1, in <module>
    from ..vendor import Vendor
  File "/tmp/hwbench/hwbench/environment/vendors/vendor.py", line 11, in <module>
    from ...utils.external import External
  File "/tmp/hwbench/hwbench/utils/external.py", line 8, in <module>
    class External(ABC):
  File "/tmp/hwbench/hwbench/utils/external.py", line 14, in External
    def run_cmd(self) -> list[str]:
TypeError: 'type' object is not subscriptable

The text was updated successfully, but these errors were encountered:

ErwanAliasr1 · 2024-07-04T09:40:04Z

Can you share a "dmidecode -t 1" & "dmidecode -t 3" and a "lscpu"

ErwanAliasr1 · 2024-07-04T10:11:58Z

Please note you are running Python 3.8 while 3.9 is the minimal release

ezekriSCW · 2024-07-04T10:16:19Z

arf! I missed that!
I'll first update python, than I'll check if I still have the same error.
Thanks

ErwanAliasr1 · 2024-07-04T10:23:19Z

I'm pushing a change to ensure no one runs below the minimal release.
#26

ErwanAliasr1 · 2024-07-04T12:53:32Z

PR #26 is merged to avoid this improper python release issue in the future.

ezekriSCW · 2024-07-04T16:31:51Z

Python upgraded to 3.9
still have an error, but a different one

# python3 -m hwbench.hwbench -j configs/simple.conf -m monitoring.cfg
Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/hwbench/hwbench/hwbench.py", line 108, in <module>
    main()
  File "/tmp/hwbench/hwbench/hwbench.py", line 30, in main
    tuning_setup.Tuning(tuning_out_dir).apply()
  File "/tmp/hwbench/hwbench/tuning/setup.py", line 15, in apply
    PerformancePowerProfile(self.out_dir).run()
  File "/tmp/hwbench/hwbench/tuning/power_profile.py", line 33, in run
    previous = file.read_text(encoding="utf-8").rstrip()
  File "/usr/lib/python3.9/pathlib.py", line 1256, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
  File "/usr/lib/python3.9/pathlib.py", line 1242, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/usr/lib/python3.9/pathlib.py", line 1110, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/sys/devices/system/cpu/cpu21/cpufreq/scaling_governor'

# dmidecode -t 1
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 3.2.0 present.

Handle 0x0100, DMI type 1, 27 bytes
System Information
	Manufacturer: Dell Inc.
	Product Name: PowerEdge C6420
	Version: Not Specified
	Serial Number: xxxxxx
	UUID: xxxxxxx
	Wake-up Type: Power Switch
	SKU Number: SKU=0757;ModelName=PowerEdge C6420
	Family: PowerEdge

# dmidecode -t 3
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 3.2.0 present.

Handle 0x0300, DMI type 3, 22 bytes
Chassis Information
	Manufacturer: Dell Inc.
	Type: Rack Mount Chassis
	Lock: Not Present
	Version: PowerEdge C6400
	Serial Number: xxxxxxx
	Asset Tag: Not Specified
	Boot-up State: Safe
	Power Supply State: Safe
	Thermal State: Safe
	Security Status: Unknown
	OEM Information: 0x00000000
	Height: 2 U
	Number Of Power Cords: 2
	Contained Elements: 0
	SKU Number: Not Specified

(hided s/n)

ErwanAliasr1 · 2024-07-11T12:09:26Z

@ezekriSCW can you provide the result of a " find /sys/devices/system/cpu", "uname -a" , "cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors " , "/sys/devices/system/cpu/cpuidle/available_governors",
please ?

ezekriSCW · 2024-07-22T09:43:20Z

find_cpu.txt

# uname -a
Linux poweredge-c6420-xxxxxx 5.8.0-38-generic #43~20.04.1-Ubuntu SMP Tue Jan 12 16:39:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
cat: /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors: No such file or directory

# cat /sys/devices/system/cpu/cpuidle/available_governors
ladder menu teo

The current code do not check if the scaling_governor is present before using it, leading to a crash. To improve this part of the code, this commit is: - considering the current_governor instead of the available ones to decide if the tuning should be done - adding a check before actually reading the file and ignore the function if no file found - do not set the tuning if tuning is already in place This commit fixes issue #25. Signed-off-by: Erwan Velu <[email protected]>

The current code do not check if the scaling_governor is present before using it, leading to a crash. To improve this part of the code, this commit is: - considering the current_governor instead of the available ones to decide if the tuning should be done - adding a check before actually reading the scaling governor file and ignore the function if no file found This commit fixes issue #25. Signed-off-by: Erwan Velu <[email protected]>

ErwanAliasr1 · 2024-08-19T15:04:51Z

Please confirm the latest code with PR30 merged is solving your case

ErwanAliasr1 · 2024-10-15T14:35:26Z

@ezekriSCW ^

ezekriSCW · 2024-10-21T15:16:06Z

Hi,
sorry for the late answer.
I have a new error regarding a missing Redfish URI (below the trace).
It's possibly because I'm using an old idrac fw version (4.22)...

# python3 -m hwbench.hwbench -j configs/simple.conf -m monitoring.cfg
copy_file( hwbench-out-20241021150918 , /proc/config.gz )  got: [Errno 2] No such file or directory: '/proc/config.gz'
Monitoring/turbostat: initialize
Monitoring/turbostat: Detected release 2024.7.26
Monitoring/turbostat: Freq metrics:20xCPU
Monitoring/BMC: initialize DELL vendor with IDRAC driver @ https://10.42.1.3 Model: '14G DCS' FW: '4.22.00.53'
Monitoring/BMC: Thermal metrics:1xNone, 2xCPU
Monitoring/BMC: Fans metrics:8xFan
ERROR:root:Parsing redfish url /redfish/v1/Managers/iDRAC.Embedded.1/Oem/Dell/DellAttributes/System.Embedded.1 failed : {'error': {'@Message.ExtendedInfo': [{'Message': 'Unable to complete the operation because the resource DellAttributes entered is not found.', 'MessageArgs': ['DellAttributes'], '[email protected]': 1, 'MessageId': 'IDRAC.2.1.SYS403', 'RelatedProperties': [], '[email protected]': 0, 'Resolution': 'Enter the correct resource and retry the operation. For information about valid resource, see the Redfish Users Guide available on the support site.', 'Severity': 'Critical'}, {'Message': 'The resource at the URI DellAttributes was not found.', 'MessageArgs': ['DellAttributes'], '[email protected]': 1, 'MessageId': 'Base.1.5.ResourceMissingAtURI', 'RelatedProperties': [''], '[email protected]': 1, 'Resolution': 'Place a valid resource at the URI or correct the URI and resubmit the request.', 'Severity': 'Critical'}], 'code': 'Base.1.5.GeneralError', 'message': 'A general error has occurred. See ExtendedInfo for more information'}}
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/hwbench/hwbench/hwbench.py", line 117, in <module>
    main()
  File "/home/hwbench/hwbench/hwbench.py", line 44, in main
    benches.parse_jobs_config()
  File "/home/hwbench/hwbench/bench/benchmarks.py", line 124, in parse_jobs_config
    self.__schedule_benchmarks(
  File "/home/hwbench/hwbench/bench/benchmarks.py", line 145, in __schedule_benchmarks
    self.__schedule_benchmark(job, pinned_cpu, emp, validate_parameters)
  File "/home/hwbench/hwbench/bench/benchmarks.py", line 165, in __schedule_benchmark
    self.monitoring = Monitoring(self.out_dir, self.jobs_config, self.hardware)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hwbench/hwbench/bench/monitoring.py", line 40, in __init__
    self.prepare()
  File "/home/hwbench/hwbench/bench/monitoring.py", line 103, in prepare
    self.vendor.get_bmc().read_power_consumption(
  File "/home/hwbench/hwbench/environment/vendors/dell/dell.py", line 46, in read_power_consumption
    if "ServerPwr.1.SCViewSledPwr" in oem_system["Attributes"]:
                                      ~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: 'Attributes'

ErwanAliasr1 · 2024-10-22T08:52:19Z

Can you give me the content of "/redfish/v1/Managers/iDRAC.Embedded.1/Oem/Dell/DellAttributes/System.Embedded.1" ?

ezekriSCW · 2024-10-22T09:18:35Z

GET /redfish/v1/Managers/iDRAC.Embedded.1/Oem/Dell/DellAttributes/System.Embedded.1
redfishtool: Transport: Response Error: status_code: 404 -- Not Found
redfishtool: raw: Error getting response

ErwanAliasr1 · 2024-10-24T15:03:48Z

I think we can close this one @ezekriSCW

ezekriSCW · 2024-10-24T15:58:09Z

resolved, thx

ErwanAliasr1 mentioned this issue Aug 19, 2024

tuning: Avoid crash in PerformancePowerProfile #30

Merged

ErwanAliasr1 self-assigned this Aug 19, 2024

ErwanAliasr1 mentioned this issue Oct 22, 2024

Adding Dell Gen14 support #33

Merged

ErwanAliasr1 added bug Something isn't working hwbench labels Oct 24, 2024

ezekriSCW closed this as completed Oct 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime error on Dell/Intel server #25

runtime error on Dell/Intel server #25

ezekriSCW commented Jul 4, 2024

ErwanAliasr1 commented Jul 4, 2024 •

edited

Loading

ErwanAliasr1 commented Jul 4, 2024

ezekriSCW commented Jul 4, 2024

ErwanAliasr1 commented Jul 4, 2024 •

edited

Loading

ErwanAliasr1 commented Jul 4, 2024 •

edited

Loading

ezekriSCW commented Jul 4, 2024

ErwanAliasr1 commented Jul 11, 2024 •

edited

Loading

ezekriSCW commented Jul 22, 2024 •

edited

Loading

ErwanAliasr1 commented Aug 19, 2024

ErwanAliasr1 commented Oct 15, 2024

ezekriSCW commented Oct 21, 2024

ErwanAliasr1 commented Oct 22, 2024

ezekriSCW commented Oct 22, 2024

ErwanAliasr1 commented Oct 24, 2024

ezekriSCW commented Oct 24, 2024

runtime error on Dell/Intel server #25

runtime error on Dell/Intel server #25

Comments

ezekriSCW commented Jul 4, 2024

ErwanAliasr1 commented Jul 4, 2024 • edited Loading

ErwanAliasr1 commented Jul 4, 2024

ezekriSCW commented Jul 4, 2024

ErwanAliasr1 commented Jul 4, 2024 • edited Loading

ErwanAliasr1 commented Jul 4, 2024 • edited Loading

ezekriSCW commented Jul 4, 2024

ErwanAliasr1 commented Jul 11, 2024 • edited Loading

ezekriSCW commented Jul 22, 2024 • edited Loading

ErwanAliasr1 commented Aug 19, 2024

ErwanAliasr1 commented Oct 15, 2024

ezekriSCW commented Oct 21, 2024

ErwanAliasr1 commented Oct 22, 2024

ezekriSCW commented Oct 22, 2024

ErwanAliasr1 commented Oct 24, 2024

ezekriSCW commented Oct 24, 2024

ErwanAliasr1 commented Jul 4, 2024 •

edited

Loading

ErwanAliasr1 commented Jul 4, 2024 •

edited

Loading

ErwanAliasr1 commented Jul 4, 2024 •

edited

Loading

ErwanAliasr1 commented Jul 11, 2024 •

edited

Loading

ezekriSCW commented Jul 22, 2024 •

edited

Loading