Atomic modesetting on RPi

[FallingSnow]

I copied the example for atomic mode setting but changed the card to /dev/dri/card1. It keeps failing with

thread 'main' panicked at 'Failed to set mode: InvalidArgument'

Any advice on how to debug it?

[Drakulix]

The wlroots instructions on DRM Debugging can be helpful for debugging atomic modesetting: https://gitlab.freedesktop.org/wlroots/wlroots/-/wikis/DRM-Debugging

Does the legacy_modesetting example work for you?

[FallingSnow]

Thanks for the link, I'll do that.

I did legacy modesetting without the example and it was working.
I'm in the middle of rewritting the application right now so I'll try atomic modesetting once I get legacy modesetting working again.

[Slabity]

I believe freedreno (the open-source driver for Qualcomm GPUs) does support atomic modesetting, but if you are using an older kernel it might not work.

If you want to get atomic modesetting working, please try installing kmscube and try running it with --atomic. That's an easy way to check if your system actually supports it.

[FallingSnow]

@Slabity Tried it. Not sure what no eglDupNativeFenceFDANDROID means.

$ uname -a
Linux raspberrypi 5.15.56-v8+ #1575 SMP PREEMPT Fri Jul 22 20:31:26 BST 2022 aarch64 GNU/Linux
$ kmscube --atomic
Using display 0x5572933c90 with EGL version 1.4
===================================
EGL information:
  version: "1.4"
  vendor: "Mesa Project"
  client extensions: "EGL_EXT_device_base EGL_EXT_device_enumeration EGL_EXT_device_query EGL_EXT_platform_base EGL_KHR_client_get_all_proc_addresses EGL_EXT_client_extensions EGL_KHR_debug EGL_EXT_platform_device EGL_EXT_platform_wayland EGL_KHR_platform_wayland EGL_EXT_platform_x11 EGL_KHR_platform_x11 EGL_MESA_platform_gbm EGL_KHR_platform_gbm EGL_MESA_platform_surfaceless"
  display extensions: "EGL_ANDROID_blob_cache EGL_EXT_buffer_age EGL_EXT_image_dma_buf_import EGL_EXT_image_dma_buf_import_modifiers EGL_KHR_cl_event2 EGL_KHR_config_attribs EGL_KHR_create_context EGL_KHR_create_context_no_error EGL_KHR_fence_sync EGL_KHR_get_all_proc_addresses EGL_KHR_gl_colorspace EGL_KHR_gl_renderbuffer_image EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_3D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_image EGL_KHR_image_base EGL_KHR_image_pixmap EGL_KHR_no_config_context EGL_KHR_reusable_sync EGL_KHR_surfaceless_context EGL_EXT_pixel_format_float EGL_KHR_wait_sync EGL_MESA_configless_context EGL_MESA_drm_image EGL_MESA_image_dma_buf_export EGL_MESA_query_driver EGL_WL_bind_wayland_display "
===================================
OpenGL ES 2.x information:
  version: "OpenGL ES 3.1 Mesa 20.3.5"
  shading language version: "OpenGL ES GLSL ES 3.10"
  vendor: "Broadcom"
  renderer: "V3D 4.2"
  extensions: "GL_EXT_blend_minmax GL_EXT_multi_draw_arrays GL_EXT_texture_filter_anisotropic GL_EXT_texture_compression_s3tc GL_EXT_texture_compression_dxt1 GL_EXT_texture_format_BGRA8888 GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth24 GL_OES_element_index_uint GL_OES_fbo_render_mipmap GL_OES_mapbuffer GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_stencil8 GL_OES_texture_3D GL_OES_texture_float GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_half_float GL_EXT_draw_instanced GL_EXT_texture_sRGB_decode GL_OES_EGL_image GL_OES_depth_texture GL_OES_packed_depth_stencil GL_EXT_texture_type_2_10_10_10_REV GL_OES_get_program_binary GL_APPLE_texture_max_level GL_EXT_discard_framebuffer GL_EXT_read_format_bgra GL_EXT_frag_depth GL_NV_fbo_color_attachments GL_OES_EGL_image_external GL_OES_EGL_sync GL_OES_vertex_array_object GL_ANGLE_pack_reverse_row_order GL_ANGLE_texture_compression_dxt3 GL_ANGLE_texture_compression_dxt5 GL_EXT_occlusion_query_boolean GL_EXT_texture_rg GL_EXT_unpack_subimage GL_NV_draw_buffers GL_NV_read_buffer GL_NV_read_depth GL_NV_read_depth_stencil GL_NV_read_stencil GL_EXT_draw_buffers GL_EXT_map_buffer_range GL_KHR_debug GL_KHR_texture_compression_astc_ldr GL_NV_pixel_buffer_object GL_OES_depth_texture_cube_map GL_OES_required_internalformat GL_OES_surfaceless_context GL_EXT_color_buffer_float GL_EXT_sRGB_write_control GL_EXT_separate_shader_objects GL_EXT_shader_implicit_conversions GL_EXT_shader_integer_mix GL_EXT_base_instance GL_EXT_compressed_ETC1_RGB8_sub_texture GL_EXT_draw_elements_base_vertex GL_EXT_primitive_bounding_box GL_EXT_shader_io_blocks GL_EXT_texture_border_clamp GL_EXT_texture_view GL_KHR_context_flush_control GL_NV_image_formats GL_OES_draw_elements_base_vertex GL_OES_primitive_bounding_box GL_OES_shader_io_blocks GL_OES_texture_border_clamp GL_OES_texture_stencil8 GL_OES_texture_storage_multisample_2d_array GL_OES_texture_view GL_EXT_buffer_storage GL_EXT_float_blend GL_EXT_geometry_point_size GL_EXT_geometry_shader GL_KHR_no_error GL_KHR_texture_compression_astc_sliced_3d GL_OES_EGL_image_external_essl3 GL_OES_geometry_point_size GL_OES_geometry_shader GL_OES_shader_image_atomic GL_EXT_texture_compression_s3tc_srgb GL_MESA_shader_integer_functions GL_KHR_parallel_shader_compile GL_EXT_EGL_image_storage GL_MESA_framebuffer_flip_y GL_EXT_texture_query_lod "
===================================
no eglDupNativeFenceFDANDROID

[Slabity]

Hmm... Well I'm not familiar with egl, but I think that is likely an entirely different issue. It does look like you are using a recent kernel though, so I'm not sure what the problem might be.

If you check your kernel logs with dmesg, do you see anything that mentions atomic modesetting? A quick dmesg | grep -i atomic might mention if that's happening.

[FallingSnow]

Yeah, there are some messages.
I think the timeout messages are coming from some modesetting I've been doing that is kind of working (flashing black screen), I've included the legacy modesetting code that is causing the flashing black screen.

$ dmesg | grep -i atomic
[    0.072085] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations
[    0.072434] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.073382] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[78540.328830] [drm:drm_atomic_helper_wait_for_flip_done [drm_kms_helper]] *ERROR* [CRTC:94:crtc-3] flip_done timed out
[78550.569300] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:94:crtc-3] commit wait timed out
[78560.809587] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:84:plane-3] commit wait timed out
[79188.028032] [drm:drm_atomic_helper_wait_for_flip_done [drm_kms_helper]] *ERROR* [CRTC:94:crtc-3] flip_done timed out
[79198.260111] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:94:crtc-3] commit wait timed out
[79208.495757] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:84:plane-3] commit wait timed out
[79258.153154] [drm:drm_atomic_helper_wait_for_flip_done [drm_kms_helper]] *ERROR* [CRTC:94:crtc-3] flip_done timed out
[79268.394048] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:94:crtc-3] commit wait timed out
[79278.634299] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:84:plane-3] commit wait timed out
[79321.129149] [drm:drm_atomic_helper_wait_for_flip_done [drm_kms_helper]] *ERROR* [CRTC:94:crtc-3] flip_done timed out
[79331.368929] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:94:crtc-3] commit wait timed out
[79341.608405] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:84:plane-3] commit wait timed out
[79411.495690] [drm:drm_atomic_helper_wait_for_flip_done [drm_kms_helper]] *ERROR* [CRTC:94:crtc-3] flip_done timed out
[79421.736150] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [CRTC:94:crtc-3] commit wait timed out
[79431.976225] [drm:drm_atomic_helper_wait_for_dependencies [drm_kms_helper]] *ERROR* [PLANE:84:plane-3] commit wait timed out
...

// let card = passed into this function
let resources = card.resource_handles().expect("Could not get resource handles");
let connectors = resources.connectors();
let connector_handle = connectors[0];
let connector = card.get_connector(connector_handle).expect("Could not get connector");
let curr_encoder = connector.current_encoder().unwrap();
let encoder = card.get_encoder(curr_encoder).expect("Could not get encoder");

let crtc = encoder.crtc().unwrap();
println!("Using {:?}@{:?}", crtc, mode);

// let fb = passed into this function
// let mode = passed into this function

card.set_crtc(crtc, fb, (0, 0), &[connector_handle], mode).expect("Could not set crtc");

[Slabity]

I'll be honest, I'm not really sure how to further debug this. I'm fairly confident in the crate's code for handling the atomic ioctl operations, but I'm less confident in the example code itself. An InvalidArgument result from an ioctl call could be a variety of different issues, including whether a platform is unable to even process that call correctly. Though I would expect dmesg to say something like broken atomic modeset userspace detected, disabling atomic, which would at least indicate that your platform doesn't handle it correctly. Unfortunately the kernel does not really return verbose errors to userspace that well.

Shooting in the dark here, but it might be worthwhile to add drm.debug=0xff to your kernel's command line (I think those are set in /boot/cmdline.txt on Raspian). That will make the DRM subsystem spit out the most information as possible into your kernel logs.

I really want to test this on an RPi4 myself (along with trying new LunarG Vulkan support added recently), but mine is currently running my 3d printer and I can't justify buying a new one with the current supply/price issues. I might be able to look closer in a few months once the prices come down, but I don't think there's much I can recommend other than trying to get kmscube --atomic working properly or try debugging the current example code as much as you can.

[FallingSnow]

Okay, I tried drm.debug=0xff while doing legacy modesetting via the legacy_modesetting example code. Did run into an interesting pair of lines

[   67.880918] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] [ENCODER:31:TMDS-31] incompatible with [CRTC:61:crtc-0]
[   67.880964] [drm:drm_atomic_check_only [drm]] atomic driver check for 00000000f664e8da failed: -22

Bigger excerpt from dmesg

...
[   67.880505] vc4-drm gpu: [drm:vc4_atomic_check [vc4]] crtc-3: Disabling, Freeing channel 0
[   67.880535] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] [CRTC:61:crtc-0] mode changed
[   67.880637] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] [CRTC:61:crtc-0] enable changed
[   67.880685] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] [CRTC:61:crtc-0] active changed
[   67.880731] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] [CRTC:94:crtc-3] mode changed
[   67.880776] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] [CRTC:94:crtc-3] enable changed
[   67.880821] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] [CRTC:94:crtc-3] active changed
[   67.880871] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] Updating routing for [CONNECTOR:32:HDMI-A-1]
[   67.880918] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] [ENCODER:31:TMDS-31] incompatible with [CRTC:61:crtc-0]
[   67.880964] [drm:drm_atomic_check_only [drm]] atomic driver check for 00000000f664e8da failed: -22
[   67.881058] [drm:drm_atomic_state_default_clear [drm]] Clearing atomic state 00000000f664e8da
[   67.881142] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 32 (6)
[   67.881224] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 32 (5)
[   67.881306] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 342 (1)
[   67.881394] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 340 (3)
[   67.881479] [drm:__drm_atomic_state_free [drm]] Freeing atomic state 00000000f664e8da
[   67.881561] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 340 (2)
[   67.881642] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 32 (4)
[   67.881731] [drm:drm_ioctl [drm]] comm="v4l2-pi-udp-rew", pid=796, ret=-22
[   67.888974] [drm:drm_release [drm]] open_count = 1
[   67.889110] [drm:drm_file_free.part.17 [drm]] comm="v4l2-pi-udp-rew", pid=796, dev=0xe201, open_count=1
[   67.889197] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 341 (1)
[   67.889282] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 340 (1)
[   67.889377] [drm:_drm_lease_revoke [drm]] revoke leases for 0000000072177b33 0
[   67.889461] [drm:drm_lease_destroy [drm]] drm_lease_destroy 0
[   67.889541] [drm:drm_lease_destroy [drm]] drm_lease_destroy done
...

[FallingSnow]

If I set fb = None then I get 'Could not set CRTC: Unknown { errno: ENOENT }'.

No error in dmesg, just seems to stop and return.

[ 1555.905486] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] Updating routing for [CONNECTOR:32:HDMI-A-1]
[ 1555.905560] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] [CONNECTOR:32:HDMI-A-1] keeps [ENCODER:31:TMDS-31], now on [CRTC:94:crtc-3]
[ 1555.905610] [drm:drm_atomic_add_encoder_bridges [drm]] Adding all bridges for [encoder:31:TMDS-31] to 0000000093d605c7
[ 1555.905700] [drm:drm_atomic_add_encoder_bridges [drm]] Adding all bridges for [encoder:31:TMDS-31] to 0000000093d605c7
[ 1555.905826] [drm:drm_atomic_get_private_obj_state [drm]] Added new private object 00000000dbf46e89 state 0000000017e984f0 to 0000000093d605c7
[ 1555.905917] [drm:drm_atomic_commit [drm]] committing 0000000093d605c7
[ 1555.907709] vc4-drm gpu: [drm:drm_calc_timestamping_constants [drm]] crtc 94: hwmode: htotal 2200, vtotal 1125, vdisplay 1080
[ 1555.907810] vc4-drm gpu: [drm:drm_calc_timestamping_constants [drm]] crtc 94: clock 148500 kHz framedur 16666666 linedur 14814
[ 1555.907911] vc4-drm gpu: [drm:drm_vblank_enable [drm]] enabling vblank on crtc 3, ret: 0
[ 1555.921046] vc4-drm gpu: [drm:vc4_atomic_commit_tail [vc4]] Running the core clock at 133650000 Hz
[ 1555.922665] vc4-drm gpu: [drm:vc4_atomic_commit_tail [vc4]] Core clock actual rate: 200000000 Hz
[ 1555.922715] [drm:drm_atomic_state_default_clear [drm]] Clearing atomic state 0000000093d605c7
[ 1555.922827] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 32 (5)
[ 1555.922910] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 32 (4)
[ 1555.923002] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 339 (2)
[ 1555.923092] [drm:drm_mode_object_put.part.5 [drm]] OBJ ID: 338 (3)
[ 1555.923190] [drm:__drm_atomic_state_free [drm]] Freeing atomic state 0000000093d605c7
[ 1555.923280] vc4-drm gpu: [drm:drm_client_dev_restore [drm]] fbdev: ret=0
[ 1555.937689] vc4-drm gpu: [drm:vblank_disable_fn [drm]] disabling vblank on crtc 3

[FallingSnow]

Side note. I plugged into a different monitor. kmscube does work but kmscube --atomic does not, same error as before.

[Drakulix]

Okay, I tried drm.debug=0xff while doing legacy modesetting via the legacy_modesetting example code. Did run into an interesting pair of lines

[   67.880918] [drm:drm_atomic_helper_check_modeset [drm_kms_helper]] [ENCODER:31:TMDS-31] incompatible with [CRTC:61:crtc-0]
[   67.880964] [drm:drm_atomic_check_only [drm]] atomic driver check for 00000000f664e8da failed: -22

That sounds like your setup is very picky about its encoders (which can be easily the reason for an atomic modeset to fail).
Can you run drm_info and post your results here? That might give us an idea, why the example is selecting a seemingly incompatible configuration.

[FallingSnow]

Note this is on the VGA monitor I switched to after my HDMI monitor was doing the flashing black screen while trying the modesetting code I pasted earlier. The HDMI monitor is slow at getting the signal from my laptop as well.

Raspberry Pi 4 with VGA monitor connected via an HDMI -> VGA adapter:

$ sudo ./drm_info > /tmp/drm_info_vga.log
drmModeGetResources: Operation not supported
Failed to retrieve information from /dev/dri/card0

drm_info_vga.log