Graphics programming • abnormal 250ms delay in h264 decoding using OPENGL ES2.0/DRM

Hi,

I'm currently developing an H.264 decoder on Raspberry Pi OS (Bookworm Lite) and would appreciate your insights and suggestions to resolve a performance issue I'm encountering.

Setup:

• Decoder API: I'm using libavcodec.
• Rendering: Utilizing OpenGL ES 2.0.
• Linux DRM: Employed for display purposes.
• I've modified the rpicam_vid application (v1.4.1 release, rpicam-apps on GitHub(https://github.com/raspberrypi/rpicam-apps/tree/v1.4.1)) to include an H.264 video decoder. I referenced Fredrum's hwdectogl project(https://github.com/Fredrum/hwdectogl/tree/master) as a design guide.

Current Workflow:

• Capturing H.264 video stream at 1280x960 @ 25 fps with rpicam_vid.
• Decoding the video stream using libavcodec.
• Rendering the decoded video on an OpenGL ES 2.0 context.
• Displaying the output on a DRM overlay plane(the overlay plane is set using drmModeSetPlane (in a similar way to https://github.com/raspberrypi/rpicam-a ... w.cpp#L405)) over the DRM camera preview .

Issue:

• Despite my expectations for smooth and almost perfect overlaying of the decoded video over the camera preview, I'm experiencing a delay of approximately 250 milliseconds.

Questions:

• Could there be an issue with incorrect libavcodec parameters?
• Are there limitations when simultaneously encoding, decoding, using OpenGL ES 2.0, and DRM?
• Is there a potential problem in how I'm handling the primary/overlay plane?

Any guidance on resolving this delay or optimizing the setup would be greatly appreciated.

Thank you!

Statistics: Posted by longo92 — Thu Apr 18, 2024 7:24 am

---