Opened 3 years ago

Closed 3 years ago

Last modified 18 months ago

#12799 closed Developer Task (fixed)

Improve OSD on Raspberry Pi

Reported by: Peter Bennett Owned by: Peter Bennett
Priority: minor Milestone: unknown
Component: Ports - rPi Version: Master Head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

The playback OSD on Raspberry Pi has some problems:

  1. Can cause the video to become choppy due to high CPU utilization.
  2. On standard definition channels or videos the OSD is low-definition.

Fix this by using OpenGL to display the OSD over the video instead of merging the OSD into the frames. Leave the software merge available as an alternative in case of problems with OpenGL support.

Change History (8)

comment:1 Changed 3 years ago by Peter Bennett

Status: newaccepted

comment:2 Changed 3 years ago by Peter Bennett

I have the OSD displaying with OpenGL ES on raspberry pi, using the existing code that displays with OpenGL. All OSDs are displaying correctly and with excellent detail, but there is a severe performance problem. Playback is badly impacted when the OSD appears. Subtitles are the worst offender. In general performance is much worse than with Software merge.

Investigation shows that the call to OSD::DrawDirect is taking from 200 ms to 700 ms. Since this is done between frames in the frame rendering thread, up to 21 frames are lost when an OSD appears or changes. CPU time is not high so I suspect that the OpenGL calls are blocking.

Removing the call to glFlush improves performance, but not enough. With this change, calls to OSD::DrawDirect take from 50 ms to 200 ms.

Options:

  1. Investigate further into which specific OGL operations are slow and try to improve them.
  1. Put OSD rendering in a separate thread. This may require a new osd renderer object and ways of synchronizing information. Simply using the existing OSD object in another thread does not work, for several reasons.
  1. Use the OSD::Draw method instead, which draws into a memory image, and then use OpenGL to display that image. The OSD::Draw is used by software merge and is fast.

comment:3 in reply to:  2 Changed 3 years ago by Stuart Auchterlonie

Replying to pbennett:

Options:

  1. Investigate further into which specific OGL operations are slow and try to improve them.

This is probably worth doing anyway, as it will improve overall UX

  1. Put OSD rendering in a separate thread. This may require a new osd renderer object and ways of synchronizing information. Simply using the existing OSD object in another thread does not work, for several reasons.
  1. Use the OSD::Draw method instead, which draws into a memory image, and then use OpenGL to display that image. The OSD::Draw is used by software merge and is fast.

Which of these 2 is the "right way"? I currently lean towards #3 as that will be the easiest in the short term, but if it's not great in the long term then #2 becomes more appealing....

Regards Stuart

comment:4 Changed 3 years ago by Peter Bennett

The performance problems are now solved. I have now implemented two new OSD rendering methods that are available when selecting Openmax video rendering:

OpenGL - This uses the existing opengl code with good performance. All OSD show without any jerkiness, except for subtitles on Raspberry Pi 2. All OSD are fine with Pi 3.

Threaded - Combines 2 and 3 of the above proposal. On Raspberry Pi 2 this can be used if you want subtitles. It gives good OSD but keyboard response is sluggish.

There is one OpenGL performance enhancement - disabling flush. Flush is called frequently in the code and I am not sure why it is needed. I have a change that disables it only for the openmax OSD. That flag can also be enabled if needed in other cases.

To get these changes, opengl must be enabled in configure.

I will commit these changes soon. Currently they are in my bennettpeter git repo.

comment:5 Changed 3 years ago by Peter Bennett <pbennett@…>

Resolution: fixed
Status: acceptedclosed

In 4852d9c9374990c98c95422e9daca8d9cee97179/mythtv:

Improve OSD on Raspberry Pi.
OpenGL support for OSD on Raspbian.

Fixes #12799

comment:6 Changed 3 years ago by Peter Bennett

The improvements only work on raspbian, not on ubuntu.

To see the improved features add these options to configure

--enable-opengl --disable-opengl-themepainter

Select playback profile "OpenMax High Quality"

See wiki page https://www.mythtv.org/wiki/Raspberry_Pi

comment:7 Changed 3 years ago by Peter Bennett <pbennett@…>

In 5f200bf4b6a85da0c2a6a659312667de1e5ac4e8/mythtv:

Improve OSD on Raspberry Pi.
OpenGL support for OSD on Raspbian.

Fixes #12799

(cherry picked from commit 4852d9c9374990c98c95422e9daca8d9cee97179)

comment:8 Changed 18 months ago by Peter Bennett

Owner: changed from Peter Bennett to Peter Bennett
Note: See TracTickets for help on using tickets.