Opened 6 years ago

Closed 5 years ago

#10984 closed Developer Task (Fixed)

Improve closed caption performance

Reported by: Jim Stichnoth Owned by: Jim Stichnoth
Priority: minor Milestone: 0.28
Component: MythTV - Captions Version: Master Head
Severity: medium Keywords:
Cc: Ticket locked: no


High CPU usage when displaying "real-time" character-by-character CEA-708 captions, with "-v vbi", on an Atom frontend, resulting in stuttering. May also be true for CEA-608 captions. Unacceptable.

Change History (13)

comment:1 Changed 6 years ago by Jack Thomasson <jkt@…>

In 30f6e33cc25191a8b57492cdc1868845eaa713d8/mythtv:

Fixes #11072. Improve cc708 thread safety.

There still appear to be potential race conditions, which can be fixed
by adding more locking. However, this will have an unknown
performance impact, so we'll address it after the release.
Refs #10984.

Signed-off-by: Jim Stichnoth <jstichnoth@…>

comment:2 Changed 5 years ago by paulh

Milestone: 0.270.28

comment:3 Changed 5 years ago by Jim Stichnoth

A fundamental issue here is that the amount of work to render one subtitle can vary wildly, depending on the number of differently-formatted chunks in the subtitle, and the complexity of formatting in osd_subtitle.xml. For the latter, larger fonts, shadows, and outlines make rendering more expensive.

Since there are always many video frames between different subtitles, the processing time for one subtitle may cause a dropped frame, even when the average per-frame overhead is small.

The best way to deal with this may be to offload as much processing as possible into a helper thread, being careful to avoid non-thread-safe MythUI work in the helper thread. Performance-tuning the single-subtitle work is also worthwhile, but probably can't be the only approach since we are largely at the mercy of the subtitle provider, the themer (osd_subtitle.xml complexity), and the user (subtitle zoom factor in the Playback OSD).

comment:4 Changed 5 years ago by Jim Stichnoth <jstichnoth@…>

In 3325c09b2fc070179047f678a02ee6feaed5cfd7/mythtv:

Speed up drawing caption text with outlines.

The old method of drawing text with outlines involves drawing the same
text many times at various offsets, with the offsets based on
traversing the perimeter square of pixels with edge length 2N+1, where
N is the specified outlinesize. This means drawing 8N extra copies of
the text. With some fonts and point sizes, this can be a very
expensive operation, leading to video stuttering while captions are

The new method draws just one extra copy, but with a pen width set to
2N+1, which is much faster. Beware that using a huge outlinesize may
lead to strange artifacts in the outline (but that was already the
case for the original outline code).

Refs #10984.

comment:5 Changed 5 years ago by Jim Stichnoth <jstichnoth@…>

In 9380024634ce98b7c5f5d3ff08a9672f80a1217e/mythtv:

Use const ref args in the constructor. Refs #10984

comment:6 Changed 5 years ago by Jim Stichnoth <jstichnoth@…>

In e623425dfe4ef279a6c1362ef907b3723c3a0158/mythtv:

Use symbolic constants instead of 8 and 64. Refs #10984

comment:7 Changed 5 years ago by Jim Stichnoth <jstichnoth@…>

In 2c8c898f5d2f4b76f06a978a09830599dff530b5/mythtv:

Be less conservative about marking windows as changed. Refs #10984.

comment:8 Changed 5 years ago by Jim Stichnoth <jstichnoth@…>

In 2427afde4690eeb87d3a2af28951cc8bd9d1369b/mythtv:

Improve performance of processing CEA-708 captions. Refs #10984.

Change the complexity of creating cc708 strings from quadratic to

Allow leading and trailing strings of spaces to be joined to their
neighbor strings even when the pen is different, to reduce the number
of chunks to process. (These chunks would have been eventually
optimized away before drawing, but now the overall processing time is

comment:9 Changed 5 years ago by Jim Stichnoth <jstichnoth@…>

In ce8183ca568ecb246d9ce479c09b430cd9b5d6a4/mythtv:

Be less conservative about marking windows as changed. Refs #10984.

Missed one opportunity in 2c8c898f5d2f4b76f06a978a09830599dff530b5.

comment:10 Changed 5 years ago by Jim Stichnoth <jstichnoth@…>

In 6181935764d9eb330e4bb08d80435be429b314f9/mythtv:

Fix clang build error after 2427afd. Refs #10984.

comment:11 Changed 5 years ago by Jim Stichnoth <jstichnoth@…>

In 181ac7f15c160ed162f8932574b8c30497fae727/mythtv:

Reduce the number of CEA-708 caption redraws. Refs #10984

Receiving the SetWindowStyle? command doesn't mean that any of the
existing caption strings have changed.

comment:12 Changed 5 years ago by Jim Stichnoth <jstichnoth@…>

In cfc3b7c8e50038d52b739077068ffa7a8f82fd56/mythtv:

Subtitles: Clean up the code and reorganize somewhat.

  • The code specific to cc608, cc708, and srt is made more object-oriented.
  • Various external caches of the SubtitleScreen?'s children list (e.g. expiration time, cc708 window number, original widget area) are integrated directly into the children list for simplified management.
  • Method definitions are reordered in the file to more logically match declaration order.
  • Most of the VB_VBI logs are reduced to the LOG_DEBUG level.
  • The subtitle creation pipeline is somewhat more amenable to tweaks and performance-related experiments.

Refs #10984

comment:13 Changed 5 years ago by Jim Stichnoth

Resolution: Fixed
Status: newclosed

Closing this as fixed, because I can't see any more to do. Description of experiments and results follows.

Adding timing to the subtitle creation pipeline showed that the dominant stage is the painting of the text chunks, in MythPainter::DrawTextPriv?(). This is invoked for a new string that is not already in the string image cache. This was sped up by 3325c09 when drawing outlines.

If MythPainter::DrawTextPriv?() can't be sped up further, the next approach is to try to spread the work over multiple frames where possible, to minimize stuttering. One approach is to use a background helper thread. However, the helper thread is limited to non-MythUI work, so it's only marginally helpful in reducing the latency. Therefore the helper thread idea was abandoned.

Another approach is to pipeline the work across multiple Pulse() calls in the UI thread. A cost model can be used to throttle the amount of work done each Pulse(). The benefit is that if a subtitle includes multiple chunks, the drawing of the chunks can be spread across time. Unfortunately, this approach led to flashing and other visual artifacts on rapid "real-time" captioning, despite lots of experimentation to try to tune and synchronize the delay timing. As a result, this approach was also abandoned (especially after it didn't improve stuttering on an ION system).

One more approach was tried. It was observed that for these real-time paint-on captions, each caption is usually almost identical to the previous caption, with just 2 characters added to the bottom line. The idea is to try, if possible, to create the same set of chunks as the last caption, with one extra chunk for the (two) new characters, thereby using the string image cache as much as possible. Unfortunately, this also did not actually help in reducing the stuttering.

In the end, the only thing that reliably eliminated the stuttering was to decrease the excessive amount of VB_VBI logging output, particular for cc708 captions.

Note: See TracTickets for help on using tickets.