Doubled EIA-608 letters in digital channels

[Alex Halovanic <halovanic@…>]

SD digital channels captured from Wide Open West (WOW) contain EIA-608 captions. On MythTV 0.25 (a2a32e605e08501f19e7e6ea052ecf9dcfdf6c50) turning on closed captions frequently (but not always) results in doubled letters in words, ie "SNONOWW" for "SNOW". Attached is a clip where the captions appear to display correctly briefly before this problem starts. CCExtractor pulls captions fine from this clip. This issue also occurred in 0.24.

[Alex Halovanic <halovanic@…>]

CCExtractor output from clip

[Alex Halovanic <halovanic@…>]

Clip at ​https://rapidshare.com/files/2916660324/12131_20110605190527.mpg

[gregorio.gervasio@…]

This seems to be a case where the stream has caption codes repeated in different packets with the same timestamp. I have attached a fix you can try out. The fix is a little ugly because this case needs to be distinguished from the valid case of multiple caption codes in the same packet. Maybe there's a better way to do this.

The patch also fixes a long-standing problem with digital streams that contain multiple backspace codes. (The decoder incorrectly suppressed all but one.)

[gregorio.gervasio@…]

Fix handling of duplicate CC608 codes.

[markk]

Passing on to Daniel, who knows far more about this than me:)

[Alex Halovanic <halovanic@…>]

Thanks for looking at this. The patch definitely fixes this particular clip. Unfortunately, I was able to capture some more that still have issues. In one case, the patch has definitely improved the caption rendering, as it's only doubled, while it looks like it was tripled or quadrupled previously. In the other, it's rendering as almost (but not quite) complete gibberish.

[Alex Halovanic <halovanic@…>]

CCExtractor output from NikToons? clip

[Alex Halovanic <halovanic@…>]

CCExtractor output from FX clip

[Alex Halovanic <halovanic@…>]

Niktoons clip (more doubled letters): ​https://rapidshare.com/files/1429176165/3233_20110606180600.mpg

FX clip (bad characters): ​https://rapidshare.com/files/3782247537/3194_20110606181315.mpg

[Alex Halovanic <halovanic@…>]

I forgot to mention the bad characters on the one clip showed up prior to applying the patch too.

[gregorio.gervasio@…]

Ah, I misunderstood the problem. It looks like the problem with the duplicate codes is that the stream has both ATSC and SCTE-20 captions, but the ffmpeg decoder stuffs them both into the same structure.

The problem with the bad characters is that the decoder does not know how to handle SCTE-20 captions for repeated fields (film mode).

I have attached a new patch that seems to fix both problems. Please try it out. (You'll need to revert the previous patch.)

[gregorio.gervasio@…]

Fix handling of duplicate CC608 codes. (v2)

[gregorio.gervasio@…]

Improved XDS check for SCTE field inversion. (Apply on top of v2 patch).

[Alex Halovanic <halovanic@…>]

Just tested. These two patches work great, I haven't been able to trigger any issues with regular or ATSC captioning. I'll try on some more channels tonight.

[Alex Halovanic <halovanic@…>]

Done a lot more testing; these work on all the digital and analog recordings I have, including very old ones from different providers.

[Github]

Refs #9829. Handle duplicate SCTE and ATSC captions with patch from Alex Halovanic.

The primary problem is that some streams have both SCTE and ATSC captions, so we need to capture them seperately and then decide which to use.

A secondary problem is inversion and repeats in 608 captions, esp SCTE.

Branch: master Changeset: d526385aa4c1466b064e9544cdb663f357007e35

[Github]

Fixes #9829. Move the CC 608 parity check up so we don't make bad decisions about handling the next field based on bogus data.

Branch: master Changeset: 408c173adfc0cb21ac5f7f4334c8d05d6f143255

[Jim Stichnoth <jstichnoth@…>]

In 68bd03ffe5b7bb6456519639caac77486ea0941c/mythtv:

Subtitles: Allow transitions between ATSC and SCTE captions.

In changeset d526385aa4c1466b064e9544cdb663f357007e35 (refs #9829),
code was added such that the first time ATSC caption data is seen, any
future SCTE caption data is explicitly ignored. This was to deal with
broadcasts containing duplicate caption data.

This policy causes problems when the recording switches back and forth
between the two sources of caption data, such as when the broadcaster
splices in commercials.

The updated solution continues to favor ATSC data, but allows a switch
back to SCTE after SCTE data is seen 10 times in a row (a somewhat
arbitrarily chosen number) without intervening ATSC data.

Refs #12054.

[Jim Stichnoth <jstichnoth@…>]

In bcdaa8865ef8700ccbb7214ec39dcbcd8392b2af/mythtv:

Subtitles: Allow transitions between ATSC and SCTE captions.

In changeset d526385aa4c1466b064e9544cdb663f357007e35 (refs #9829),
code was added such that the first time ATSC caption data is seen, any
future SCTE caption data is explicitly ignored. This was to deal with
broadcasts containing duplicate caption data.

This policy causes problems when the recording switches back and forth
between the two sources of caption data, such as when the broadcaster
splices in commercials.

The updated solution continues to favor ATSC data, but allows a switch
back to SCTE after SCTE data is seen 10 times in a row (a somewhat
arbitrarily chosen number) without intervening ATSC data.

Refs #12054.

(cherry picked from commit 68bd03ffe5b7bb6456519639caac77486ea0941c)