Doubled EIA-608 letters in digital channels

[Alex Halovanic <halovanic@…>]

SD digital channels captured from Wide Open West (WOW) contain EIA-608 captions. On MythTV 0.25 (a2a32e605e08501f19e7e6ea052ecf9dcfdf6c50) turning on closed captions frequently (but not always) results in doubled letters in words, ie "SNONOWW" for "SNOW". Attached is a clip where the captions appear to display correctly briefly before this problem starts. CCExtractor pulls captions fine from this clip. This issue also occurred in 0.24.

[Alex Halovanic <halovanic@…>]

CCExtractor output from clip

[Alex Halovanic <halovanic@…>]

Clip at ​https://rapidshare.com/files/2916660324/12131_20110605190527.mpg

[gregorio.gervasio@…]

This seems to be a case where the stream has caption codes repeated in different packets with the same timestamp. I have attached a fix you can try out. The fix is a little ugly because this case needs to be distinguished from the valid case of multiple caption codes in the same packet. Maybe there's a better way to do this.

The patch also fixes a long-standing problem with digital streams that contain multiple backspace codes. (The decoder incorrectly suppressed all but one.)

[gregorio.gervasio@…]

Fix handling of duplicate CC608 codes.

[markk]

Passing on to Daniel, who knows far more about this than me:)

[Alex Halovanic <halovanic@…>]

Thanks for looking at this. The patch definitely fixes this particular clip. Unfortunately, I was able to capture some more that still have issues. In one case, the patch has definitely improved the caption rendering, as it's only doubled, while it looks like it was tripled or quadrupled previously. In the other, it's rendering as almost (but not quite) complete gibberish.

[Alex Halovanic <halovanic@…>]

CCExtractor output from NikToons? clip

[Alex Halovanic <halovanic@…>]

CCExtractor output from FX clip

[Alex Halovanic <halovanic@…>]

Niktoons clip (more doubled letters): ​https://rapidshare.com/files/1429176165/3233_20110606180600.mpg

FX clip (bad characters): ​https://rapidshare.com/files/3782247537/3194_20110606181315.mpg

[Alex Halovanic <halovanic@…>]

I forgot to mention the bad characters on the one clip showed up prior to applying the patch too.

[gregorio.gervasio@…]

Ah, I misunderstood the problem. It looks like the problem with the duplicate codes is that the stream has both ATSC and SCTE-20 captions, but the ffmpeg decoder stuffs them both into the same structure.

The problem with the bad characters is that the decoder does not know how to handle SCTE-20 captions for repeated fields (film mode).

I have attached a new patch that seems to fix both problems. Please try it out. (You'll need to revert the previous patch.)

[gregorio.gervasio@…]

Fix handling of duplicate CC608 codes. (v2)

[gregorio.gervasio@…]

Improved XDS check for SCTE field inversion. (Apply on top of v2 patch).

[Alex Halovanic <halovanic@…>]

Just tested. These two patches work great, I haven't been able to trigger any issues with regular or ATSC captioning. I'll try on some more channels tonight.

[Alex Halovanic <halovanic@…>]

Done a lot more testing; these work on all the digital and analog recordings I have, including very old ones from different providers.

[Github]

Refs #9829. Handle duplicate SCTE and ATSC captions with patch from Alex Halovanic.

The primary problem is that some streams have both SCTE and ATSC captions, so we need to capture them seperately and then decide which to use.

A secondary problem is inversion and repeats in 608 captions, esp SCTE.

Branch: master Changeset: d526385aa4c1466b064e9544cdb663f357007e35

[Github]

Fixes #9829. Move the CC 608 parity check up so we don't make bad decisions about handling the next field based on bogus data.

Branch: master Changeset: 408c173adfc0cb21ac5f7f4334c8d05d6f143255