NULL descriptions cause invalid data to be stored when using EIT.

[Kristopher <setnes@…>]

Null descriptions in EIT data will cause MythTV to store descriptions from other shows. It's like a variable isn't being reset when null data is retrieved.

[stuartm]

Please confirm that this problem exists with a more recent version e.g. 0.22-fixes or 0.23 RC2

[setnes@…]

I'm using Gentoo. I recently upgraded MythTV to 0.22_p23069. I believe the issue still exists. I will look closely tonight to confirm it's happening on newly recorded episodes.

I am also using over-the-air program guide information and NOT Schedules Direct. I want to be clear about that.

[setnes@…]

Replying to stuartm:

Please confirm that this problem exists with a more recent version e.g. 0.22-fixes or 0.23 RC2

I actually confirmed it right now. I'm on 0.22 and the problem still exists. I'm not sure how things are getting messed up. I used a converter box to verify the program information. There were many episodes that had NULL descriptions on the converter box. All of those had messed up descriptions in MythTV. Some of the messed up descriptions on MythTV were on episodes that did NOT have NULL descriptions on the converter box though. I get the feeling the descriptions somehow "shift" to other episodes when NULLs occurs.

[Janne Grunau]

Are the wrong descriptions from adjacent programs?

Which channels have the broken descriptions?

Please don't say all but give more details. the usage of the term "converter box" is the only hint that you're located in the USA and probably using ATSC. This does matter since MythTV has different code for different digital TV standards carrying EIT data worldwide.

[setnes@…]

Replying to janne:

Are the wrong descriptions from adjacent programs?

Which channels have the broken descriptions?

Please don't say all but give more details. the usage of the term "converter box" is the only hint that you're located in the USA and probably using ATSC. This does matter since MythTV has different code for different digital TV standards carrying EIT data worldwide.

Yes, I'm very much using ATSC as the source for EIT. I'm in a suburb of Minneapolis, MN. I see this problem on many channels, so it's not a problem with one channel. Perhaps I need to do some database queries to evaluate the data more closely.

These don't appear to be from adjacent programs. For example. "The Late Late Show With Craig Ferguson" airs on WCCO-DT at 11:37PM April 13th and has a description of "When a priest comes forward with a person's confession related to the death of a 9-year-old boy, police reopen the case, and the boy's uncle becomes a prime suspect. (CC)" This is the description of a "Cold Case" episode. "Cold Case" airs on Sundays on WCCO-DT (4.1), but reruns also air on KSTCDT (45.1). Because of this I can't even be sure that the wrong descriptions are coming from shows on the same channel. In this example, the program that comes before Ferguson's show is "Late Show With David Letterman." The program that follows is "Oprah Winfrey." "Cold Case" is not on any of the other channels at the same time as Ferguson's show.

What other information can I provide?

[hwertz]

Actually, I have the same problem. I live in Iowa City, IA, and get 2, 12, 15, 20, 26, 32, and 48 over the air (and 7 and 9 off cable.) 12, 32, and 26 will frequently give me shifted descriptions. 26 has plenty of corruption (a 30 minute show can record 28 minutes long, with no noticeable dropouts, just frequent 1 or 2 frame corruption... this made me think "corrupted program guide data". BUT, 12 and 32 ore both PBS but totally different transmitter sites, and both usually have the same bad EIT data. In both cases, I can fire up mysql, "truncate table program;" and it will often reload the correct data (I don't usually bother since it acts up so frequently.) And then replace it later with incorrect data.

I'll turn on EIT debugging when I am not in the middle of recording something, I've really wanted to capture the EIT directly to see if there's corruption or not. If the EIT truly sent as a table rather than independent "packets" for each show, perhaps the entire EIT has to be accepted (if no corruption) or rejected atomically.

[TheBitPit]

I see this problem in Austin Texas on all the over the air channels. I have line of sight to the antennas about 20 miles away. It existed when I started using MythTV 0.21 and is still in gentoo mythtv-0.22_p23069.

[robertm]

Please read the ticket howto, this is not the place for "me too."

[danielk]

Are you mixing EIT with data from another source?

[danielk]

The only way I can see this happening is if we think that some program information is an update to an existing program info, but it isn't. I'll need you to clear the program table then run "mythbackend -v channel,eit,database --loglevel debug" logs and an indication of a program or two that gets updated incorrectly. Ideally, use a lineup with just one channel known to have this problem and the backend as soon as you've seen it insert one of these problem items, then I'll have less data to sift through.

I'm going to close this as unverified, since I'm not really expecting a reply for such an old ticket. But if I get the log I'll reopen.

Thanks.

[bonnevil@…]

Not the original reporter, seeing the same issue on 0.24.1. EIT over the air is the only data source. Attempting to collect the requested information using just '-v channel,eit,database' since there seems to be no '--loglevel debug' option. Using my current testing configuration, so not just one channel.

I've got a number of issues where things are screwed up now, and about 68 MB of mythbackend.log starting from the table reset. Obvious issue where Channel 4.1 has "Bold and the Beautiful" on 12/15 at 12:30 PM with the same description as "Survivor" on 12/14 at 07:00 PM. Is there some way in which I can reduce the amount of data that needs to be attached to this bug or do some pre-analysis here?

The issue doesn't show up right away on start up; once it shows up data seems to get progressively more scrambled. I suspect there's a variable with bad data being passed to insertions into the program table, haven't gone through the log myself yet to try to sort it out.

[bonnevil@…]

I do have a 10000 line excerpt over a half hour period that includes what looks like two EIT updates, one with a reasonable description and the next with the wrong one. That excerpt is 4 MB in size, still too big to attach. Further review shows that once it goes bad, it stays with the same bad value for each update after that. In the log I have 19 'UPDATE program' entries with the "good" description ('Jackie confesses to Eric; Bill plans a holiday for his family.'), followed by 4 'UPDATE program' entries with the "bad" description ('Then There Were Five: When five original Upolu members turn on each other, things become tense at the camp; two contestants on Redemption Island compete for a spot as the sixth jury member.'), each entry spaced about one half hour apart.

Is there a way to collect what the station is broadcasting directly, rather than depending on the backend to do this, to troubleshoot this and see what the OTA data is?

[mackenziema@…]

I added this code to DBEvent::UpdateDB(

MSqlQuery &query, uint chanid, const DBEvent &match)

...

if (match.description.length() >= ldesc.length()) {

LOG(VB_EIT, LOG_INFO, QString("Setting Description: %1 - %2 %3 - %4").arg(ltitle).arg(ldesc).arg(match.title).arg(match.description)); ldesc = match.description;

} else {

LOG(VB_EIT, LOG_INFO, QString("Description As Is: %1 - %2 %3 - %4").arg(ltitle).arg(ldesc).arg(match.title).arg(match.description));

}

...

After running the backend with -v eit I saw lines like:

2012-10-14 16:42:00.888848 I Setting Description: Brother Bear - A young Native American hunter is magically transformed into a bear while seeking revenge on the bear that took the life of his older brother, but in order to survive life in his new surroundings he must rely on the guidance of a young bear cub. Brother Bear - After learning that a friend's marriage is in trouble, Audrey and Jeff agree to let him stay over as he goes through a difficult divorce, but they soon begin to regret their invitation after they discover his penchant for giving inappropriate hugs.

Since I know that "Brother Bear" isn't about marriage trouble, the 2nd description is most likely inaccurate.

[Michael MacKenzie <mackenziema@…>]

mbe.out is the output of mythbackend -v eit,database,channel --loglevel debug

When I comment out the "if (match.description.length() >= ldesc.length()) { " stanza, the problem goes away, but some programs get no description.

[bonnevil@…]

It seems to me that that's reasonable though -- isn't it possible that there *is* no description transmitted in the EIT/ETT data, so the description field is blank? A/69 sections 6.1, 6.4, and 6.5 seem to state that description text is optional information for the broadcaster to transmit, if I'm understanding this correctly --

​http://www.atsc.org/cms/index.php/standards/standards/53-atsc-a65-standard

​http://www.interactivetvweb.org/tutorials/dtv_intro/atsc_psip/eit

[danielk]

Michael, I don't think your problems are related the problems reported in this ticket.

bonneville, you may be having the same issue. But you'll need to upgrade to 0.26 at least and preferably master before we can debug the issue. If you do please open a new ticket though.