Opened 6 years ago

Last modified 5 years ago

#11509 assigned Bug Report - General

mythfilldatabase pulls incorrect channel information from Schedules Direct

Reported by: ccoager@… Owned by: sphery
Priority: minor Milestone: unknown
Component: MythTV - General Version: 0.26-fixes
Severity: medium Keywords:
Cc: Ticket locked: no

Description

I was having this issue on version 0.25 and recently upgraded to 0.26.0_p2013022 (Gentoo) in hopes of fixing it, but no luck. I am using the HDHR Prime with Verizon FIOS and Schedules Direct. My issue is I have missing channels, duplicate channels and channels with the wrong channel numbers.

I tried re-adding the video sources after I upgraded. Did a mythfilldatabase and grabbed all the channels from Schedules Direct.

The issue is 100% reproducible.

Attachments (5)

duplicate channel.txt (2.1 KB) - added by ccoager@… 6 years ago.
duplicate channel
incorrect channel.txt (2.5 KB) - added by ccoager@… 6 years ago.
incorrect channel
missing channel.txt (161 bytes) - added by ccoager@… 6 years ago.
missing channel
linueup.html.zip (9.2 KB) - added by ccoager@… 6 years ago.
Schedules Direct Lineup
mythfilldatabase.log (53.4 KB) - added by ccoager@… 6 years ago.
mythfilledatabase debug log

Download all attachments as: .zip

Change History (19)

Changed 6 years ago by ccoager@…

Attachment: duplicate channel.txt added

duplicate channel

Changed 6 years ago by ccoager@…

Attachment: incorrect channel.txt added

incorrect channel

Changed 6 years ago by ccoager@…

Attachment: missing channel.txt added

missing channel

comment:1 Changed 6 years ago by gigem

Milestone: unknown0.27
Resolution: Invalid
Status: newclosed

This sounds like the well known SchedulesDirect? problem that usually requires dropping and re-adding the appropriate lineup at SD. See http://www.mythtv.org/pipermail/mythtv-users/2013-March/347938.html for a similar issue.

comment:2 Changed 6 years ago by ccoager@…

Resolution: Invalid
Status: closednew

I have reviewed the documentation you linked. Unfortunately after following the directions it did not fix my issue. The issue is still 100% reproducible.

comment:3 Changed 6 years ago by sphery

You'll need to provide something that indicates that mythfilldatabase is actually doing something wrong, as mythfilldatabase can't add channels that aren't part of the lineup. If nothing else, a log showing it actually adding the channel (ideally a run with --loglevel debug), and please also include the lineup report from the Schedules Direct website (after logging in, click Report next to your Subscribed Lineups).

Please reproduce the issue and get logs using a database with a clean Video Source (do so by using "Delete all video sources" in mythtv-setup, then re-creating the Video Source, then re-connecting it to a given input). Ideally, you'll do this several hours (or a day) after (once again) trying "Re-Add All Lineups" on the Schedules Direct website.

comment:4 Changed 6 years ago by sphery

Also, please do not cut any lines from the log. Please see http://www.mythtv.org/wiki/Posting_Logs#Getting_Useful_Logs

comment:5 Changed 6 years ago by sphery

Milestone: 0.27
Status: newinfoneeded_new

Changed 6 years ago by ccoager@…

Attachment: linueup.html.zip added

Schedules Direct Lineup

Changed 6 years ago by ccoager@…

Attachment: mythfilldatabase.log added

mythfilledatabase debug log

comment:6 Changed 6 years ago by ccoager@…

I reproduced the issue again. I deleted the video sources and channels from the database using mythtv-setup. I added everything back in and ran a 'mythfilldatabase --do-channel-updates --loglevel debug'.

As an example, Schedules Direct shows channel 506 WRGBDT (WRGB-DT). The mythfilldatabase log shows "DataDirect?: Adding channel 506 'WRGBDT (WRGB-DT)' (WRGBDT)". However, the database shows this:

mysql> select chanid,channum,name from channel where name like '%WRGBDT%'; +--------+---------+--------------------+ | chanid | channum | name | +--------+---------+--------------------+ | 1506 | 6 | WRGBDT (WRGB-DT) | | 1006 | 6 | WRGBDT (WRGB-DT) | | 1468 | 468 | WRGBDT2 (WRGB-DT2) | +--------+---------+--------------------+ 3 rows in set (0.00 sec)

comment:7 Changed 6 years ago by sphery

All 3 of those channels are listed in your Schedules Direct lineup:

Logical Channel Broadcast Channel XMLid Call Sign Name
6 6 34968 WRGBDT WRGBDT (WRGB-DT)
468 6 34969 WRGBDT2 WRGBDT2 (WRGB-DT2)
506 6 34968 WRGBDT WRGBDT (WRGB-DT)

which correspond to the mythfilldatabase log lines:

2013-11-03 12:32:18.011958 I DataDirect?: Adding channel 506 'WRGBDT (WRGB-DT)' (WRGBDT). 2013-11-03 12:32:18.048569 I DataDirect?: Adding channel 6 'WRGBDT (WRGB-DT)' (WRGBDT). 2013-11-03 12:32:18.056551 I DataDirect?: Adding channel 468 'WRGBDT2 (WRGB-DT2)' (WRGBDT2).

meaning that your provider makes the same channel (WRGBT) available with the logical channel number 468 and an "aliased" logical channel number 6. Providers often do this so that customers can use the same "local"/broadcast channel number (that's often mentioned on air) rather than remember a provider-specific channel number to tune the channel.

I'll look to see why it's using the channel number 6 for both of WRGB-DT channels, but if you want only one in your Video Source, you should remove the unwanted one from your Schedules Direct lineup.

Are there any missing channels or are the "duplicate" channels (duplicate channel number for aliased channels) the only issue?

comment:8 Changed 6 years ago by ccoager@…

Channel 6 and 506 are different channels. Channel 6 is standard while 506 is HD. I'm missing channel 506 and cannot access the HD version of this channel. The same is true for WXXA on channel 508.

I can, however, access this channel using the cable box.

comment:9 Changed 6 years ago by sphery

Actually, channel 6 and channel 506 (I'm glad you figured out my typo in my post, where I had said 468) are the same channel in MythTV because:

a) they have the same call sign, which says they should be treated as exactly identical for the purposes of scheduled recordings--meaning you do not care which one is used

b) they have the same channel number (6), which says that they should be treated as exactly identical for the purposes of Live TV--meaning you do not care which one MythTV tunes when you type in 6 in Live TV

c) they have the same call sign /and/ channel number, so they are treated as identical for the purposes of the EPG (guide)--meaning you only want to see the channel once in the guide

To tell MythTV they are different, you will need to use mythtv-setup or MythWeb's or mythfrontend's Channel Editor to (manually) change either the call sign (so you can schedule recordings for one or the other channel, specifically, using the "this channel" filter on recording rules), or the channel number (so you can tell MythTV which one to tune during Live TV), or both. Once you do so, both channels will become available/listed separately in the guide/independently-selectable for use in MythTV.

So, the only thing remaining is to see why MythTV is choosing to use the same channel number for both channels, which I'll look into. I'm guessing it's doing so because the lineup data tells it to, but I'll verify there's no uninitialized data type problem or insufficiently-specific data retrieval problem in our code causing us to reuse one channel's number for the other.

comment:10 Changed 6 years ago by ccoager@…

If I manually make this change in mythtv-setup, will it get overwritten again when I run a 'mythfilldatabase --do-channel-updates'?

comment:11 Changed 6 years ago by sphery

Yes. Running mythfilldatabase --do-channel-updates explicitly tells mythfilldatabase to make unsafe changes to all existing channels (even ones that your provider hasn't recently changed) because the user does not want to bother with updating channels himself. It says, "Treat the information from the listings provider as the definitive channel listing, and forget all modifications I have made to my channels." Unsafe changes include changes to call sign, name, channel number (and ATSC major/minor channel numbers), and frequency(tuning) ID.

If you use --do-channel-updates, you will need to make the same changes each time, because you're saying to use the information from your listings provider and forget all your changes. Alternatively, once you have the majority of your channels defined, edit them manually to reflect any changes made by your provider.

Note, too, that mythfilldatabase will make /safe/ changes to your channels every time it's run. Safe changes include adding an XMLTV ID and setting name and call sign if the channel is in the database but without XMLTV ID, or inserting new channels (that don't exist in your database). So, since providers normally just add new channels to your lineup or shuffle around channels within your lineup, you can take advantage of the safe-update behavior of mythfilldatabase by deleting the xmltvid from (only) channels that are being changed or (use the messier option of) deleting channels and letting mythfilldatabase add the new ones--and doing so never requires using --do-channel-updates.

(Note, also, that mythfilldatabase is designed to fill the database with program listings, not to pick up changes to the channel list. Using it as a tool to do channel maintenance--anything other than initial population of channels for unscannable interfaces--is going beyond its design purposes, and the results of doing so are your problem to deal with. If someone wants a good "compare channels in the database with those in my listings" tool, they can create one--ideally an interactive one--as an external tool, for example, a script using the Python bindings. Making unsafe changes to channels without user interaction (or explicit user permission) is a bad idea, which is why it is only done when the user says to with mythfilldatabase --do-channel-updates. Even if we add a mechanism to detect channel changes in the future, users will be required to approve the changes before they're made, due to the danger of making these unsafe changes automatically.)

comment:12 Changed 6 years ago by ccoager@…

Thank you for the detailed explanation on how that works. I know exactly what I need to change in case I need to run the '--do-channel-updates' in the future which would override my changes.

I am interested in a long term fix for this. If you find it is a Schedules Direct issue, let me know and I will report the issue to them. Thanks!

comment:13 Changed 5 years ago by sphery

Owner: set to sphery
Status: infoneeded_newassigned

comment:14 Changed 5 years ago by Stuart Auchterlonie

Milestone: unknown
Note: See TracTickets for help on using tickets.