mythfilldatabase pulls incorrect channel information from Schedules Direct

[ccoager@…]

I was having this issue on version 0.25 and recently upgraded to 0.26.0_p2013022 (Gentoo) in hopes of fixing it, but no luck. I am using the HDHR Prime with Verizon FIOS and Schedules Direct. My issue is I have missing channels, duplicate channels and channels with the wrong channel numbers.

I tried re-adding the video sources after I upgraded. Did a mythfilldatabase and grabbed all the channels from Schedules Direct.

The issue is 100% reproducible.

[ccoager@…]

duplicate channel

[ccoager@…]

incorrect channel

[ccoager@…]

missing channel

[gigem]

This sounds like the well known SchedulesDirect? problem that usually requires dropping and re-adding the appropriate lineup at SD. See http://www.mythtv.org/pipermail/mythtv-users/2013-March/347938.html for a similar issue.

[ccoager@…]

I have reviewed the documentation you linked. Unfortunately after following the directions it did not fix my issue. The issue is still 100% reproducible.

[sphery]

You'll need to provide something that indicates that mythfilldatabase is actually doing something wrong, as mythfilldatabase can't add channels that aren't part of the lineup. If nothing else, a log showing it actually adding the channel (ideally a run with --loglevel debug), and please also include the lineup report from the Schedules Direct website (after logging in, click Report next to your Subscribed Lineups).

Please reproduce the issue and get logs using a database with a clean Video Source (do so by using "Delete all video sources" in mythtv-setup, then re-creating the Video Source, then re-connecting it to a given input). Ideally, you'll do this several hours (or a day) after (once again) trying "Re-Add All Lineups" on the Schedules Direct website.

[sphery]

Also, please do not cut any lines from the log. Please see http://www.mythtv.org/wiki/Posting_Logs#Getting_Useful_Logs

[ccoager@…]

Schedules Direct Lineup

[ccoager@…]

mythfilledatabase debug log

[ccoager@…]

I reproduced the issue again. I deleted the video sources and channels from the database using mythtv-setup. I added everything back in and ran a 'mythfilldatabase --do-channel-updates --loglevel debug'.

As an example, Schedules Direct shows channel 506 WRGBDT (WRGB-DT). The mythfilldatabase log shows "DataDirect?: Adding channel 506 'WRGBDT (WRGB-DT)' (WRGBDT)". However, the database shows this:

mysql> select chanid,channum,name from channel where name like '%WRGBDT%'; +--------+---------+--------------------+ | chanid | channum | name | +--------+---------+--------------------+ | 1506 | 6 | WRGBDT (WRGB-DT) | | 1006 | 6 | WRGBDT (WRGB-DT) | | 1468 | 468 | WRGBDT2 (WRGB-DT2) | +--------+---------+--------------------+ 3 rows in set (0.00 sec)

[sphery]

All 3 of those channels are listed in your Schedules Direct lineup:

Logical Channel	Broadcast Channel	XMLid	Call Sign	Name
6	6	34968	WRGBDT	WRGBDT (WRGB-DT)
468	6	34969	WRGBDT2	WRGBDT2 (WRGB-DT2)
506	6	34968	WRGBDT	WRGBDT (WRGB-DT)

which correspond to the mythfilldatabase log lines:

2013-11-03 12:32:18.011958 I DataDirect?: Adding channel 506 'WRGBDT (WRGB-DT)' (WRGBDT). 2013-11-03 12:32:18.048569 I DataDirect?: Adding channel 6 'WRGBDT (WRGB-DT)' (WRGBDT). 2013-11-03 12:32:18.056551 I DataDirect?: Adding channel 468 'WRGBDT2 (WRGB-DT2)' (WRGBDT2).

meaning that your provider makes the same channel (WRGBT) available with the logical channel number 468 and an "aliased" logical channel number 6. Providers often do this so that customers can use the same "local"/broadcast channel number (that's often mentioned on air) rather than remember a provider-specific channel number to tune the channel.

I'll look to see why it's using the channel number 6 for both of WRGB-DT channels, but if you want only one in your Video Source, you should remove the unwanted one from your Schedules Direct lineup.

Are there any missing channels or are the "duplicate" channels (duplicate channel number for aliased channels) the only issue?

[ccoager@…]

Channel 6 and 506 are different channels. Channel 6 is standard while 506 is HD. I'm missing channel 506 and cannot access the HD version of this channel. The same is true for WXXA on channel 508.

I can, however, access this channel using the cable box.

[sphery]

Actually, channel 6 and channel 506 (I'm glad you figured out my typo in my post, where I had said 468) are the same channel in MythTV because:

a) they have the same call sign, which says they should be treated as exactly identical for the purposes of scheduled recordings--meaning you do not care which one is used

b) they have the same channel number (6), which says that they should be treated as exactly identical for the purposes of Live TV--meaning you do not care which one MythTV tunes when you type in 6 in Live TV

c) they have the same call sign /and/ channel number, so they are treated as identical for the purposes of the EPG (guide)--meaning you only want to see the channel once in the guide

To tell MythTV they are different, you will need to use mythtv-setup or MythWeb's or mythfrontend's Channel Editor to (manually) change either the call sign (so you can schedule recordings for one or the other channel, specifically, using the "this channel" filter on recording rules), or the channel number (so you can tell MythTV which one to tune during Live TV), or both. Once you do so, both channels will become available/listed separately in the guide/independently-selectable for use in MythTV.

So, the only thing remaining is to see why MythTV is choosing to use the same channel number for both channels, which I'll look into. I'm guessing it's doing so because the lineup data tells it to, but I'll verify there's no uninitialized data type problem or insufficiently-specific data retrieval problem in our code causing us to reuse one channel's number for the other.

[ccoager@…]

If I manually make this change in mythtv-setup, will it get overwritten again when I run a 'mythfilldatabase --do-channel-updates'?

[sphery]

Yes. Running mythfilldatabase --do-channel-updates explicitly tells mythfilldatabase to make unsafe changes to all existing channels (even ones that your provider hasn't recently changed) because the user does not want to bother with updating channels himself. It says, "Treat the information from the listings provider as the definitive channel listing, and forget all modifications I have made to my channels." Unsafe changes include changes to call sign, name, channel number (and ATSC major/minor channel numbers), and frequency(tuning) ID.

If you use --do-channel-updates, you will need to make the same changes each time, because you're saying to use the information from your listings provider and forget all your changes. Alternatively, once you have the majority of your channels defined, edit them manually to reflect any changes made by your provider.

Note, too, that mythfilldatabase will make /safe/ changes to your channels every time it's run. Safe changes include adding an XMLTV ID and setting name and call sign if the channel is in the database but without XMLTV ID, or inserting new channels (that don't exist in your database). So, since providers normally just add new channels to your lineup or shuffle around channels within your lineup, you can take advantage of the safe-update behavior of mythfilldatabase by deleting the xmltvid from (only) channels that are being changed or (use the messier option of) deleting channels and letting mythfilldatabase add the new ones--and doing so never requires using --do-channel-updates.

(Note, also, that mythfilldatabase is designed to fill the database with program listings, not to pick up changes to the channel list. Using it as a tool to do channel maintenance--anything other than initial population of channels for unscannable interfaces--is going beyond its design purposes, and the results of doing so are your problem to deal with. If someone wants a good "compare channels in the database with those in my listings" tool, they can create one--ideally an interactive one--as an external tool, for example, a script using the Python bindings. Making unsafe changes to channels without user interaction (or explicit user permission) is a bad idea, which is why it is only done when the user says to with mythfilldatabase --do-channel-updates. Even if we add a mechanism to detect channel changes in the future, users will be required to approve the changes before they're made, due to the danger of making these unsafe changes automatically.)

[ccoager@…]

Thank you for the detailed explanation on how that works. I know exactly what I need to change in case I need to run the '--do-channel-updates' in the future which would override my changes.

I am interested in a long term fix for this. If you find it is a Schedules Direct issue, let me know and I will report the issue to them. Thanks!

[Gary Buhrmaster]

This ticket should likely be closed. First, it references the use of the legacy dd grabber with mythfilldatabase (now deleted from the code base), second the user explicitly choose to force the system to do what it did, rather than letting it do the safe thing (so be careful what you ask, you may get it). And, as mentioned, external tools could be (and have been) created (as documented in the wiki) to do what the op had originally tried to accomplish in a safer manner (as Mr. Dean suggested seven years ago).