Opened 5 months ago

Last modified 5 days ago

#13472 assigned Developer Task

Channel Scan improvements

Reported by: Klaas de Waal Owned by: Klaas de Waal
Priority: minor Milestone: needs_triage
Component: MythTV - Channel Scanner Version: Master Head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

This ticket is intended to document and track non-trivial changes to the Channel Scan of mythtv-setup.

The idea is to add patches to this ticket that can be applied to the latest master. This gives the possibility for feedback and early testing, before the changes are committed to master.

The attached patch, 20190724-keep-channum-ok-all.patch, implements the following features.

1. Add "OK All" button

After a channel scan you can get a pop-up dialog with text like this:

"Channel xxx has channel number 1 but that is already in use. Please enter a unique channel number. Default is 5."

This question is repeated for each channel with a duplicate channel number, requiring a press on the OK button for each. There is now an "OK All" button that answers OK for all channels.

2. Keep existing channel numbers with a rescan

In mythtv-setup the channel number is by default the service ID of that channel, unless there is a LCN Logical Channel Number present. The channel numbers can later be modified with the Channel Editor of mythtv-setup.
However, this leads to conflicts when the video source is scanned again.
On a rescan mythtv-setup now keeps the existing channel numbers.
Previously, the only way known to me to have user-defined channel numbers is to run a SQL script over the channel table after each scan.

3. No more channel conflicts on service ID's

A "conflicting channel" in mythtv-setup is a channel that does not have a unique channel number. It is essential to have a unique channel number for all channels in a video source otherwise mythbackend can arbitrarily select any channel with the same channel number for a recording.
However, mythtv-setup also considers a channel conflicting if the service ID is the same as another channel and then forces the user to enter a unique channel number.
In my understanding this is a bug because A. it does not change the service ID and B. service ID's do not need to be unique.
For DVB the test on service ID is removed and all DVB channels are now considered "non-conflicting" to start with. Note that later on the check on uniqueness of the channel number is still done.

The test case for this is the Hotbird 13.0E satellite. This satellite has a number of different networks. Each network has probably unique service ID's but, over all channels of the satellite, there are many channels with identical service ID's and identical Logical Channel Numbers.
With this patch, mythtv-setup can now scan all channels of the Hotbird 13.0E with a minimum number of key presses by the user.

Attachments (5)

20190724-keep-channum-ok-all.patch (14.1 KB) - added by Klaas de Waal 5 months ago.
Keep channel numbers after resecan; added "OK All" button; remove check on service ID
setup4.log (31.0 KB) - added by jpilk 3 months ago.
Tuning log of DVB-T/T2 system
setup.log (39.4 KB) - added by jpilk 3 months ago.
setup5.log (25.9 KB) - added by jpilk 3 months ago.
Successful scan of DVB-T transports: setup5.log
msetup2.txt (53.9 KB) - added by jpilk 3 months ago.
Scan of all known transports for DVB-T only and DVB-T/T2 tuners at 2dcfeb4

Download all attachments as: .zip

Change History (18)

Changed 5 months ago by Klaas de Waal

Keep channel numbers after resecan; added "OK All" button; remove check on service ID

comment:1 Changed 4 months ago by Klaas de Waal <kdewaal@…>

In 85314f4e9/mythtv:

Channel Scan improvements

Add the following features:

  1. Add "OK All" button for automatic assignment of non-conflicting channel numbers.
  2. Keep existing channel numbers with a rescan.
  3. Allow duplicate service ID's in a single video source.

For a complete description please see ticket 13472.

Refs #13472

comment:2 Changed 4 months ago by Klaas de Waal

In mythtv commit 4636d7e4146a70195b33ba9a565f01e9ed43a521

Check video source and capture card before doing a channel scan

In mythtv-setup page Channel Editor, check that a video source is selected and that the selected video source is connected to a capture card before starting the Channel Scan dialog after pressing the Channel Scan button. Give a popup message when the checks fail.

Previously, if a video source was selected that did not have a capture card connected the Channel Scan page would automatically switch to another video source, one that did have a capture card connected, without warning.

comment:3 Changed 4 months ago by Klaas de Waal <kdewaal@…>

In 1c8a75d3d/mythtv:

Full search for old channels

In mythtv-setup page Channel Scan there is now a new
option "Full search for old channels".

If set, compare all channels in the database with the channels found in
the scan; otherwise only the channels in the same transport are compared.
This option is useful when you want to keep channel data such as
the xmltvid and the icon path when doing a rescan
after channels have been rearranged across transports.

For ATSC the comparison is done only on the major and minor numbers.
For DVB the comparison of channels in the same transport is done
on service id but the comparison of channels in different transports
is done on network id and service id.

Without this option, when channels have moved to another transport
the old channels are classified as "off-air" and can only be deleted.
The moved channels are classified as "New" and they then do not have
the non-scanned data anymore such as the xmltvid and the icon path
that may have been added to the old channel.

The option is currently unchecked by default. If this option proves to
be a succes then this can be changed to checked by default.

Refs #13472

comment:4 Changed 3 months ago by jpilk

Minor observation. I just rescanned my single-transmitter UK Freeview DVB-T-only system to see if anything had changed. It went very smoothly, (not) finding 1 off-air channel, but reported 7 transports while listing only 6 distinct frequencies. On repeat, it found no changed channels and 6 transports.

Scan params:  All types, Unencrypted only, LCN required, Complete data, Full Search
              All known transports, ignore timeout, follow links


MythTV Version : v31-Pre-477ddfdaf5b
MythTV Branch : HEAD -> master

.....
2019-09-04 11:31:56.160298 I  CardUtil[1]: Set delivery system: DVB-T

Off-air channels (1):
qam_64:538000000:Loveworld:270:15000:0: 10270:cnt(pnum:1,channum:1)

2019-09-04 11:32:47.134756 I  ChanImport: Old channels found in same transport: 134
2019-09-04 11:32:47.134766 I  ChanImport: Old channels found in other transport: 0
2019-09-04 11:32:47.134773 I  ChanImport: Old channels not found (off-air): 1

Channel list (134):
qpsk:514000000:Notts TV:7:12323:9018:32844:32780=32780:dvb      10007:cnt(pnum:1,channum:1)
....
qam_64:762000000:QVC Beauty:35:12323:9018:28288:24640=24640:dvb 10035:cnt(pnum:1,channum:1)

Found 7 transports:
Channels: FTA Enc Dec
ATSC        0   0   0
DVB       134   0   0
SCTE        0   0   0
MPEG        0   0   0
NTSC        0
Unique: prog 134 atsc 0 atsc minor 0 channum 134
Max atsc major count: 0

=========
(rescan)
....
qam_64:762000000:QVC Beauty:35:12323:9018:28288:24640=24640:dvb 10035:cnt(pnum:1,channum:1)

Found 6 transports:
Channels: FTA Enc Dec
ATSC        0   0   0
DVB       134   0   0
SCTE        0   0   0
MPEG        0   0   0
NTSC        0
Unique: prog 134 atsc 0 atsc minor 0 channum 134
Max atsc major count: 0

....
qam_64:762000000:QVC Beauty:35:12323:9018:28288:24640=24640:dvb

2019-09-04 11:43:41.151328 I  ChanImport: Found 0 unused transports.
2019-09-04 11:44:03.451605 I  OpenGL2: Deleting OpenGL Resources
2019-09-04 11:44:03.452743 I  OpenGL: Deleting OpenGL Resources
[john@HP_Box ~]$

comment:5 Changed 3 months ago by Klaas de Waal

Questions:

  • Did you delete the off-air channels at the end of the first scan?
  • Is the 538MHz present in the channel list of the first scan?
  • Is the list of 134 channels found in the first scan identical to the 134 channels found in the second scan?

If you still have the complete log then maybe you can add it to the ticket?

N.B. I do appreciate you giving the values of the scan parameters.

comment:6 Changed 3 months ago by jpilk

IIRC I accepted the changes from the first scan and exited mythtv-setup. Then re-entered and scanned again, same params. The frequencies listed were the same in each run, and are those known to the Transport Editor. I think the channel list was unchanged by the second scan, but I didn't check it in detail. No changes were reported. I'm afraid I didn't save the log, and rebooted afterwards, so it has gone.

As you can see from the timestamps it was all quite quick. Obviously I can't create new channels for testing but I will save logs the next time I see anything noteworthy. And thanks!

Changed 3 months ago by jpilk

Attachment: setup4.log added

Tuning log of DVB-T/T2 system

Changed 3 months ago by jpilk

Attachment: setup.log added

comment:7 Changed 3 months ago by jpilk

Log attached from MyGica? 230 tuner under 5.2.11-100.fc29.x86_64 File setup4.log

The DVB-T tuner is having trouble with 2 transports and its logs have ended unexpectedly. Perhaps the rain problem again. The teed displayed logs ended with 'Handling Aborted'. File setup.log

It's also just conceivable that the unexpected exit was caused by a current kdeinit5 closure bug in F29.

Changed 3 months ago by jpilk

Attachment: setup5.log added

Successful scan of DVB-T transports: setup5.log

comment:8 Changed 3 months ago by Klaas de Waal <kdewaal@…>

In 796c4c582/mythtv:

No more channel conflicts on ATSC major/minor numbers.

This change is similar to that for channel conflicts on service ID's for DVB.

Rationale (also the same):
A "conflicting channel" in mythtv-setup is a channel that does not have a unique channel number.
It is essential to have a unique channel number for all channels in a video source otherwise
mythbackend can arbitrarily select any channel with the same channel number for a recording.
However, mythtv-setup also considers a channel conflicting if the ATSC major/minor number
combination is the same as another channel and then forces the user to enter a unique channel number.

In my understanding this is a bug because A. it does not change the ATSC major/minor numbers
and B. ATSC major/minor number combinations do not need to be unique in MythTV.
For ATSC the test on the major/minor number combination uniqueness is removed and all ATSC channels
are now considered "non-conflicting" to start with.
The ATSC major/minor number combination is used to create default channel numbers
but later on the check on the uniqueness of the channel number is still done.

Refs #13472

comment:9 Changed 3 months ago by Klaas de Waal <kdewaal@…>

In 6ab6329ce1/mythtv:

Number of transports in mythtv-setup log output

The number of transports in the summary of the log output
is now always the number of transports found in the scan.
In the processing of the scan results, at some point all
transports of channels in the database that are not found in the scan,
called the off-air channels, are added to the list of transports.
This caused an incorrect number of transports to be presented.
This is now fixed.
Also added a summary of the scan parameters in the log output.
Also added, when option -v chanscan is given, a separate
listing of all transports found, in order of ascending frequency.

Refs #13472

Changed 3 months ago by jpilk

Attachment: msetup2.txt added

Scan of all known transports for DVB-T only and DVB-T/T2 tuners at 2dcfeb4

comment:10 Changed 3 months ago by jpilk

This scan was the second tried after the build, and looks OK. The first one, with wind and rain, again had problems with one DVB-T transport and included 26 lines of CVCT debug info at the start of the DVB-T/T2 section; but those results looked OK too.

comment:11 Changed 4 weeks ago by Klaas de Waal <kdewaal@…>

In be306250bd/mythtv:

Suggested channel number in case of conflicts

When there are after a channel scan duplicate channel numbers
a suggested channel number is presented to the user.
The algorithm to create this number is now slightly changed.
For ATSC the algorithm is not changed.
For DVB the sequence is now:

  • Try the existing channel number
  • Try the service ID
  • If there is a Frequency ID (channel) try frequency ID - service ID
  • Try Service ID - Network ID
  • Try Service ID - Transport ID
  • Find an unused channel number.

This has the following advantages:

  • Duplicate channels are now in the Channel Editor sorting adjacent

to the original channel. This makes it easy to select which one to keep.

  • The unused channel numbers are kept unused as long as possible;

allocating a channel to an unused channel number can cause another
duplicate later in the scanning process.
Note that this problem happens in practice only when scanning
satellites; usually the service ID, which is used as channel number
when there is no Logical Channel Number available, is unique.

Refs #13472

comment:12 Changed 3 weeks ago by Klaas de Waal <klaas@…>

In 952e76461c/mythtv:

Show transport ID and frequency in Channel Options editor

In mythtv-setup, show the transport stream ID and the frequency of the
transport stream in the Channel Editor, page "Channel Options - Common".
Add and extend some help texts in this page.
The transport stream ID or the frequency is needed to find
the transport stream of a channel in the "Edit Transports" page.

Refs #13472

comment:13 Changed 5 days ago by Klaas de Waal <kdewaal@…>

In d7a79b6a5/mythtv:

Mythtv-setup option "Complete scan data required"

Check for all channels the PAT/PMT consistency.
Check only for non-ATSC channels the PAT/PMT/SDT consistency.
This solves the problem that selecting this option caused
all ATSC channels to be filtered out.
This option is now selected by default again.
This reverts commit 1c0d46a.

Refs #13472

Note: See TracTickets for help on using tickets.