refactoring dvb eit scanning and eit cache

[Janne <janne-mythtv@…>]

I just splitted the patch, not tried if every stage for itself is ok but i hope so. for one eitscanner on each backend it should be enough to just apply the first three. The db queries need then some work, but it should work without further changes.

[Janne <janne-mythtv@…>]

patch 05 implements cache pruning its done every 6 hours and removes all entries that have ended at least 6 hours before.

Added event's endtime in UNIX format to the cache signature. Added original_network_id to the key since only the 3-tuple onetworkid, transportid, serviceid identifies a service. The event ids are only for a service unique.

[Janne <janne-mythtv@…>]

corrected small error in previous file

[Janne <janne-mythtv@…>]

Last part fixes active scanning for cards with more than one source.

Scanning completes first all channels one source and switches then to the next. Each card selects the source with the highest priority it supports for scanning. So on a backend with with two source each connected to a different card, both sources are scanned paralell.

[Janne <janne-mythtv@…>]

fix for patch 06

[Janne <janne-mythtv@…>]

fixes patch 03... and adds features

[Janne <janne-mythtv@…>]

all in one patch of my changes to eit scanning

[Janne <janne-mythtv@…>]

Daniel, sorry for the mess with all the patches.

All patches except 05_cache_pruning_and_cleanup.patch applied together should be the same as 20060425_eit_refactoring.patch.

For following features you need to apply:

1. Disable per card active scan: 01_disable_active_eitscan.diff

2. active channel scanner in each backend: 02_one_active_eitscanner_per_backend.diff

3. One persistant EITCache per backend: 03_make_eitcache_persistant.diff

4. Control all cards from active Eitscanner in the Master backend: 04_activate_slave_control.diff and 04_fix_for_activate_slave_control.diff

5. eitcache pruning and refactoring: 05_cache_pruning_and_cleanup.patch and 07_fix_time_for_prune_and_do_not_readd_pruned_entries.patch

6. support for multi source cards in the active eitscanner and cleanup: 06_multi_source_cards_and_cleanup.patch and 06_fix_for_multi_source_cards_and_cleanup.patch added

20060425_eit_refactoring.patch is working on one master and one slave backend for over two hours. Assigning channels to specific backends should work. The slave cache is signifcant smaller since only one half of the transponders get there scanned. The number is even and two cards are scanning.

Daniel, if you want to apply only one feature, please mail my and I'll prepare a single patch.

[danielk]

(In [9812]) Refs #1728. Adds EIT Cache expiration to EIT cache.

This doesn't yet expire the cache on a regular basis. I plan to make the cache global to a backend first.

[Janne <janne-mythtv@…>]

fixes a segfault mythtv-eit branch

[Janne <janne-mythtv@…>]

With the attached patch the mythtv-eit branch works for dvb channels.

[danielk]

(In [9841]) Refs #1035, refs #1728. Fixes a cut-n-paste error in DVBRecorder which caused a segfault in DVB Land (w/ patch from Janne).

[Janne <janne-mythtv@…>]

The eit branch is stable after [9918] for me. I'll start with refactoring the eit scanning based on following ideas posted by danielk: 1/ Master backend has a list of all the channels/services being scanned. 2/ For each transport we have a list of the channels/services which

it provides EIT information, and this list tells us if now/next information is available and if future information is available.

3/ Each recorder has a transport to scan and a list of channels/services

for which it collects EIT and a list of channels/services it should ignore.

4/ If a recorder sees EIT for a channel/service which is not on either

list it requests permission to collect this information from the master backend.

5/ The master backend responds to a request by telling the slave to

either ignore or collect the EIT info. The logic for which to do is simple, if no one else is collecting info on that service then allow collection. If someone else is collecting information, but they only see current/next and we see future then tell the other recorder to ignore that channel/service and give the recorder with better information permission to collect the data. Otherwise, tell the recorder to ignore the data.

6/ Anytime a recorder is given permission to collect EIT info on a

channel/service it loads the cache for that channel/service from the DB. When a recorder ends the scan of a channel/service it saves the cache and then tells the master backend it is done with the channel.

7/ The permission to collect EIT on a channel/service expires after

30 minutes. So if a backend dies the master can assign those channels to some other recorder to scan. We can cap the time spent on any one transport to 25 minutes. If this is a passive scan (during a recording) then the recorder just has to save the info and stop the scan before 25 minutes are up, or request permission to collect the EIT again.

8/ When a recorder stops collecting EIT on a transport it saves the

list of channels/services (and whether we have now/next and/or future data) available on the transport, and then requests a new transport to scan. This allows us to update the lists when cross listing information changes.

The master backend should keep separate sorted lists of the transports with EIT on each source (sorted from channel with the most cross-listing to the least cross-listing). When a recorder needs a new transport to scan we look for first transport on the list which doesn't conflict with an in progress recording. If none can be found we look for the first transport for which we have seen future information, and for which we are only scanning current/next info currently. If we find one of these we tell the recorder with that channel/service to quit scanning that channel/service, and then assign the transport to the requesting recorder. Whenever a transport is assigned we remove it from the list, and whenever the list is empty (or no transport can be found to scan for a requesting recorder), we reload the list. But if a recorder terminates a scan early so that a recording, scheduled or livetv, can be started it should be returned to the master backend and reinserted into the list.

We should also keep track of the last time a list was [re]loaded, and when a recorder needing a transport to scan has more than one source assigned we should prefer the one that was [re]loaded a longer time ago.

There may be other criteria for the master backend assignments. For instance, with DVB-S with a rotor we would probably want to prefer transports on the same satellite as the last channel on that recorder. So I'm not terribly concerned with getting the master backend transport scheduling correct right now. I'm more concerned that we get the protocol right so that only one scanner collects EIT on any one channel/service at a time we collect as much information as possible, with as little communication as possible.

[Janne <janne-mythtv@…>]

eitcache_db.diff implements storing the eitcache in the database. The cache is splitted to a per-channel cache. This allows easy on demand loading and pruning per channel.

[Janne <janne-mythtv@…>]

correct one tiny error in the previous patch

[danielk]

(In [9976]) Refs #1035. Fixes #1728. Applies Janne's EIT Cache 'save to DB' patch.

This allows the EIT version cache to be save to the DB so that it can be shared amoung recorders on several backends.

[danielk]

(In [10238]) Refs #1035. Refs #1728. Refs #1801. Refs #1884. Refs #1615. EIT branch merge.

• Improves the EIT Fixup functions for the UK and Sweden.
• Improves the EIT version cache (lowering memory and CPU requirements).
• Fixes various small memory leaks and lowers memory requirements with earlier deletes.
• Fixes a spurious error message in filldata with EIT only sources.