Approach to scanning 'other' frequencies

[mythdev@…]

The current DVB-T scan ignores 'other frequencies' which means that it is broken for some transmiters eg Moel Y Parc - which is a UK DVB-T transmitter

This note is here as "low priority enhancement" because there is an easy workaround - import channels.conf - I'm submitting this to summarise what I have found to date - to help anyone who looks at this in the future.

MoelYParc maintains 6 transport streams reported in the NIT as below: (all transports have the Other_Frequency flag set to 1,

the frequencies marked with are those which carry the signal associated with the transport, those marked with a single * also tune because they are carrying the signal for "other" transports)

tsid: 4158  Centre Frequency 546000.000 (*)
Other Frequencies:
    530166.067
    746000.000
    505833.033
    722000.000
    682166.067
**  738000.000
    530166.067
    706000.000

tsid: 8205   Centre Frequency 578000.000 (*)
Other Frequencies:
    562166.067 kHz)
    825833.033 kHz)
    481833.033 kHz)
    690000.000 kHz)
    714166.067 kHz)
**  770000.000 kHz)
    482166.067 kHz)
*   738166.067 kHz)

tsid: 12289  Centre Frequency 625833.033
Other Frequencies:
    489833.033
    777833.033
    529833.033
    642000.000
    618000.000
**  794000.000
    506166.067
*   770166.067

tsid: 16384  Centre Frequency 705833.033
Other Frequencies:
    513833.033
    801833.033
    561833.033
    666000.000
    641833.033
**  818000.000
    562166.067
*   794166.067

tsid: 20480  Centre Frequency 649833.03
Other Frequencies:
    538166.067
    850000.000
    474000.000
    482166.067
    665833.033
**  546000.000
    538166.067
*   818166.067

tsid: 24576  Centre Frequency 673833.033
Other Frequencies:
    570166.067
    834166.067
    553833.033
    530166.067
    697833.033
**  578000.000
    570166.067
    674000.000

The issue with the current scan against this nit is that it does not look at 'other frequencies' and inserts only the centre-frequencies into dtv_multiplex therefore in the case above the scan will only detect channels for tsid 4158 and 8205. (Which are actually the channels for tsid 20480 and 24576 respectively - but this doesn't matter because we never use the tsid as a crosscheck) An additional consequence is that the transports stored in dtv_multiplex are invalid in that either the frequency doesn't tune or - in all cases the transportid is associated with the wrong frequency.

My reading of the DVB-T specifications suggest that it is not clear whether or not: a) The centre frequency may carry a signal when the 'other_frequency' flag is set b) A single transport may be carried on multiple frequencies from a single transmitter. or c) There is any standardised or dependent 'ordering' of transports/frequencies in the NIT

The sample code in the checknit attached assumes that:

If the other_frequency is set - the centre frequency is not in use for that transport.

A single transport may be carried on multiple frequencies from the same transmitter - so all 'other' frequencies need to be scanned if the 'other_frequency' flag is set.

The sample code checks the 'other frequencies as follows:

Tune to each frequency.

if SYNC bytes are detected then save the frequency, the tsid and signal strength as 'valid'

When the scan is complete:

Sort the saved frequency list in ascending order of frequency and transportid

• the nit above produces the following 'valid frequency' list:

    546000.000 20480
    578000.000 24576
    738000.000 4158
  X 738166.067 8205
    770000.000 8205
  X 770166.067 12289
    794000.000 12289
  X 794166.067 16384
    818000.000 16384
  X 818166.067 20480
  

Then scan through this list, in order, removing any frequency that is within 2 offsets of the previous frequency. - this removes all those above marked with an X.

(At the moment the code doesn't use the signal strength but at some point the strongest signal for a given transport id may need to be chosen?)

This works for the moelyparc transmitter and accidentally? gets the transport id associated with the right frequency. However the approach as coded looks unsatisfactory - eg if 8205 had 793834.000 as a possible 'other frequency' then the 12289 transport would be 'lost'.

I think that what is needed is to ensure that all valid frequencies are captured (those within 2 offsets being equivalent) and to ensure that these have a 'unique' transportid as a key into the multiplex table. Its not clear to me at the moment what sort of convoluted parsing of this list would be necessary to ensure that is true!

The first patch below should be 'safe' it simply decodes the other_frequency flag.

The second patch implements the approach above - it modifies the commented out parts of the existing CheckNIT and provides a 'tuneTransport' function in DVBChannel. The updates to 'wait_for_backend' work on bt878 and ttusb_dec but the comments already in here look like this is a minefield :) other_frequency patch:

Index: mythtv/libs/libmythtv/siparser.cpp
===================================================================
--- mythtv/libs/libmythtv/siparser.cpp	(revision 8325)
+++ mythtv/libs/libmythtv/siparser.cpp	(working copy)
@@ -1743,6 +1743,10 @@
     default:
         retval.TransmissionMode = "auto";
     }    
+
+    //Other_frequency - true if additional frequency is in use
+    retval.Other_frequency =  (( buffer[8] & 0x01 ) == 1);
+
     return retval;
 }
 
Index: mythtv/libs/libmythtv/sitypes.h
===================================================================
--- mythtv/libs/libmythtv/sitypes.h	(revision 8325)
+++ mythtv/libs/libmythtv/sitypes.h	(working copy)
@@ -419,6 +419,8 @@
     QString GuardInterval;
     QString TransmissionMode;
     QString Inversion;
+    // other_frequency  true if one or more of additional frequencies is in use
+    bool Other_frequency;
 
     //Additional frequencies
     QValueList<unsigned> frequencies;
Index: mythtv/libs/libmythtv/sitypes.cpp
===================================================================
--- mythtv/libs/libmythtv/sitypes.cpp	(revision 8325)
+++ mythtv/libs/libmythtv/sitypes.cpp	(working copy)
@@ -344,6 +344,7 @@
     GuardInterval = "auto";
     TransmissionMode = "a";
     Inversion = "a";
+    Other_frequency = false;
 }
 
 void NetworkObject::Reset()

CheckNIT patch:

Index: mythtv/libs/libmythtv/dvbchannel.h
===================================================================
--- mythtv/libs/libmythtv/dvbchannel.h	(revision 8325)
+++ mythtv/libs/libmythtv/dvbchannel.h	(working copy)
@@ -65,6 +65,7 @@
     bool SwitchToInput(const QString &inputname, const QString &chan);
     bool SwitchToInput(int newcapchannel, bool setstarting);
     bool Tune(const dvb_channel_t& channel, bool force_reset=false);
+    uint TuneTransport(const DVBTuning& tuning, uint signalTimeout=5);
 
     // Set/Get/Command just for SIScan/ScanWizardScanner
     void SetMultiplexID(int mplexid)          { currentTID = mplexid; };
Index: mythtv/libs/libmythtv/dvbchannel.cpp
===================================================================
--- mythtv/libs/libmythtv/dvbchannel.cpp	(revision 8325)
+++ mythtv/libs/libmythtv/dvbchannel.cpp	(working copy)
@@ -686,6 +686,59 @@
     return true;
 }
 
+/** \fn DVBChannel::TuneTransport(const DVBTuning&, uint)
+ *  \brief Tunes the card to a frequency but does not deal with PIDs.
+ *
+ *  \param tuning         Transport to tune to
+ *  \param signalTimeout  Maximum time to wait for FE_HAS_SYNC
+ *  \return signal strength on success, 0 on failure
+ */
+uint DVBChannel::TuneTransport(const DVBTuning& tuning,  uint signalTimeout)
+{
+    bool has_diseq = (FE_QPSK == info.type) && diseqc;
+    struct dvb_frontend_parameters params = tuning.params;
+
+    if (fd_frontend < 0)
+    {
+        ERROR("DVBChannel::TuneTransport: Card not open!");
+        return 0;
+    }
+
+    // Remove any events in queue before tuning.
+    drain_dvb_events(fd_frontend);
+
+    // Send DisEq commands to external hardware if we need to.
+    if (has_diseq && !handle_diseq(tuning, diseqc, true))
+    {
+        ERROR("DVBChannel::TuneTransport: Failed to transmit DisEq commands");
+        return 0;
+    }
+
+    CHANNEL("TTOld Params: "<<toString(prev_tuning.params, info.type));
+    CHANNEL("TTNew Params: "<<toString(tuning.params, info.type));
+
+    // Adjust for Satelite recievers which offset the frequency.
+    params.frequency = tuned_frequency(tuning, info.type, NULL);
+
+    if (ioctl(fd_frontend, FE_SET_FRONTEND, &params) < 0)
+    {
+        ERRNO("DVBChannel::TuneTransport: "
+              "Setting Frontend tuning parameters failed.");
+        return 0;
+    }
+    if ( !wait_for_backend( fd_frontend, signalTimeout) )
+        return 0;
+
+    prev_tuning.params = params;
+    CHANNEL("DVBChannel::TuneTransport: Frequency has signal.");
+
+    uint16_t sig = 0;
+    if (ioctl(fd_frontend, FE_READ_SIGNAL_STRENGTH, &sig) < 0 )
+      return 100;
+    else
+      return sig;
+}
+
 /** \fn DVBChannel::GetTuningParams(DVBTuning& tuning) const
  *  \brief Fetches DVBTuning params from driver
  *  \return true on success, false on failure
@@ -815,15 +868,34 @@
  */
 static bool wait_for_backend(int fd, int timeout_ms)
 {
-    struct timeval select_timeout = { 0, (timeout_ms % 1000) * 1000 /*usec*/};
-    fd_set fd_select_set;
-    FD_ZERO(    &fd_select_set);
-    FD_SET (fd, &fd_select_set);
+    struct timeval select_timeout;
+    struct dvb_frontend_event event;
+    bzero(&event, sizeof(struct dvb_frontend_event));
 
-    // Try to wait for some output like an event, unfortunately
-    // this fails on several DVB cards, so we have a timeout.
-    select(fd+1, &fd_select_set, NULL, NULL, &select_timeout);
+    MythTimer t;
+    t.start();
 
+    while (((event.status & FE_TIMEDOUT)==0) &&
+           ((event.status & FE_HAS_SYNC)==0) && (t.elapsed()< timeout_ms)) 
+    {
+        select_timeout.tv_sec = 0; select_timeout.tv_usec = 10000;
+        fd_set fd_select_set;
+        FD_ZERO(    &fd_select_set);
+        FD_SET (fd, &fd_select_set);
+
+        // Try to wait for some output like an event, unfortunately
+        // this fails on several DVB cards, so we have a timeout.
+        if (select(fd+1, &fd_select_set, NULL, NULL, &select_timeout) > 0)
+        {
+            bzero(&event, sizeof(struct dvb_frontend_event));
+            if (ioctl(fd, FE_GET_EVENT, &event) < 0)
+	    {
+                if (errno != EOVERFLOW)
+                    return false;
+            }
+        }
+    }
+
     // This is supposed to work on all cards, post 2.6.12...
     fe_status_t status;
     if (ioctl(fd, FE_READ_STATUS, &status) < 0)
@@ -833,10 +905,13 @@
                 .arg(strerror(errno)));
         return false;
     }
-
     VERBOSE(VB_CHANNEL, QString("dvbchannel.cpp:wait_for_backend: Status: %1")
             .arg(toString(status)));
-    return true;
+
+    if ( status & FE_HAS_SYNC )
+      return true;
+    else
+      return false;
 }
 
 /** \fn handle_diseq(const DVBTuning&, DVBDiSEqC*, bool)
Index: mythtv/libs/libmythtv/siscan.cpp
===================================================================
--- mythtv/libs/libmythtv/siscan.cpp	(revision 8325)
+++ mythtv/libs/libmythtv/siscan.cpp	(working copy)
@@ -707,9 +707,6 @@
     return ok;
 }
 
-#undef SIP_RESET
-#undef SIP_PMAP
-
 void SIScan::ScanTransport(const transport_scan_items_it_t transport)
 {
     QString offset_str = (transport.offset()) ?
@@ -1554,10 +1551,43 @@
 #endif // USE_SIPARSER
 
 #ifdef USE_SIPARSER
+/** \fn SIScan::CheckNIT(NITObject& NIT)
+ *
+ *  Check 'other' frequencies in the NIT transports. The transport.frequency 
+ *  may not be valid - ie one or more of the transport.frequencies may be used 
+ *  by the network so 
+ *
+ *  If the other_frequency flag is set this check should
+ *  a) Tune to each frequency in the frequencies table
+ *  b) If the tuning works - check the transportid of packets received
+ *     if these match - the frequency is valid for the transport
+ *     (The frequency could have tuned because it is valid for a different transport)
+ *
+ * However its not easy to check the transportid in any packets received in this part
+ *  of the code - therefore we just use the first one identified in the transport
+ *
+ */
+class TsFrequency
+{
+public:
+  TsFrequency() : fr(0),tsid(0) {}
+  TsFrequency( unsigned transportid, unsigned frequency, uint signalStrength )
+	  : fr(frequency),tsid(transportid),sstr(signalStrength) {}
+  virtual ~TsFrequency() {}
+  unsigned fr;        // Frequency
+  unsigned tsid;      // Transport stream ID
+  uint sstr;          // Signal Strength
+  bool operator <( const TsFrequency& f1)
+       { return ( ( fr < f1.fr ) ||
+	           ((fr == f1.fr) && (tsid < f1.tsid)) ); }
+};
+
 void SIScan::CheckNIT(NITObject& NIT)
 {
     (void) NIT;
     dvb_channel_t chan_opts;
+    QValueList<TsFrequency> validFrequencies;
+
     QValueList<TransportObject>::Iterator t;
     for (t = NIT.Transport.begin() ; t != NIT.Transport.end() ; ++t )
     {
@@ -1602,30 +1632,61 @@
         else  //Lets hope we never get here
             break;
 
-#if 0 // disable for now
-        //Start off with the main frequency,
-        if (GetDVBChannel()->TuneTransport(chan_opts, true, channelTimeout))
-        {
-//             cerr << "Tuned to main frequency "<< transport.Frequency << "\n";
-            continue;
-        }
+        // If the 'other frequency in use' flag is set then tune to each 
+        // otherwise just check the main frequency
+	//
+	uint sstr = 0;
+	DVBChannel *dvbc = GetDVBChannel();
+        if ( !transport.Other_frequency ) 
+	{
+            SIP_RESET();
+            sstr = dvbc->TuneTransport(chan_opts.tuning, signalTimeout);
+            if ( sstr > 0 )
+                validFrequencies.append(TsFrequency(transport.TransportID,transport.Frequency,sstr));
+	} 
+	else  // other_frequency is in use
+	{	
+            emit TransportScanUpdateText(QString(
+                  "Checking alternative Frequencies for Transport: %1")
+                      .arg(transport.TransportID));
 
-        //Now the other ones in the list
-        QValueList<unsigned>::Iterator f;
-        for (f=transport.frequencies.begin();f!=transport.frequencies.end();f++)
-        {
-            unsigned nfrequency = *f;
-            chan_opts.tuning.params.frequency = nfrequency;
-//            cerr << "CheckNIT " << nfrequency << "\n";
-            if (GetDVBChannel()->TuneTransport(chan_opts, true, channelTimeout))
+            QValueList<unsigned>::Iterator f;
+            for (f=transport.frequencies.begin();f!=transport.frequencies.end();f++)
             {
-//                cerr << "Tuned frequency "<< nfrequency << "\n";
-                transport.Frequency = nfrequency;
-                break;
+                unsigned nfrequency = *f;
+                chan_opts.tuning.params.frequency = nfrequency;
+                SIP_RESET();
+                sstr = dvbc->TuneTransport(chan_opts.tuning, signalTimeout);
+                if ( sstr > 0 )
+                    validFrequencies.append(TsFrequency(transport.TransportID,nfrequency,sstr));
             }
         }
-#endif
     }
+    // sort the valid frequencies in order of frequency, transportid
+    qHeapSort(validFrequencies);
+    // Run through the valid list removing frequencies within 33400Hz of the previous frequency
+    unsigned lastFrequency = 0;
+    QValueListIterator<TsFrequency> it;
+    for ( it = validFrequencies.begin(); it != validFrequencies.end();  )
+    {  
+        if (((*it).fr - lastFrequency ) < 334000 )
+	     it = validFrequencies.remove(it);
+        else
+	 {   lastFrequency = (*it).fr; ++it; }
+
+    }
+
+    // finally update the transports with the first valid frequency
+    //   if signal strength was being reported we could pick the strongest here
+    for (t = NIT.Transport.begin() ; t != NIT.Transport.end() ; ++t )
+    {
+        for ( it = validFrequencies.begin(); it != validFrequencies.end(); ++it )
+        {
+        if ( (*t).TransportID == (*it).tsid )
+	    { (*t).Frequency = (*it).fr; break; }
+	}
+    }
+   
 }
 #endif // USE_SIPARSER
 

[mythdev@…]

oops - misassigned, sorry

[danielk]

This appears to be fixed in SVN head thanks to Stuart A.'s efforts.