Opened 5 months ago

Closed 5 months ago

Last modified 5 months ago

#13623 closed Bug Report - General (fixed)

mythfilldatabase may not process valid XMLTV files if program starttime order is not increasing

Reported by: Gary Buhrmaster Owned by: Bill Meek
Priority: minor Milestone: 31.1
Component: MythTV - Mythfilldatabase Version: Master Head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

mythfilldatabase (with a9aa006139) may no longer process valid XMLTV files if the program starttime order is not increasing.

Originally reported to the XMLTV project, where it was pointed out that the issue is actually due to a MythTV issue. As the original reporter chose not to create a MythTV ticket (s/he found their own workaround), here is what seems to be the essential part of the original issue so that it can be addressed here:

In a9aa006139 a new (to mythfilldatabase) test was added to check that the starttime was increasing with each new program element on the same channel, and reject the input file if it was not.

While many (most?) XMLTV grabbers appear to return the data in starttime order (and for that matter, channel and starttime order), any such order requirement is not part of the specification, and the original reporter ran into this issue with their chosen grabber for a particular day.

The order check should be removed to support compliant source files.

FWIW, as I recall (it has been a long time since I last looked at the code in question), MythTV itself does its own internal sorting of the program data that it processes for its own purposes.

The mitigation until this issue is resolved is to run the output of a grabber through TV_SORT (but note that if there is a lot of source data TV_SORT can be very resource intensive (lots of memory and cpu)).

COMPLETELY untested patch (not compiled, let alone tested, just sort of looks right):

diff --git a/mythtv/programs/mythfilldatabase/xmltvparser.cpp b/mythtv/programs/mythfilldatabase/xmltvparser.cpp
index 78f22170cd..bbce136c9d 100644
--- a/mythtv/programs/mythfilldatabase/xmltvparser.cpp
+++ b/mythtv/programs/mythfilldatabase/xmltvparser.cpp
@@ -205,8 +205,6 @@ bool XMLTVParser::parseFile(
     QString aggregatedTitle;
     QString aggregatedDesc;
     bool haveReadTV = false;
-    QString last_channel = ""; //xmltvId of the last program element we read
-    QDateTime last_starttime; //starttime of the last program element we read
     while (!xml.atEnd() && !xml.hasError() && (! (xml.isEndElement() && xml.name() == "tv")))
     {
         if (xml.readNextStartElement())
@@ -709,23 +707,6 @@ bool XMLTVParser::parseFile(
                 else
                 {
                     // so we have a (relatively) clean program element now, which is good enough to process or to store
-                    if (pginfo->m_channel != last_channel) {
-                        //we have a channel change here
-                        last_channel = pginfo->m_channel;
-                        last_starttime = QDateTime(QDate(1970, 1, 1), QTime(0, 0, 0)); //initialize it to a time far, far away ...
-                    }
-                    else {
-                        //we are still on the same channel
-                        if (pginfo->m_starttime >= last_starttime) {
-                            last_starttime = pginfo->m_starttime;
-                        }
-                        else {
-                            LOG(VB_GENERAL, LOG_ERR, QString("Malformed XML file, program out of order at line %1, %2").arg(xml.lineNumber()).arg(xml.errorString()));
-                            delete pginfo;
-                            return false;
-                        }
-                    }
-
                     if (pginfo->m_clumpidx.isEmpty())
                         (*proglist)[pginfo->m_channel].push_back(*pginfo);
                     else

Change History (4)

comment:1 Changed 5 months ago by Bill Meek

Owner: set to Bill Meek
Status: newaccepted

comment:2 Changed 5 months ago by Bill Meek

Milestone: needs_triage31.1

comment:3 Changed 5 months ago by Gary Buhrmaster <gary.buhrmaster@…>

Resolution: fixed
Status: acceptedclosed

In 1f8b759dd/mythtv:

mythfilldatabase: remove program starttime order check

Closes #13623

Signed-off-by: Bill Meek <billmeek@…>

comment:4 Changed 5 months ago by Gary Buhrmaster <gary.buhrmaster@…>

In fc9048228/mythtv:

mythfilldatabase: remove program starttime order check

Closes #13623

Signed-off-by: Bill Meek <billmeek@…>
(cherry picked from commit 1f8b759dd7b047bde1b5c52f2471c79cd1619e30)

Note: See TracTickets for help on using tickets.