Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#13010 closed Patch - Bug Fix (fixed)

Patch: Improve fromXMLTVDate processing for mythfilldatabase

Reported by: Gary Buhrmaster <gary.buhrmaster@…> Owned by: stuartm
Priority: minor Milestone: unknown
Component: MythTV - Mythfilldatabase Version: Master Head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

Improve fromXMLTVDate processing for mythfilldatabase

The mythfilldatabase xmltv datetime processing is interestingly challenged when converting some datetimes. It will attempt to convert the provided initial part of the datetimes using QDateTime which uses the BE's local timezone. If the local timezone has such things as daylight savings time, and invalid date/time combinations in the DST transition range are provided, the result in (many many) messages of the form:

   CoreContext xmltvparser.cpp:168 (fromXMLTVDate) - Ignoring unknown timestamp format: 20170312020000

In the US (in those locations where DST are observed), there is no 2am on March 12th 2017 (the DST cut over time), as 2am magically turns into 3am, so the failed QDateTime conversion is correct, this is an invalid local datetime. But the actual provided start is "20170312020000 +0000", but in the current code, the initial conversion is done in localtime.

The included patch changes the date/time processing to avoid local timezone issues. It will break up the xmltv datetimes into date part, the time part, and zone part, and convert separately, and combine and adjust at the end.

  • Note to the dev reviewing the patch:
  1. I believe stuartm was the last to attack the XMLTV date processing. Should someone other then he be the reviewer, touching base with stuartm might be appropriate.
  2. Unfortunately (for code review by a dev), git will create the patch matching on brackets and bare returns, so the patch can be a bit confusing. I recommend just reviewing the resulting code.
  3. With QT5.2, the (mis)confusion that resulted in the needed patch for Mythtv ticket #11538 (adjusting for UTC from Localtime) has been addressed upstream. Since MythTV now requires Qt 5.2, that code fragment fixup was eliminated in this revised code.
    QT related info:
      Task-number: QTBUG-26161
      Task-number: QTBUG-29666
      Change-Id: If3cc7fc9778ca2b831644408ae749448d5975a3a
    

github ref: https://github.com/garybuhrmaster/mythtv/commit/7a3652ccad4a952b6579aba4545cb80375220499

github git-am ref: https://github.com/garybuhrmaster/mythtv/commit/7a3652ccad4a952b6579aba4545cb80375220499.patch

Change History (7)

comment:1 Changed 2 years ago by Peter Bennett

I see my mythfilldatabase log is full of these errors. However I do not see any gaps in the schedule on mythweb at 1 AM or 3 AM Sunday morning, which I would have expected if inserts were failing.

I will install this and #13011 locally and see how it goes. There is a very short window for testing these in the live environment, after Sunday the problem will not happen again for a year.

comment:2 Changed 2 years ago by Peter Bennett

Looking at the schedules again, I see in mythweb the 9PM Saturday shows are missing off the schedule, and the prior shows (e.g. 8PM shows) are shown as being an extra hour in length.

comment:3 in reply to:  2 Changed 2 years ago by Gary Buhrmaster <gary.buhrmaster@…>

Replying to pbennett:

Looking at the schedules again, I see in mythweb the 9PM Saturday shows are missing off the schedule, and the prior shows (e.g. 8PM shows) are shown as being an extra hour in length.

mythfilldatabase (and the libraries) attempts to adjust/fill-in/move programs as what appears to be needed to make the schedules accurate. If the source data is poor/bad, the results can be "interesting". GIGO.

comment:4 Changed 2 years ago by Peter Bennett

I have tested this fix and #13011 on master and it works perfectly. No error messages from mythfilldatabase and 9PM shows on Saturday are fine.

comment:5 Changed 2 years ago by Peter Bennett

FYI the patch fails on fixes/0.28 unless you first cherry-pick changeset:"55eb77fd"

comment:6 Changed 2 years ago by Gary Buhrmaster <gary.buhrmaster@…>

Resolution: fixed
Status: newclosed

In 841073ea8c1f49e858f5bb6c18e3c0ade04eb512/mythtv:

Improve fromXMLTVDate processing for mythfilldatabase

In cases where the system is running in localtime, the
current processing will apply timezone validation to
the input datetime, which can result in rejection of valid
datetimes.

Fixes #13010

Signed-off-by: Peter Bennett <pbennett@…>

comment:7 Changed 2 years ago by Gary Buhrmaster <gary.buhrmaster@…>

In 12232f081c672165f67819de8cd9f437c991a082/mythtv:

Improve fromXMLTVDate processing for mythfilldatabase

In cases where the system is running in localtime, the
current processing will apply timezone validation to
the input datetime, which can result in rejection of valid
datetimes.

Fixes #13010

Signed-off-by: Peter Bennett <pbennett@…>
(cherry picked from commit 841073ea8c1f49e858f5bb6c18e3c0ade04eb512)

Note: See TracTickets for help on using tickets.