Opened 4 weeks ago

Closed 3 weeks ago

Last modified 3 weeks ago

#13510 closed Bug Report - General (fixed)

Several TV metadata retrievals are failing

Reported by: steveerl Owned by: mspieth
Priority: minor Milestone: 31.0
Component: MythTV - Mythmetadatalookup Version: Master Head
Severity: medium Keywords: metadata
Cc: Ticket locked: no

Description

Metadata retrieval for a bunch of my recording rules failed, starting on November 13, 2019. These recording types have previously worked fine. The affected titles were:

Chicago Med, Chicago Fire, Chicago P.D., Riverdale, Nancy Drew (2019), Full Frontal with Samantha Bee

To debug, I ran, by hand, the metadata retrieval commands which are invoked by mythtv.

terra[66] /usr/share/mythtv/metadata/Television/ttvdb.py -l en -a US -M "Full Frontal With Samantha Bee"
<?xml version='1.0' encoding='UTF-8'?>
<metadata>
  <item>
    <language>en</language>
    <title>Full Frontal with Samantha Bee</title>
    <inetref>303438</inetref>
    <collectionref>303438</collectionref>
    <description>Applying her smart and satirical point of view to current and relevant issues, Samantha Bee provides her nuanced perspective on world events, repartees with newsmakers from across ideological spectrums and stabs injustice with the hot poker of comedy.</description>
    <releasedate>2015-5-5</releasedate>
    <images>
      <image type="banner" url="http://www.thetvdb.com/banners//banners/posters/303438-4.jpg" thumb="http://www.thetvdb.com/banners/_cache//banners/posters/303438-4.jpg"/>
    </images>
  </item>
</metadata>

As you can see above, the initial lookup based on the recording Title works fine. From it, we pull out the inetref number 303438 which is used in the next invocation.

terra[67] /usr/share/mythtv/metadata/Television/ttvdb.py -l en -a US -C 303438
! Error: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/usr/share/mythtv/metadata/Television/ttvdb.py", line 2674, in <module>
    sys.exit(main())
  File "/usr/share/mythtv/metadata/Television/ttvdb.py", line 2462, in main
    t._getShowData(series_season_ep[0], opts.language)
  File "/usr/local/lib/python2.7/dist-packages/MythTV/ttvdb/tvdb_api.py", line 1070, in _getShowData
    self._parseBanners(sid)
  File "/usr/local/lib/python2.7/dist-packages/MythTV/ttvdb/tvdb_api.py", line 961, in _parseBanners
    for banner_info in banners_info:
TypeError: 'NoneType' object is not iterable
Exit 1

Here, we see that line 961 in tvdb_api.py is the location of the failure.

 959         for cur_banner in bannersEt.keys():
 960             banners_info = self._getetsrc(self.config['url_seriesBannerInfo'] % (sid, cur_banner))
 961             for banner_info in banners_info:

I added a print statement to see what the value of cur_banner is when these failures happen. For each of the Titles listed above, the failure happened when cur_banner = 'seasonwide'.

I suspect that there has been some change implemented in the database from which the metadata is retrieved to eliminate 'seasonwide' banner information for some TV shows.

The fix is to add one line to verify that banners_info is not None, and to indent the block which follows.

959         for cur_banner in bannersEt.keys():
960             banners_info = self._getetsrc(self.config['url_seriesBannerInfo'] % (sid, cur_banner))
961             if banners_info is not None:
962                 for banner_info in banners_info:

I tested this, and found that metadata retrieval for all these recordings work, once again.

I'm running MythTv? version v31-Pre-487-gf3c90896bf. Earlier releases may be impacted by the same issue.

Attachments (2)

mythtv-13510-Several_TV_metadata_retrievals_are_failing.patch (3.2 KB) - added by steveerl 4 weeks ago.
Fix for metadata retrieval failure
mythtv-13510-Several_TV_metadata_retrievals_are_failing-2.patch (1.6 KB) - added by steveerl 3 weeks ago.
Making tvdb_api.py more fault tolerant

Download all attachments as: .zip

Change History (10)

Changed 4 weeks ago by steveerl

Fix for metadata retrieval failure

comment:1 Changed 4 weeks ago by Stuart Auchterlonie

Milestone: needs_triage31.0
Owner: changed from JYA to Raymond Wagner
Status: newassigned

comment:2 Changed 4 weeks ago by Stuart Auchterlonie

Owner: changed from Raymond Wagner to Bill Meek

comment:3 Changed 4 weeks ago by jhoyt4

It looks like thetvdb updated their API to v3 https://forums.thetvdb.com/viewtopic.php?f=3&t=60048

comment:4 Changed 3 weeks ago by steveerl

My suggested fix was almost exactly implemented independently by the solution to ticket #13511, which is now merged in. It looks like my patch indented one too many lines, and the patch by mspieth indented one too few. When I tested the #13511 solution, I ran into failures due to banners[btype][raw] being set to None.

Due to the thetvdb transition in process while I was testing, I also ran into failures when Series Info data came back empty. I've added a check to make the script more fault tolerant in such a scenario.

Changed 3 weeks ago by steveerl

Making tvdb_api.py more fault tolerant

comment:5 Changed 3 weeks ago by mspieth

Thanks! Your changes have been merged. 1 minor change is that show data is set always so may be an empty dict. I though this was better than no data.

comment:6 Changed 3 weeks ago by steveerl

I approve of your minor change. Thanks for merging my updated patch.

comment:7 Changed 3 weeks ago by Mark Spieth <mspieth@…>

Resolution: fixed
Status: assignedclosed

In 18bc87279a/mythtv:

Improve robustness of banner parsing

Based on a patch by steveerl
fixes #13510

comment:8 Changed 3 weeks ago by Stuart Auchterlonie

Owner: changed from Bill Meek to mspieth
Note: See TracTickets for help on using tickets.