Opened 10 years ago

Closed 10 years ago

Last modified 7 years ago

#8634 closed defect (fixed)

tmdb.py metadata image download fails

Reported by: mcosta1quake@… Owned by: robertm
Priority: minor Milestone: 0.23-fixes
Component: Plugin - MythVideo Version: 0.23-fixes
Severity: medium Keywords:
Cc: Ticket locked: yes

Description

Cannot download any images from themoviedb.org in the past few days.

2010-07-02 12:19:15.926 Video Data Query: Executing "'/usr/share/mythtv/mythvideo/scripts/Movie/tmdb.py' -l en -D 34106" 2010-07-02 12:19:17.016 Copying 'http://hwcdn.themoviedb.org/posters/2fa/4c21e43b7b9aa130e50002fa/you-can-t-take-it-with-you-original.jpg' -> '/media/sda1/images/34106_coverart.jpg'... 2010-07-02 12:19:17.031 Copying 'http://hwcdn.themoviedb.org/backdrops/245/4c21e4b97b9aa1321e000245/you-can-t-take-it-with-you-original.jpg' -> '/media/sda1/fanart/34106_fanart.jpg'... 2010-07-02 12:19:18.007 Coverart download finished: Tried to write /media/sda1/images/34106_coverart.jpg, but it appears to be an HTML redirect (filesize 231). 1 2010-07-02 12:19:18.844 Fanart download finished: Tried to write /media/sda1/fanart/34106_fanart.jpg, but it appears to be an HTML redirect (filesize 233). 1

It appears that the site themoviedb.org is redirecting the request to a mirror site when the actual image is located. When I type the above address into a browser, it sends me to http://209.197.21.111/x9z7i7v3/cds/backdrops/245/4c21e4b97b9aa1321e000245/you-can-t-take-it-with-you-original.jpg?dopvhost=hwcdn.themoviedb.org&doppl=92d6222edef803d3dea803839bb8bbaddea8032e&dopsig=7931e333b84ff51a6096a12b048480db

Looks like whatever server changes themoviedb.org has done recently has broken the metadata download script.

Mythtv info: MythTV Version : 24158 MythTV Branch : branches/release-0-23-fixes Network Protocol : 56 Library API : 0.23.20100314-1 QT Version : 4.6.2 Options compiled in:

linux debug using_oss using_alsa using_pulse using_jack using_pulseoutput using_backend using_dvb using_firewire using_frontend using_glx_proc_addr_arb using_hdhomerun using_hdpvr using_iptv using_ivtv using_joystick_menu using_libudev using_lirc using_mheg using_opengl_video using_opengl_vsync using_qtdbus using_qtwebkit using_v4l using_x11 using_xrandr using_xv using_xvmc using_xvmc_vld using_xvmcw using_bindings_perl using_bindings_python using_opengl using_vdpau using_ffmpeg_threads using_libavc_5_3 using_live using_mheg

Attachments (2)

mythplugins-mythvideo-8634-handle_301_redirects.patch (1.3 KB) - added by sphery 10 years ago.
Cleaned up version of anonymous's patch that's not corrupted by wiki formatting
localimagedl.diff (681 bytes) - added by robertm 10 years ago.
Maybe this will make it happier?

Download all attachments as: .zip

Change History (29)

comment:1 Changed 10 years ago by MarcT <myrdhn@…>

Same issue on trunk as well. ISOs are local on my frontend, the rest of my video is in a storage group on the backend. Jamu is uneffected but only downloads for the storage group.

2010-07-02 00:12:21.201 Metadata Image Download: http://hwcdn.themoviedb.org/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg ->/media/ISOs/metadata/Covers/19995_coverart.jpg
2010-07-02 00:12:21.260 Tried to write http://hwcdn.themoviedb.org/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg, but it appears to be an HTML redirect (filesize 210).
2010-07-02 00:12:21.261 Metadata Image Download: http://hwcdn.themoviedb.org/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg ->/media/ISOs/metadata/Fanart/19995_fanart.jpg
2010-07-02 00:12:21.319 Tried to write http://hwcdn.themoviedb.org/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg, but it appears to be an HTML redirect (filesize 212).
Please attach all output as a file in bug reports.
MythTV Version   : 25243M
MythTV Branch    : trunk
Network Protocol : 57
Library API      : 0.23.20100630-3
QT Version       : 4.6.2
Options compiled in:
 linux debug using_oss using_alsa using_backend using_frontend using_lirc using_mheg using_opengl_video using_opengl_vsync using_qtdbus using_qtwebkit using_v4l using_x11 using_xrandr using_xv using_bindings_perl using_bindings_python using_opengl using_vdpau using_ffmpeg_threads using_mheg

comment:2 Changed 10 years ago by valdemar@…

But I think the issue here is misplaced. AFAIK tmdb.py doesn't actually downloads the images from themoviedb.org. It merely parses the information for a given movie using the tmdb API:

# '/usr/share/mythtv/mythvideo/scripts/Movie/tmdb.py' -l en -D 285 | grep Coverart
Coverart:http\://hwcdn.themoviedb.org/posters/ecc/4bc90631017a3c57fe001ecc/pirates-
of-the-caribbean-at-world-s-end-original.jpg,http://hwcdn.themoviedb.org/posters
/ec7/4bc9062f017a3c57fe001ec7/pirates-of-the-caribbean-at-world-s-end-
original.jpg,http\://hwcdn.themoviedb.org/posters/ed1/4bc90631017a3c57fe001ed1
/pirates-of-the-caribbean-at-world-s-end-original.jpg

(the http\:... is just to prevent the antispam engine to interpret the link as URL) So I'm not sure what actually downloads the images, i.e. if its mythfrontend or mythbacked, but the error message in mythfrontend.log:

2010-07-02 22:07:31.500 Copying 'http://hwcdn.themoviedb.org/posters
/ecc/4bc90631017a3c57fe001ecc/pirates-of-the-caribbean-at-world-s-end-original.jpg' 
-> 'myth://Coverart@192.168.1.40:6543/285_coverart.jpg'...
2010-07-02 22:07:31.814 Coverart download finished: Tried to write 
myth://Coverart@192.168.1.40:6543/285_coverart.jpg, but it appears to be an HTML 
redirect (filesize 243). 1

... shows two things:

  1. Using for instance wget to download the image using this URL, http://hwcdn.themoviedb.org/posters/ecc/4bc90631017a3c57fe001ecc/pirates-of-the-caribbean-at-world-s-end-original.jpg, you can see that wget is getting a HTTP 301 code. And whatever is downloading the graphics is not able to interprete that HTTP code correctly, and just follow the new link.
  2. Its properly mythbackend which is doing the actual download of the fanart, and is not able to interprete the 301 it gets from themoviedb.org. (And btw its not a HTML redirect ... its a HTTP redirect.)

just my two cents

comment:3 Changed 10 years ago by anonymous

The following diff fixes the issue. I don't know if the reformatting of the http address to remove the :80 is correct. Subject to someone who is better informed reviewing the changelist.

+++ mythvideo/mythvideo/videodlg.cpp (working copy) @@ -137,10 +137,17 @@

public slots:

void InspectHeader?(const QHttpResponseHeader &header) {

  • if (header.statusCode() == 302)
+ if (header.statusCode() == 302
header.statusCode() == 301)

{

  • QString m_redirectUrl = header.value("Location");
  • m_redirectCount++;

+ QString redirectUrl = header.value("Location"); + int iPos = redirectUrl.indexOf(":80"); + if(iPos!=-1) + { + redirectUrl.remove(iPos,3); + } + m_redirectUrl = redirectUrl; + m_redirectCount++;

} else if (header.statusCode() == 404) {

@@ -186,6 +193,7 @@

if (!m_redirectUrl.isEmpty() && m_redirectCount <= 8) {

m_url.setUrl(m_redirectUrl);

+ m_http.setHost(m_url.host());

m_data_buffer.reset(); StartCopy?(); return;

Changed 10 years ago by sphery

Cleaned up version of anonymous's patch that's not corrupted by wiki formatting

comment:4 Changed 10 years ago by jmbox

Which code base is this diff against?

I have the latest SVN trunk and there is no InspectHeader? function in videodlg.cpp or anywhere else.

comment:5 Changed 10 years ago by anonymous

Sorry, about the formatting and the lack of cl I'll try to remember that for future posts. My version is 23889. videodlg.cpp is in svn/mythplugins/mythvideo/mythvideo/videodlg.cpp.

comment:6 Changed 10 years ago by robertm

Owner: changed from Anduin Withers to robertm
Status: newassigned

comment:7 Changed 10 years ago by robertm

(In [25287]) Metadata Classes: Add redirect support, don't know how I neglected to add this in the first place, it was definitely on the list. Refs #8634, and fixes it in trunk. .23-fixes will need a different cure and while I haven't looked closely at the patch on the ticket, it doesn't feel right.

comment:8 Changed 10 years ago by robertm

Status: assignedinfoneeded

.23 users affected by this, I can't see why removing the port should ever be necessary, what happens if one just adds the 301 status code line?

comment:9 Changed 10 years ago by stuartm

(In [25327]) Probable fix for the redirect issues affecting download of artwork in mythvideo in 0.23. Refs #8634

comment:10 Changed 10 years ago by anonymous

Just adding the 301 status code is not enough. m_redirectUrl is defined as a local QString which means that none of the redirects are handled. Fixing that problem, without removing the port. was causing a http 400 error - Bad request. I don't remember what problem the SetHost? fixed.

comment:11 Changed 10 years ago by stuartm

(In [25328]) Fix a stupid typo which prevented url redirection working in mythvideo 0.23 and 0.22. Refs #8634

comment:12 Changed 10 years ago by MarcT <myrdhn@…>

Downloading is still broken on Trunk after the alleged fix.

mythfrontend --version
Please attach all output as a file in bug reports.
MythTV Version   : 25299M
MythTV Branch    : trunk
Network Protocol : 57
Library API      : 0.23.20100708-1
QT Version       : 4.6.2
Options compiled in:
 linux debug using_oss using_alsa using_backend using_frontend using_hdpvr using_lirc using_mheg using_opengl_video using_opengl_vsync using_qtdbus using_qtwebkit using_v4l using_x11 using_xrandr using_xv using_bindings_perl using_bindings_python using_opengl using_vdpau using_ffmpeg_threads using_mheg

My masterbackend is on a different machine than the frontend. The frontend does have a slave backend, if that matters, for capturing via hd-pvr.

2010-07-11 16:36:31.828 MSqlQuery::exec(DBManager1) UPDATE videometadata SET title = 'Avatar', subtitle = '', tagline = 'Enter the World', director = 'James Cameron', plot = 'A paraplegic former Marine finds a new life on the distant planet of Pandora. Only to find himself battling humankind alongside the planet's indigenous Na'vi race in this ambitious digital 3D sci-fi epic from Academy Award-winning Titanic director James Cameron.', rating= 'PG-13', year = '2009', releasedate = '2009-12-18', userrating = '7.8', length = '162', season = '0', episode = '0', filename = '/media/ISOs/ISOs/Movies/AVATAR.ISO', hash = '488f98d755486ca3', trailer = '', showlevel = '1', coverfile = '', screenshot = '', banner = '', fanart = '', inetref = '19995', homepage = 'http://www.themoviedb.org/movie/19995', browse = 'true', watched = 'false', host = '', playcommand = '', childid = '-1', category = '0', processed = 'true'  WHERE intid = '14342'
2010-07-11 16:36:31.829 MSqlQuery::exec(DBManager0) INSERT INTO videometadatagenre (idvideo, idgenre) VALUES ('14342', '109')
2010-07-11 16:36:31.831 MSqlQuery::exec(DBManager0) INSERT INTO videometadatagenre (idvideo, idgenre) VALUES ('14342', '108')
2010-07-11 16:36:31.831 MSqlQuery::exec(DBManager0) INSERT INTO videometadatagenre (idvideo, idgenre) VALUES ('14342', '50')
2010-07-11 16:36:31.833 MSqlQuery::exec(DBManager0) INSERT INTO videometadatagenre (idvideo, idgenre) VALUES ('14342', '110')
2010-07-11 16:36:31.833 MSqlQuery::exec(DBManager0) INSERT INTO videometadatagenre (idvideo, idgenre) VALUES ('14342', '21')
2010-07-11 16:36:31.834 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacountry (idvideo, idcountry) VALUES ('14342', '2')
2010-07-11 16:36:31.835 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacountry (idvideo, idcountry) VALUES ('14342', '1')
2010-07-11 16:36:31.837 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2534')
2010-07-11 16:36:31.839 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '1588')
2010-07-11 16:36:31.840 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2533')
2010-07-11 16:36:31.842 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2532')
2010-07-11 16:36:31.843 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '1654')
2010-07-11 16:36:31.845 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '1866')
2010-07-11 16:36:31.846 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '831')
2010-07-11 16:36:31.848 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2531')
2010-07-11 16:36:31.850 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2815')
2010-07-11 16:36:31.851 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2530')
2010-07-11 16:36:31.853 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '1050')
2010-07-11 16:36:31.854 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '1167')
2010-07-11 16:36:31.858 Warning: container 'mythvideo' is missing child 'banner'
2010-07-11 16:36:31.870 Metadata Image Download: http://hwcdn.themoviedb.org/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg ->/media/ISOs/metadata/Covers/19995_coverart.jpg
2010-07-11 16:36:31.874 MythUIHelper: LoadCacheImage(/media/ISOs/metadata/Covers/26715_coverart.jpg,-media-ISOs-metadata-Covers-26715_coverart.jpg--32x55.png)
2010-07-11 16:36:31.875 MythUIImage(0x7f05c046e310): Load(), spawning thread to load '/media/ISOs/metadata/Covers/26715_coverart.jpg'
2010-07-11 16:36:31.875 MythUIImage(0x7f05c046e310): LoadImage(/media/ISOs/metadata/Covers/26715_coverart.jpg) Object coverart
2010-07-11 16:36:31.875 MythUIHelper: LoadCacheImage(/media/ISOs/metadata/Covers/26715_coverart.jpg,-media-ISOs-metadata-Covers-26715_coverart.jpg--32x55.png)
2010-07-11 16:36:31.876 MythUIImage(0x7f05c046e310): LoadImage Not Found in cache. Loading Directly :/media/ISOs/metadata/Covers/26715_coverart.jpg:
2010-07-11 16:36:31.876 MythUIHelper: LoadScaleImage(/media/ISOs/metadata/Covers/26715_coverart.jpg)
2010-07-11 16:36:31.877 MythUIHelper, Error: LoadScaleImage(/media/ISOs/metadata/Covers/26715_coverart.jpg)Unable to find image file
2010-07-11 16:36:31.877 MythUIImage(0x7f05c046e310): LoadImage Could not load :/media/ISOs/metadata/Covers/26715_coverart.jpg:
2010-07-11 16:36:31.877 MythUIHelper: LoadCacheImage(/media/ISOs/metadata/Covers/20504_coverart.jpg,-media-ISOs-metadata-Covers-20504_coverart.jpg--32x55.png)
2010-07-11 16:36:31.878 MythUIImage(0x4904b60): Load(), loading '/media/ISOs/metadata/Covers/20504_coverart.jpg' in foreground
2010-07-11 16:36:31.878 MythUIImage(0x4904b60): LoadImage(/media/ISOs/metadata/Covers/20504_coverart.jpg) Object coverart
2010-07-11 16:36:31.878 MythUIHelper: LoadCacheImage(/media/ISOs/metadata/Covers/20504_coverart.jpg,-media-ISOs-metadata-Covers-20504_coverart.jpg--32x55.png)
2010-07-11 16:36:31.879 MythUIImage(0x4904b60): LoadImage found in cache :-media-ISOs-metadata-Covers-20504_coverart.jpg--32x55.png: RefCount = 3
2010-07-11 16:36:31.882 DownloadManager: download('http://hwcdn.themoviedb.org/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg', '', 47123424)
2010-07-11 16:36:32.490 DownloadManager: downloadProgress(210, 210) (for reply 50887872)
2010-07-11 16:36:32.490 DownloadManager: downloadProgress: http://hwcdn.themoviedb.org/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg to  is at 210 of 210 bytes downloaded
2010-07-11 16:36:32.829 DownloadManager: downloadFinished(50887872)
2010-07-11 16:36:32.829 DownloadManager: redirectUrl()
2010-07-11 16:36:32.829 DownloadManager: downloadFinished(50887872): Redirect: http://hwcdn.themoviedb.org/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg -> http://69.16.184.117:80/x9z7i7v3/cds/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg?dopvhost=hwcdn.themoviedb.org&doppl=450e5b000934769e096476ce4c74cee39db1c796&dopsig=d536a0f5d58c52f58b1b8c0812fd6a68
2010-07-11 16:36:34.096 DownloadManager: downloadFinished(43670496)
2010-07-11 16:36:34.096 DownloadManager: redirectUrl()
2010-07-11 16:36:34.096 downloadFinished(43670496): COMPLETE: http://69.16.184.117:80/x9z7i7v3/cds/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg?dopvhost=hwcdn.themoviedb.org&doppl=450e5b000934769e096476ce4c74cee39db1c796&dopsig=d536a0f5d58c52f58b1b8c0812fd6a68
2010-07-11 16:36:34.098 Image Download: Error Writing Image to file: /media/ISOs/metadata/Covers/19995_coverart.jpg
2010-07-11 16:36:34.098 Metadata Image Download: http://hwcdn.themoviedb.org/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg ->/media/ISOs/metadata/Fanart/19995_fanart.jpg
2010-07-11 16:36:34.099 DownloadManager: download('http://hwcdn.themoviedb.org/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg', '', 51046592)
2010-07-11 16:36:34.704 DownloadManager: downloadProgress(212, 212) (for reply 50135200)
2010-07-11 16:36:34.704 DownloadManager: downloadProgress: http://hwcdn.themoviedb.org/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg to  is at 212 of 212 bytes downloaded
2010-07-11 16:36:35.029 DownloadManager: downloadFinished(50135200)
2010-07-11 16:36:35.029 DownloadManager: redirectUrl()
2010-07-11 16:36:35.029 DownloadManager: downloadFinished(50135200): Redirect: http://hwcdn.themoviedb.org/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg -> http://69.16.184.201:80/x9z7i7v3/cds/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg?dopvhost=hwcdn.themoviedb.org&doppl=450e5b00093476a1096476f14c74cedf9db1c7a4&dopsig=5dbbbb5af8184e71396d2aaf3b69e20b
2010-07-11 16:36:37.341 DownloadManager: downloadFinished(50766368)
2010-07-11 16:36:37.341 DownloadManager: redirectUrl()
2010-07-11 16:36:37.341 downloadFinished(50766368): COMPLETE: http://69.16.184.201:80/x9z7i7v3/cds/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg?dopvhost=hwcdn.themoviedb.org&doppl=450e5b00093476a1096476f14c74cedf9db1c7a4&dopsig=5dbbbb5af8184e71396d2aaf3b69e20b
2010-07-11 16:36:37.343 Image Download: Error Writing Image to file: /media/ISOs/metadata/Fanart/19995_fanart.jpg

My permisions are set correctly for the mythtv user to write to the metadata folder.

# ls -al /media/ISOs/metadata
total 0
drwxr-xr-x 7 mythtv users 79 Jul  1 23:03 .
drwxr-xr-x 4 mythtv users 32 Jul  1 23:31 ..
drwxr-xr-x 2 mythtv users  6 Jul  1 23:02 Banners
drwxr-xr-x 2 mythtv users  6 Jul 11 16:37 Covers
drwxr-xr-x 2 mythtv users 29 Jul 11 16:36 Fanart
drwxr-xr-x 2 mythtv users  6 Jul  1 23:03 Screenshots
drwxr-xr-x 2 mythtv users  6 Jul  1 23:03 Trailers

Instead of the expected jpeg, we get a 1 byte file written to all the media folders that had downloads.

# ls -al /media/ISOs/metadata/Fanart
total 4
drwxr-xr-x 2 mythtv users  29 Jul 11 16:36 .
drwxr-xr-x 7 mythtv users  79 Jul  1 23:03 ..
-rw-r--r-- 1 mythtv mythtv  4 Jul 11 16:36 19995_fanart.jpg

comment:13 Changed 10 years ago by MarcT <myrdhn@…>

correction 4 byte

comment:14 Changed 10 years ago by robertm

Resolution: fixed
Status: infoneededclosed

Working in current trunk here, even for the above example of Avatar. Problem appears to be a user/configuration issue at this point, especially as the verbose logs show a complete download of the full file and correct following of redirects.

comment:15 Changed 10 years ago by robertm

Resolution: fixed
Status: closednew

Hmm, I'll leave this open pending my testing local files more completely-- but Storage Groups at the minimum work fine.

comment:16 Changed 10 years ago by mdrons@…

I am still seeing the error.

mythbackend --version Please attach all output as a file in bug reports. MythTV Version : 25329M MythTV Branch : branches/release-0-23-fixes Network Protocol : 56 Library API : 0.23.201000617-1 QT Version : 4.6.3

2010-07-12 11:13:38.792 Video Data Query: Executing "'/usr/local/share/mythtv/myt hvideo/scripts/Movie/tmdb.py' -l en -D 22972" 2010-07-12 11:13:39.036 Copying 'http://hwcdn.themoviedb.org/posters/e0d/4bc95eca 017a3c57fe02ae0d/green-zone-original.jpg' -> 'myth://Coverart@10.100.10.200:6543/ 22972_coverart.jpg'... 2010-07-12 11:13:39.061 Copying 'http://hwcdn.themoviedb.org/backdrops/24e/4c268f dc7b9aa1446a00024e/green-zone-original.jpg' -> 'myth://Fanart@10.100.10.200:6543/ 22972_fanart.jpg'... 2010-07-12 11:13:39.103 GetVideoList? returned 1 possible matches 2010-07-12 11:13:39.103 Video Data Query: Executing "'/usr/local/share/mythtv/myt hvideo/scripts/Movie/tmdb.py' -l en -D 22972" 2010-07-12 11:13:39.132 Fanart download finished: Tried to write myth://Fanart@10 .100.10.200:6543/22972_fanart.jpg, but it appears to be an HTML redirect (filesiz e 216). 1 2010-07-12 11:13:39.157 Coverart download finished: Tried to write myth://Coverar t@10.100.10.200:6543/22972_coverart.jpg, but it appears to be an HTML redirect (f ilesize 214). 1 2010-07-12 11:13:39.323 GetVideoList? returned 1 possible matches 2010-07-12 11:13:39.323 Video Data Query: Executing "'/usr/local/share/mythtv/myt hvideo/scripts/Movie/tmdb.py' -l en -D 22972" 2010-07-12 11:13:39.369 Copying 'http://hwcdn.themoviedb.org/posters/e0d/4bc95eca 017a3c57fe02ae0d/green-zone-original.jpg' -> 'myth://Coverart@10.100.10.200:6543/ 22972_coverart.jpg'... 2010-07-12 11:13:39.393 Copying 'http://hwcdn.themoviedb.org/backdrops/24e/4c268f dc7b9aa1446a00024e/green-zone-original.jpg' -> 'myth://Fanart@10.100.10.200:6543/ 22972_fanart.jpg'... 2010-07-12 11:13:39.470 Fanart download finished: Tried to write myth://Fanart@10 .100.10.200:6543/22972_fanart.jpg, but it appears to be an HTML redirect (filesiz e 216). 1 2010-07-12 11:13:39.492 Coverart download finished: Tried to write myth://Coverar t@10.100.10.200:6543/22972_coverart.jpg, but it appears to be an HTML redirect (f ilesize 214). 1 2010-07-12 11:13:39.676 Copying 'http://hwcdn.themoviedb.org/posters/e0d/4bc95eca 017a3c57fe02ae0d/green-zone-original.jpg' -> 'myth://Coverart@10.100.10.200:6543/ 22972_coverart.jpg'... 2010-07-12 11:13:39.701 Copying 'http://hwcdn.themoviedb.org/backdrops/24e/4c268f dc7b9aa1446a00024e/green-zone-original.jpg' -> 'myth://Fanart@10.100.10.200:6543/ 22972_fanart.jpg'... 2010-07-12 11:13:39.810 Coverart download finished: Tried to write myth://Coverar t@10.100.10.200:6543/22972_coverart.jpg, but it appears to be an HTML redirect (f ilesize 214). 1 2010-07-12 11:13:39.831 Fanart download finished: Tried to write myth://Fanart@10 .100.10.200:6543/22972_fanart.jpg, but it appears to be an HTML redirect (filesiz e 216). 1

comment:17 Changed 10 years ago by robertm

Status: newinfoneeded_new

Read carefully: This patch is for *TRUNK* users with local files. I have no idea if it will work but it's worth a test. .23 issue is still pending, I need to have a .23 system to look at it, but we're likely fairly close to the solution on that anyway.

Anyway, Marc or whomever, feel free to test this and see if local images write out properly.

Changed 10 years ago by robertm

Attachment: localimagedl.diff added

Maybe this will make it happier?

comment:18 Changed 10 years ago by mdrons@…

I dont have that file in .23 release. I confused why the fix is only TRUNK. According to the change set 25328, there is a fix for both .22 and .23.

comment:19 Changed 10 years ago by robertm

mdrons, again, the patch above is something I would like *trunk users* with locally hosted video and image files to try. The .23 fix is necessarily different, and being worked in parallel.

comment:20 Changed 10 years ago by MarcT <myrdhn@…>

Robert, no change. Rebuilt both machines, restarted the backends and then the frontends same messages. Same version as before 25299, but I applied your patch to both machines.

2010-07-12 16:56:46.621 Running Grabber: /usr/share/mythtv/metadata/Movie/tmdb.py -l en -D 19995
2010-07-12 16:56:48.645 Returning Metadata Results: Avatar 0 0
2010-07-12 16:56:48.650 MSqlQuery::exec(DBManager1) UPDATE videometadata SET title = 'Avatar', subtitle = '', tagline = 'Enter the World', director = 'James Cameron', plot = 'A paraplegic former Marine finds a new life on the distant planet of Pandora. Only to find himself battling humankind alongside the planet's indigenous Na'vi race in this ambitious digital 3D sci-fi epic from Academy Award-winning Titanic director James Cameron.', rating= 'PG-13', year = '2009', releasedate = '2009-12-18', userrating = '7.8', length = '162', season = '0', episode = '0', filename = '/media/ISOs/ISOs/Movies/AVATAR.ISO', hash = '488f98d755486ca3', trailer = '', showlevel = '1', coverfile = '', screenshot = '', banner = '', fanart = '', inetref = '19995', homepage = 'http://www.themoviedb.org/movie/19995', browse = 'true', watched = 'false', host = '', playcommand = '', childid = '-1', category = '0', processed = 'true'  WHERE intid = '14342'
2010-07-12 16:56:48.651 MSqlQuery::exec(DBManager0) INSERT INTO videometadatagenre (idvideo, idgenre) VALUES ('14342', '109')
2010-07-12 16:56:48.652 MSqlQuery::exec(DBManager0) INSERT INTO videometadatagenre (idvideo, idgenre) VALUES ('14342', '108')
2010-07-12 16:56:48.653 MSqlQuery::exec(DBManager0) INSERT INTO videometadatagenre (idvideo, idgenre) VALUES ('14342', '50')
2010-07-12 16:56:48.654 MSqlQuery::exec(DBManager0) INSERT INTO videometadatagenre (idvideo, idgenre) VALUES ('14342', '110')
2010-07-12 16:56:48.655 MSqlQuery::exec(DBManager0) INSERT INTO videometadatagenre (idvideo, idgenre) VALUES ('14342', '21')
2010-07-12 16:56:48.656 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacountry (idvideo, idcountry) VALUES ('14342', '2')
2010-07-12 16:56:48.657 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacountry (idvideo, idcountry) VALUES ('14342', '1')
2010-07-12 16:56:48.658 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2534')
2010-07-12 16:56:48.660 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '1588')
2010-07-12 16:56:48.661 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2533')
2010-07-12 16:56:48.663 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2532')
2010-07-12 16:56:48.665 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '1654')
2010-07-12 16:56:48.666 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '1866')
2010-07-12 16:56:48.667 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '831')
2010-07-12 16:56:48.669 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2531')
2010-07-12 16:56:48.671 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2815')
2010-07-12 16:56:48.673 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '2530')
2010-07-12 16:56:48.674 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '1050')
2010-07-12 16:56:48.675 MSqlQuery::exec(DBManager0) INSERT INTO videometadatacast (idvideo, idcast) VALUES ('14342', '1167')
2010-07-12 16:56:48.681 Warning: container 'mythvideo' is missing child 'banner'
2010-07-12 16:56:48.695 MSqlQuery::exec(DBManager0) SELECT data FROM settings WHERE value = 'videoartworkdir' AND hostname = 'ionitx' <<<< Returns 1 row(s)
2010-07-12 16:56:48.695 MSqlQuery::next(DBManager0) Result: "data = /media/ISOs/metadata/Covers"
2010-07-12 16:56:48.696 Metadata Image Download: http://hwcdn.themoviedb.org/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg ->/media/ISOs/metadata/Covers/19995_coverart.jpg
2010-07-12 16:56:48.700 MythUIHelper: LoadCacheImage(/media/ISOs/metadata/Covers/26715_coverart.jpg,-media-ISOs-metadata-Covers-26715_coverart.jpg--32x55.png)
2010-07-12 16:56:48.701 MythUIImage(0x7f41c479e040): Load(), spawning thread to load '/media/ISOs/metadata/Covers/26715_coverart.jpg'
2010-07-12 16:56:48.701 MythUIImage(0x7f41c479e040): LoadImage(/media/ISOs/metadata/Covers/26715_coverart.jpg) Object coverart
2010-07-12 16:56:48.701 MythUIHelper: LoadCacheImage(/media/ISOs/metadata/Covers/26715_coverart.jpg,-media-ISOs-metadata-Covers-26715_coverart.jpg--32x55.png)
2010-07-12 16:56:48.702 MythUIImage(0x7f41c479e040): LoadImage Not Found in cache. Loading Directly :/media/ISOs/metadata/Covers/26715_coverart.jpg:
2010-07-12 16:56:48.702 MythUIHelper: LoadScaleImage(/media/ISOs/metadata/Covers/26715_coverart.jpg)
2010-07-12 16:56:48.703 MythUIHelper, Error: LoadScaleImage(/media/ISOs/metadata/Covers/26715_coverart.jpg)Unable to find image file
2010-07-12 16:56:48.703 MythUIHelper: LoadCacheImage(/media/ISOs/metadata/Covers/20504_coverart.jpg,-media-ISOs-metadata-Covers-20504_coverart.jpg--32x55.png)
2010-07-12 16:56:48.703 MythUIImage(0x7f41c479e040): LoadImage Could not load :/media/ISOs/metadata/Covers/26715_coverart.jpg:
2010-07-12 16:56:48.704 MythUIImage(0x2b09720): Load(), loading '/media/ISOs/metadata/Covers/20504_coverart.jpg' in foreground
2010-07-12 16:56:48.704 MythUIImage(0x2b09720): LoadImage(/media/ISOs/metadata/Covers/20504_coverart.jpg) Object coverart
2010-07-12 16:56:48.704 MythUIHelper: LoadCacheImage(/media/ISOs/metadata/Covers/20504_coverart.jpg,-media-ISOs-metadata-Covers-20504_coverart.jpg--32x55.png)
2010-07-12 16:56:48.704 MythUIImage(0x2b09720): LoadImage found in cache :-media-ISOs-metadata-Covers-20504_coverart.jpg--32x55.png: RefCount = 2
2010-07-12 16:56:48.707 DownloadManager: download('http://hwcdn.themoviedb.org/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg', '', 139920310837968)
2010-07-12 16:56:49.367 DownloadManager: downloadProgress(210, 210) (for reply 29430336)
2010-07-12 16:56:49.368 DownloadManager: downloadProgress: http://hwcdn.themoviedb.org/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg to  is at 210 of 210 bytes downloaded
2010-07-12 16:56:49.736 DownloadManager: downloadFinished(29430336)
2010-07-12 16:56:49.736 DownloadManager: redirectUrl()
2010-07-12 16:56:49.736 DownloadManager: downloadFinished(29430336): Redirect: http://hwcdn.themoviedb.org/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg -> http://69.16.184.117:80/x9z7i7v3/cds/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg?dopvhost=hwcdn.themoviedb.org&doppl=450e5b000935d8da0965d88a4c7560a79db069d2&dopsig=2df7fdf6a4286fbd67eabc7245597384
2010-07-12 16:56:51.240 DownloadManager: downloadFinished(139920308545264)
2010-07-12 16:56:51.240 DownloadManager: redirectUrl()
2010-07-12 16:56:51.241 downloadFinished(139920308545264): COMPLETE: http://69.16.184.117:80/x9z7i7v3/cds/posters/a32/4bc95853017a3c57fe027a32/avatar-original.jpg?dopvhost=hwcdn.themoviedb.org&doppl=450e5b000935d8da0965d88a4c7560a79db069d2&dopsig=2df7fdf6a4286fbd67eabc7245597384
2010-07-12 16:56:51.242 Image Download: Error Writing Image to file: /media/ISOs/metadata/Covers/19995_coverart.jpg
2010-07-12 16:56:51.244 MSqlQuery::exec(DBManager1) SELECT data FROM settings WHERE value = 'mythvideo.fanartdir' AND hostname = 'ionitx' <<<< Returns 1 row(s)
2010-07-12 16:56:51.244 MSqlQuery::next(DBManager1) Result: "data = /media/ISOs/metadata/Fanart"
2010-07-12 16:56:51.244 Metadata Image Download: http://hwcdn.themoviedb.org/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg ->/media/ISOs/metadata/Fanart/19995_fanart.jpg
2010-07-12 16:56:51.244 DownloadManager: download('http://hwcdn.themoviedb.org/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg', '', 139920310837968)
2010-07-12 16:56:51.849 DownloadManager: downloadProgress(212, 212) (for reply 139920310527328)
2010-07-12 16:56:51.850 DownloadManager: downloadProgress: http://hwcdn.themoviedb.org/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg to  is at 212 of 212 bytes downloaded
2010-07-12 16:56:52.208 DownloadManager: downloadFinished(139920310527328)
2010-07-12 16:56:52.208 DownloadManager: redirectUrl()
2010-07-12 16:56:52.208 DownloadManager: downloadFinished(139920310527328): Redirect: http://hwcdn.themoviedb.org/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg -> http://69.16.184.201:80/x9z7i7v3/cds/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg?dopvhost=hwcdn.themoviedb.org&doppl=450e5b000935d8dd0965d88d4c7560a39db069d8&dopsig=429efc5b19ac952c4d36d23042ad97e0
2010-07-12 16:56:54.582 DownloadManager: downloadFinished(139920310554624)
2010-07-12 16:56:54.582 DownloadManager: redirectUrl()
2010-07-12 16:56:54.582 downloadFinished(139920310554624): COMPLETE: http://69.16.184.201:80/x9z7i7v3/cds/backdrops/a0e/4bc95848017a3c57fe027a0e/avatar-original.jpg?dopvhost=hwcdn.themoviedb.org&doppl=450e5b000935d8dd0965d88d4c7560a39db069d8&dopsig=429efc5b19ac952c4d36d23042ad97e0
2010-07-12 16:56:54.602 Image Download: Error Writing Image to file: /media/ISOs/metadata/Fanart/19995_fanart.jpg

comment:21 Changed 10 years ago by robertm

(In [25339]) Fix Metadata Image download for all image storage types in trunk. Refs #8634. Thanks to stupidity adjustment of author by Chris Pinkham and Greg Estabrooks.

comment:22 Changed 10 years ago by Tianon Gravi <admwiggin@…>

For release-0-23-fixes, The patches made for the 301 redirect were important, but they weren't a complete fix. The code as it is only handles redirects from one host within itself (which resulted in a lot of 400 Bad Request responses, since it was asking the wrong server for the file). I went ahead and applied the following changes to fix that. Now it should handle any redirect (up to the limit of 8 times) and should also handle alternate port numbers.

  • mythplugins/mythvideo/mythvideo/videodlg.cpp

    diff --git mythplugins/mythvideo/mythvideo/videodlg.cpp mythplugins/mythvideo/mythvideo/videodlg.cpp
    index 6a9be5f..fa56906 100644
    namespace 
    120120      public:
    121121        void StartCopy()
    122122        {
    123             m_id = m_http.get(m_url.toEncoded(), &m_data_buffer);
     123            m_http.setHost(m_url.host(), m_url.port(80));
     124            QByteArray path = m_url.encodedPath();
     125            if (m_url.hasQuery())
     126            {
     127                path.append("?");
     128                path.append(m_url.encodedQuery());
     129            }
     130            m_id = m_http.get(path, &m_data_buffer);
    124131
    125132            m_timer.start(gContext->GetNumSetting("PosterDownloadTimeout", 60)
    126133                          * 1000);
    namespace 
    168175            connect(&m_timer, SIGNAL(timeout()), SLOT(OnDownloadTimeout()));
    169176           
    170177            m_timer.setSingleShot(true);
    171             m_http.setHost(m_url.host());
     178            m_http.setHost(m_url.host(), m_url.port(80));
    172179        }
    173180
    174181        ~ImageDownloadProxy() {}

This on top of the changes made earlier in the repository have successfully fixed the image downloading issues for me.

comment:23 Changed 10 years ago by robertm

Resolution: fixed
Status: infoneeded_newclosed

(In [25356]) Patch from Tianon Gravi to finish redirect fix of image grabbing in .23-fixes. I am not sure I would have accomplished it in exactly the same way, but providing a fix is more urgent than waiting on me to get a VM set up with .23 and decide what I like. Should fix #8634. Not aware of any more outstanding issues with image download in trunk or .23.

comment:24 Changed 9 years ago by stuartm

Milestone: unknown0.23-fixes

comment:25 Changed 9 years ago by Tianon Gravi <admwiggin@…>

After rereading, I feel the need to clarify a little. I'm not nuts about my patch either, but it worked well for what it was: a quick fix to a problem I had. I'm glad it was enough to help someone else, too.

comment:26 Changed 7 years ago by fruityfart@…

I was getting this error:

MetadataImageDownload? metadataimagedownload.cpp:235 (run) Tried to write http://d3gtl9l2a4fn1j.cloudfront.net/t/p/original/rhfRVeUbYJhVVVzMIkhU5EsjKAB.jpg, but it appears to be an HTML redirect or corrupt file (filesize 0).

This was the only page I could google that gave any insight to the problem.

I just wanted to suggest to anyone else having this problem still, that they should check for firewalls and services such as pgl (peer guardian).

I had to stop the pgl service, then it worked fine

comment:27 Changed 7 years ago by Kenni Lund [kenni a kelu dot dk]

Ticket locked: set
Note: See TracTickets for help on using tickets.