Opened 8 years ago
Closed 8 years ago
Last modified 8 years ago
#10662 closed Bug Report - General (Upstream Bug)
mythfilldatabase download error
Reported by: | Owned by: | beirdo | |
---|---|---|---|
Priority: | minor | Milestone: | unknown |
Component: | MythTV - Mythfilldatabase | Version: | 0.25-fixes |
Severity: | medium | Keywords: | |
Cc: | Ticket locked: | no |
Description
This is for the recent "schedulesdirect not working" threads on the mailing list. The gist of it is that I get no guide data because mythfilldatabase errors out with a download error. Things work fine if I use xmltv utilities to grab the data from Schedules Direct, then manually load it with "mythfilldatabase --file". Several others on the list are reporting the same issue, although it does seem to be working for most people. I filed a Schedules Direct ticket, and Robert Kulagowski investigated and determined there is nothing wrong with my SD account or with my ability to get the data from Tribune, and he suggested I file a MythTV ticket, so here it is.
I will attach a log of a mythfilldatabase run showing the error.
This started immediately after my upgrade to 0.25, things were working fine before that.
Attachments (2)
Change History (30)
comment:1 Changed 8 years ago by
comment:2 Changed 8 years ago by
Owner: | changed from stuartm to beirdo |
---|---|
Status: | new → assigned |
comment:3 Changed 8 years ago by
Status: | assigned → infoneeded |
---|
Please try the attached patch, and let me know if that solves it. If it doesn't, please generate another log file with it in place. Thanks.
Changed 8 years ago by
Attachment: | downloadmanager.diff added |
---|
comment:4 Changed 8 years ago by
Sorry to say that the patch did not fix it for me, I still get a download error. The new log is at:
http://www.gregandeva.net/mythfilldatabase.20120427152000.23018.log.txt
comment:5 Changed 8 years ago by
Please try again with the same logging as your first run (-v file,network --loglevel debug) The log messages I put in are in file/debug.
comment:6 Changed 8 years ago by
The new log is at:
http://www.gregandeva.net/mythfilldatabase.20120428095501.25294.log.txt
comment:7 Changed 8 years ago by
Ahh. Thanks. I see. OK, in the code that was patched, change the 60 to something much higher (say give it 600 for 10min) and see if it still times out.
I have noticed in the past that the way DataDirect? transfers happen, there is a small burst of data up front, and then you wait for the server to compile all the listings for you, and then it continues the transfer. Seems that in your case, that delay is > 1min. I think once we figure out what the timeout would have to be, we might want to be pushing back on TMS through Schedules Direct to see if that long of a delay is "normal" or not.
comment:9 Changed 8 years ago by
Sorry, and if it works or not, please post a log with -v file,network --loglevel debug
Thanks,
comment:10 Changed 8 years ago by
I changed the 60 (as in the patch) to 600, and it still fails after a 10 minute wait:
http://www.gregandeva.net/mythfilldatabase.20120428125438.15182.log.txt
I'm not certain exactly what all it is waiting for during that time. I ran "time tv_grab_na_dd" on my gateway system. That entire operation takes about 11 minutes, but the download part takes just 170 seconds. If I didn't have an important recording scheduled to start in 15 minutes, I'd try it again with a 20 minute timeout. I'll do that later tonight. I can't install new packages on my backend because it's running Fedora 14 which is no longer supported, so I had to install xmltv elsewhere and copy the file over to do a manual guide update. The gateway system is a slower 32-bit system so I would expect the mythfilldatabase processing to be faster on the actual backend, so I don't expect going to 20 minutes to make a difference, but I'll try it later anyway just to be sure.
comment:11 Changed 8 years ago by
Hmmm, we may need to look at packet dumps from tcpdump to determine what's going on. In particular, as root:
tcpdump -s0 -i any -w /tmp/capture.pcap host 144.142.232.53
Capture the whole failing attempt, and then I'd need to peruse it. Don't worry, your username/password is sent in htdigest form, and is not visible in cleartext.
comment:12 Changed 8 years ago by
Log:
mythfilldatabase.20120429075139.18419.log
Packet capture:
capture.pcap
Both at www.gregandeva.net (can't include full links or the &*#(*@(!@ spam filter rejects me)
comment:13 Changed 8 years ago by
Thank you. That is very useful. It seems that somehow, it is not hitting the authentication callback, or the callback isn't working. This is odd. Please try adding this new patch, we should only need a logfile this time :)
Thanks for your patience, BTW.
Changed 8 years ago by
Attachment: | datadirect-2.diff added |
---|
comment:14 Changed 8 years ago by
This new patch just adds more logging around the callback to see if it is actually hitting.
comment:15 Changed 8 years ago by
Log with datadirect-2 patch is at www.gregandeva.net/mythfilldatabase.20120429120905.15752.log.txt
comment:16 Changed 8 years ago by
What version of Qt are you running?
Also, please verify that your schedule direct username and password are set correctly (don't know why it would have changed in an upgrade, but please verify).
comment:17 Changed 8 years ago by
From your pcap:
Authorization: Digest username="greg", realm="gregandeva.net", nonce="97c2cbdfe2b03f1807970e2cd4ba0696", uri="/schedulesdirect/tvlistings/xtvdService", opaque="2b99186424025b388825c77349e75a2b", response="84f7bc1b9b7f82bc69bb75661dbe0582", qop=auth, nc=00000001, cnonce="ab193f6bb95e3b46bb34453fe339e249"
OK. That is not right. the realm is being changed. I'll assume your username for SchedulesDirect? is in fact "greg", and not "Greg"?
comment:18 Changed 8 years ago by
I am running qt-4.7.1-7.fc14.x86_64
I have verified (by looking in the videosource table) that my SchedulesDirect? userid and password are set correctly (which isn't a surprise given previous discussion in the thread; at least one other person experiencing this problem already tried resetting and verifying his login info several times).
My SD username is actually "greg@…".
comment:21 Changed 8 years ago by
Ooooooh. I think I'm understanding the issue now ;) Now how to fix it....
It seems when you set the user to a user in the form of an email... Qt takes the host part as being the realm, which is stupid. I'll have to go read their code to see how to get around this mess.
comment:22 Changed 8 years ago by
https://bugreports.qt-project.org/browse/QTBUG-15070
This was apparently fixed in Qt4.7.2
comment:24 Changed 8 years ago by
Can we just set the realm after setting the user as a workaround for Qt 4.7.1?
comment:25 Changed 8 years ago by
No. There is not "setRealm" and the setUser() strips off everything from the first @ on. It's a bug in Qt that has been fixed. The way to work around it is to downgrade to Qt 4.7.0 or upgrade to 4.7.2 or 4.8.
Or, you can go edit the code for Qt itself and recompile, but that takes a very very long time.
comment:27 Changed 8 years ago by
Resolution: | → Upstream Bug |
---|---|
Status: | infoneeded → closed |
As discussed on #mythtv there is a workaround that doesn't involve patching Qt.
Go to https://www.schedulesdirect.org/account/edit Change the login name so that it does not have an "@" sign in it. Run mythtv-setup and change the Schedules Direct login name to match the new login name.
As such I'm closing this as an upstream bug.
My attempt to attach the log failed, it was rejected by a spam filter. The log can be viewed at:
http://www.gregandeva.net/mythfilldatabase.20120424133425.30422.log.txt