Opened 7 years ago

Last modified 7 years ago

#11437 new Bug Report - General

mythmetadatalookup not using proxy

Reported by: brian@… Owned by: sphery
Priority: minor Milestone: unknown
Component: MythTV - Mythmetadatalookup Version: 0.25-fixes
Severity: medium Keywords:
Cc: Ticket locked: no

Description (last modified by Raymond Wagner)

Using 0.25.3-fixes from 20130104 at 010e576 I'm finding that even though mythmetadatalookup's environment has an "http_proxy=" set, mythmetadatalookup is is ignoring that and still trying to fetch from the Internet directly.

Change History (7)

comment:1 Changed 7 years ago by Raymond Wagner

Status: newinfoneeded_new
Version: 0.23-fixes0.25-fixes

'mythmetadatalookup' does not access the internet in any capacity. It calls external grabber scripts which in turn access the internet. Which grabber script in particular are you referring to?

comment:2 in reply to:  1 Changed 7 years ago by brian@…

Replying to wagnerrp:

'mythmetadatalookup' does not access the internet in any capacity. It calls external grabber scripts which in turn access the internet. Which grabber script in particular are you referring to?

That's not what strace is telling me here. I've followed the cloning of process/threads up the tree from the process that is making the direct connections. i.e.

connect(29, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "2400:cb00:2048:1::be5d:fc6e", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28 <unfinished ...>

The process tree leading down to 24472 from the initial process is as follows:

32679 <... clone resumed> child_stack=0xae2f83e4, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|C+++ LONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0xae2f8ba8, {entry_number:6, base_addr:0xae2f8b40, limit+++ :1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xae2f8ba+++ 8) = 313
\_313   <... clone resumed> child_stack=0xadaf73e4, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|C+++ LONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0xadaf7ba8, {entry_number:6, base_addr:0xadaf7b40, limit+++ :1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xadaf7ba+++ 8) = 314
  \_314   <... clone resumed> child_stack=0xacaf53e4, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|C+++ LONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0xacaf5ba8, {entry_number:6, base_addr:0xacaf5b40, limit+++ :1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xacaf5ba+++ 8) = 320
    \_320   connect(29, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "2400:cb00:2048:1::be5d:fc6e", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28 <unfinished ...>
      320   <... connect resumed> )           = 0
      ...
      320   write(29, "GET /banners/seasonswide/79349-6"..., 264 <unfinished ...>
      320   <... write resumed> )             = 264

None of those intervening processes invokes an execve() so they presumably are threads. One of those threads very much appears to be doing a direct grab.

comment:3 Changed 7 years ago by Raymond Wagner

Component: MythTV - MythmetadatalookupContributed Scripts & Apps
Description: modified (diff)
Owner: set to sphery
Status: infoneeded_newnew
Summary: mythmetadatalookup not using proxythetvdb.py not using proxy

comment:4 Changed 7 years ago by Raymond Wagner

Status: newinfoneeded_new

Can anyone else who uses a proxy confirm this? The way I'm reading this, proxies and the http_proxy attribute should be handled automatically by the Python HTTP libraries.

http://code.mythtv.org/cgit/mythtv/tree/mythtv/bindings/python/MythTV/ttvdb/tvdb_api.py#n364 http://docs.python.org/2/library/urllib2.html#urllib2.build_opener http://docs.python.org/2/library/urllib2.html#urllib2.ProxyHandler

Brian. The ttvdb.py script is called by mythmetadatalookup, which is run by mythbackend through the JobQueue?. Are you certain mythbackend is being run with the http_proxy attribute in its environment?

comment:5 Changed 7 years ago by Raymond Wagner

Component: Contributed Scripts & AppsMythTV - Mythmetadatalookup
Description: modified (diff)
Status: infoneeded_newnew
Summary: thetvdb.py not using proxymythmetadatalookup not using proxy

Just to clarify, this actually is MML, not the grabber script, and is caused by the image downloads after the grabber script provides the URLs to them.

comment:6 in reply to:  4 Changed 7 years ago by brian@…

Replying to wagnerrp:

Can anyone else who uses a proxy confirm this? The way I'm reading this, proxies and the http_proxy attribute should be handled automatically by the Python HTTP libraries.

Yes, that is my understanding also.

Brian. The ttvdb.py script is called by mythmetadatalookup, which is run by mythbackend through the JobQueue?. Are you certain mythbackend is being run with the http_proxy attribute in its environment?

In my case (which is the case I filed this bug for) I am running from the shell prompt with:

$ env | grep -i proxy
http_proxy=http://proxy:3128/
$ mythmetadatalookup --refresh-all-artwork

I'm happy to carry out any tests to confirm/deny that ttvdb.py is the culprit since I'm not entirely convinced that that tool is the culprit.

comment:7 Changed 7 years ago by brian@…

Anything I can do to progress this issue?

Note: See TracTickets for help on using tickets.