Opened 13 years ago

Closed 13 years ago

Last modified 12 years ago

#2438 closed defect (fixed)

Infinite loop in mythbackend

Reported by: ian@… Owned by: cpinkham
Priority: minor Milestone: unknown
Component: mythtv Version: 0.20
Severity: high Keywords: CPU load
Cc: jonas@… Ticket locked: no

Description

With mythbackend from 0.20+fixes svn branch, I see the backend going into a state where it is using 100% of CPU. I compiled with profiling and used oprofile, which shows

Profiling through timer interrupt samples % linenr info image name app name symbol name 15006 96.2725 videooutbase.cpp:513 libmythtv-0.20.so.0.20.0 mythbackend VideoOutput::GetVisibleOSDBounds(float&, float&) const

This is with the backend doing nothing apparent at all. ie, not watching live tv, not recording, not commercial flagging. This is weird, because I don't know why the backend would be spending time in GetVisibleOSDBounds at all.

If I stop and restart the backend, it settles back to normal, then something unidentified triggers it to go into this tight loop.

Attachments (5)

mythbackend.backtrace (13.3 KB) - added by anonymous 13 years ago.
Backtrace (but not very helpful it seems)
bug2348.patch (430 bytes) - added by cpinkham 13 years ago.
mythbackend-10062006.backtrace.txt (9.2 KB) - added by anonymous 13 years ago.
opannotate-10122006.txt (107.2 KB) - added by thetaz 13 years ago.
Very high uPnp usage shown in opannotate summary
mythbackend.gdb.txt.gz (4.7 KB) - added by adeffs.mythtv@… 13 years ago.
my backend trace

Download all attachments as: .zip

Change History (26)

comment:1 Changed 13 years ago by ian@…

I tried to add an opannotate listing as an attached file, but got an Internal Error:

Akismet rejected spam

comment:2 Changed 13 years ago by skamithi

Priority: majorminor

comment:3 in reply to:  description Changed 13 years ago by halovanic@…

I can confirm the exact same problem here. I've recompiled from source several different times and tried the binary packages, but all have the same problem, and I can't find anything in the logs about it. I was wondering if there was something peculiar in my hardware or database that was causing this since nobody else had noticed it.

Changed 13 years ago by anonymous

Attachment: mythbackend.backtrace added

Backtrace (but not very helpful it seems)

comment:4 Changed 13 years ago by JonasPed <jonas@…>

Cc: jonas@… added

Can confirm same thing on gentoo with ebuild mythtv-0.20_p11244. Would be really nice to have fixed in 0.20 fixes.

Changed 13 years ago by cpinkham

Attachment: bug2348.patch added

comment:5 Changed 13 years ago by cpinkham

Please try the attached patch and see if it fixes the issues.

comment:6 Changed 13 years ago by ian@…

It seems to work great so far, though I haven't tested it much, my mythbackend is loafing along at 2-3% again.

comment:7 in reply to:  description Changed 13 years ago by halovanic@…

The patch hasn't fixed the problem for me, CPU usage is still hitting 50% for mythbackend after running overnight.

comment:8 Changed 13 years ago by cpinkham

If this is still happening for you, please submit a backtrace of mythbackend from when the problem is occurring. Follow the instructions in the HOWTO for information on how to get a backtrace when the program isn't actually crashing.

comment:9 Changed 13 years ago by cpinkham

(In [11299]) Carries over [11298] from trunk.

Don't try to reinit the OSD if we're using_null_videoout in NVP (ie, transcoding or commercial flagging or building the preview pixmap).

References #2438.

comment:10 Changed 13 years ago by Isaac Richards

Resolution: fixed
Status: newclosed

Original issue should be fixed. Please reopen with backtrace and/or profiling info if not.

comment:11 in reply to:  description Changed 13 years ago by halovanic@…

Sorry, I think I failed to clean out the old version properly the first time. After reinstalling, mythbackend has taken everything I can throw at it.

comment:12 Changed 13 years ago by thetaz

Resolution: fixed
Status: closedreopened

I am still seeing high cpu load on my mythbackend running the latest .20-fixes branch from tuesday. The only way to get rid of the high cpu load is to reboot the backend process. I have been unable to determine when this begins but it almost seems to be random now after the above fix had been applied. I have compiled profiling in, however I am unfamiliar wtih oprofile and I need to know which oprofile commands will produce the most relevant results as to why the back end is looping.

From what I can tell so far, my backend is entering some of the QT header files. I do not run X server on that system and do not have a display attached, should that be happening? It is strictly mythbackend only.

comment:13 Changed 13 years ago by thetaz

I don't know how applicable this is, but here is the backtrace from the running mythbackend process while it is using high cpu load: (gdb) backtrace #0 0xffffe410 in kernel_vsyscall () #1 0xb5e830c1 in select () from /lib/tls/i686/cmov/libc.so.6 #2 0xb642aa8d in QEventLoop::processEvents () from /usr/lib/libqt-mt.so.3 #3 0xb649e947 in QEventLoop::enterLoop () from /usr/lib/libqt-mt.so.3 #4 0xb649e86a in QEventLoop::exec () from /usr/lib/libqt-mt.so.3 #5 0xb6484965 in QApplication::exec () from /usr/lib/libqt-mt.so.3 #6 0x08084a07 in main (argc=6, argv=0xfffffdfe) at main.cpp:682

Changed 13 years ago by anonymous

comment:14 Changed 13 years ago by cpinkham

Owner: changed from Isaac Richards to cpinkham
Status: reopenednew

comment:15 Changed 13 years ago by thetaz

Any updates to this? I will be more than happy to supply any additional information. I have recompiled a couple times with the latest SVN revision's and nothing has improved. The backend still goes into a 100% cpu usage cycle randomly.

comment:16 Changed 13 years ago by cpinkham

I replied to the comment in email, but I guess you don't read -dev. Do you use the new UPNP feature or do you have any UPNP devices on your network? Both backtraces in this ticket show UPNP code being executed at the time so I'm wondering if this is related.

comment:17 Changed 13 years ago by thetaz

Sorry, I don't read -dev at all, but yes..

I have a linksys WRT54G router with custom firmware that has uPnp turned on. This is the only device that has it turned on when this is happening. It seems like this is happening when I do not have any other computers or devices turned on that are connected to my WRT54G switch. Ie, I will have mythbackend at 0% cpu usage when going to sleep, shut down all devices/systems then wake up in the morning at mythbackend is cycling at 99.9% cpu. Mythbackend will still allow me to connect so it seems like it is a thread. Therefore, it may be possible that the router is sending a packet out to the network that is triggering the uPnp on my backend. The branch I'm using is .20-fixes but I see that the other branch has a --noupnp option so I will compile that and run it with that option to see if my results improve.

I also ran opannotate after I had left oprofile running for a couple of days and allowed the backend to loop for over a day at 100% cpu, I will attach a copy of the summary output to this ticket but it looks like it is in fact using a lot of cycles in the uPnp feature.

Thanks

Changed 13 years ago by thetaz

Attachment: opannotate-10122006.txt added

Very high uPnp usage shown in opannotate summary

Changed 13 years ago by adeffs.mythtv@…

Attachment: mythbackend.gdb.txt.gz added

my backend trace

comment:18 Changed 13 years ago by thetaz

Update: I've ran the backend for 2 days now using Thursday afternoon's SVN of the unstable .20 branch with the --noupnp flag and have not had any further issues with the backend going into a 100% cpu loop. I'm seeing consistently average 0% cpu usage by the backend, just as was the case before I upgraded into the .20 release of MythTV.

I am willing to work with development to fix the loop in the uPnP code if a possible fix has not already been researched.

Thanks

comment:19 Changed 13 years ago by cpinkham

(In [11533]) Carries over [11461] from trunk.

Under a loose definition of the word "fix", I'm applying the --noupnp patch to -fixes since it allows people with the "mythbackend is consuming 100% cpu" issue to actually use mythbackend right now. The real fix is inside the upnp code somewhere, but this is a stop-gap measure for now.

References #2438.

comment:20 Changed 13 years ago by cpinkham

Resolution: fixed
Status: newclosed

I believe this was fixed a while back and the ticket was left open.

comment:21 Changed 12 years ago by anonymous

sadly, it's not fixed using the latest svn... :(

Note: See TracTickets for help on using tickets.