Opened 7 years ago

Closed 6 years ago

#10702 closed Bug Report - Hang/Deadlock (Fixed)

mythbackend hanging

Reported by: anders.franzen@… Owned by: danielk
Priority: major Milestone: 0.25.2
Component: MythTV - General Version: Master Head
Severity: medium Keywords: backend hanging
Cc: Ticket locked: no

Description (last modified by beirdo)

Just start the mythbackend and let it sitt for awhile, it will lookup completly. Also after doing exiting a livetv view, some resources dont get released, so running out of tv-cards/tuners very quick.

Connect to port 6543 dont answers any commands:

socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(6543), sin_addr=inet_addr("192.168.1.67")}, 16) = 0
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"35      ", 8}, {"MYTH_PROTO_VERSION 74 SingingPot"..., 35}], msg_controllen=0, msg_flags=0}, 0) = 43
recvfrom(3, 

Hanging in recv forever.

backtrace and thread info from backend:

(gdb) bt
#0  0x000000390700b3b4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000003a6d672fe3 in ?? () from /usr/lib64/libQtCore.so.4
#2  0x0000003a6d66f9f5 in QMutex::lock() () from /usr/lib64/libQtCore.so.4
#3  0x00000000004a5d44 in QMutexLocker (this=0x27097f0, request=...) at /usr/include/QtCore/qmutex.h:102
#4  Scheduler::Reschedule (this=0x27097f0, request=...) at scheduler.cpp:1634
#5  0x0000000000473986 in MainServer::customEvent (this=0x2869fb0, e=0x7f8c6c017d60) at mainserver.cpp:1009
#6  0x0000003a6d77162c in QObject::event(QEvent*) () from /usr/lib64/libQtCore.so.4
#7  0x0000003a6d75d0dc in QCoreApplication::notifyInternal(QObject*, QEvent*) () from /usr/lib64/libQtCore.so.4
#8  0x0000003a6d7608c5 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /usr/lib64/libQtCore.so.4
#9  0x0000003a6d788133 in ?? () from /usr/lib64/libQtCore.so.4
#10 0x00000033cd841e33 in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
#11 0x00000033cd842610 in ?? () from /lib64/libglib-2.0.so.0
#12 0x00000033cd8428ad in g_main_context_iteration () from /lib64/libglib-2.0.so.0
#13 0x0000003a6d7882cf in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQtCore.so.4
#14 0x0000003a6d75c472 in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQtCore.so.4
#15 0x0000003a6d75c6bc in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/libQtCore.so.4
#16 0x0000003a6d760b7b in QCoreApplication::exec() () from /usr/lib64/libQtCore.so.4
#17 0x000000000050ae75 in run_backend (cmdline=<value optimized out>) at main_helpers.cpp:658
#18 0x000000000044d4e3 in main (argc=8, argv=<value optimized out>) at main.cpp:127
(gdb) 

(gdb) info threads
  31 Thread 0x7f8c8f26e700 (LWP 7192)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  30 Thread 0x7f8c8ea6d700 (LWP 7193)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  29 Thread 0x7f8c8e26c700 (LWP 7195)  0x0000003906cd9093 in select () from /lib64/libc.so.6
  28 Thread 0x7f8c8da6b700 (LWP 7196)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  27 Thread 0x7f8c8ccb8700 (LWP 7204)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  26 Thread 0x7f8c77fff700 (LWP 7206)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  25 Thread 0x7f8c777fe700 (LWP 7210)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  24 Thread 0x7f8c76ffd700 (LWP 7211)  0x000000390700b3b4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  23 Thread 0x7f8c767fc700 (LWP 7216)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  22 Thread 0x7f8c75ffb700 (LWP 7217)  0x000000390700b3b4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  21 Thread 0x7f8c757fa700 (LWP 7219)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  20 Thread 0x7f8c74ff9700 (LWP 7220)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  19 Thread 0x7f8c57fff700 (LWP 7221)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  18 Thread 0x7f8c577fe700 (LWP 7222)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  17 Thread 0x7f8c56ffd700 (LWP 7223)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  16 Thread 0x7f8c567fc700 (LWP 7228)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  15 Thread 0x7f8c55ffb700 (LWP 7229)  0x000000390700b3b4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  14 Thread 0x7f8c557fa700 (LWP 7231)  0x000000390700b3b4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  13 Thread 0x7f8c54ff9700 (LWP 7233)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  12 Thread 0x7f8c37fff700 (LWP 7234)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  11 Thread 0x7f8c377fe700 (LWP 7235)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  10 Thread 0x7f8c36ffd700 (LWP 7237)  0x000000390700b3b4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  9 Thread 0x7f8c367fc700 (LWP 7243)  0x0000003906cd7283 in poll () from /lib64/libc.so.6
---Type <return> to continue, or q <return> to quit--- 
  8 Thread 0x7f8c037fe700 (LWP 7322)  0x000000390700b3b4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  7 Thread 0x7f8c1ce5b700 (LWP 7575)  0x000000390700e6ed in nanosleep () from /lib64/libpthread.so.0
  6 Thread 0x7f8c03fff700 (LWP 7576)  0x000000390700e6ed in nanosleep () from /lib64/libpthread.so.0
  5 Thread 0x7f8bfbfff700 (LWP 7577)  0x000000390700b3b4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  4 Thread 0x7f8bfb7fe700 (LWP 7578)  0x000000390700b3b4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  3 Thread 0x7f8c1dffb700 (LWP 14647)  0x000000390700b71e in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  2 Thread 0x7f8c1e7fc700 (LWP 14648)  0x0000003906cd9093 in select () from /lib64/libc.so.6
* 1 Thread 0x7f8c95306840 (LWP 7191)  0x000000390700b3b4 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0

Attachments (2)

gdb.txt (28.3 KB) - added by anders.franzen@… 7 years ago.
threads apply all bt
backendversion (876 bytes) - added by anders.franzen@… 7 years ago.
mythbackend --version

Download all attachments as: .zip

Change History (14)

comment:1 Changed 7 years ago by beirdo

Description: modified (diff)
Milestone: 0.26unknown

comment:2 Changed 7 years ago by beirdo

We will need a full backtrace of all threads (use thread apply all bt). info threads is not nearly enough information.

Also, please attach the backtrace in a file :)

comment:3 Changed 7 years ago by danielk

Anders, is this on CentOS? We've been seeing similar reports on CentOS that we've been unable to trace to anything wrong on our end.

Changed 7 years ago by anders.franzen@…

Attachment: gdb.txt added

threads apply all bt

comment:4 in reply to:  3 Changed 7 years ago by anders.franzen@…

Replying to danielk:

Anders, is this on CentOS? We've been seeing similar reports on CentOS that we've been unable to trace to anything wrong on our end.

Hi, no, this is on Fedora 14. I will attach full thread backtrace. /A

comment:5 Changed 7 years ago by karl@…

Im seeing this issue on Ubuntu 12.04 - need to restart nightly - unfortunately need to often kill.

comment:6 Changed 7 years ago by anders.franzen@…

After playing around with the mythtv-setup, I found out that if I disable the EIT as Listing grabber in the Video Sources menu, the problem goes away. So I think it might be related to the EIT...

/Anders

comment:7 Changed 7 years ago by sphery

Please provide mythbackend --version information (attach as a file to this ticket). This sounds like it could be a dup of #10541 , which was fixed in master in a1d90159244 .

Changed 7 years ago by anders.franzen@…

Attachment: backendversion added

mythbackend --version

comment:8 Changed 7 years ago by anders.franzen@…

mythbackend version attached.

/Anders

comment:9 Changed 6 years ago by anders.franzen@…

I noticed in the mythbacken log that the last line in the log regarding any specific recorder, was like this:

2012-05-14 11:43:27.714030 E [14686/14711] TVRecEvent tv_rec.cpp:3660 (TuningFrequency?) - TVRec(9): Failed to set channel to 65. Reverting to kState_None

It seems I had an old channel specified that has been removed by the provider/distributor.

After removing that channel from my defined channels, the backend does not hang anymore. I tried to look in the src to see if I could come to any conclusions about this, but I dont have knowledge enough about how it works. I guess all the tuning is done from the main TVrec::run loop??

/Anders

comment:10 Changed 6 years ago by beirdo

This looks a lot like #10771, the fixes there might fix this.

comment:11 Changed 6 years ago by beirdo

Owner: set to danielk
Status: newassigned

comment:12 Changed 6 years ago by stuartm

Milestone: unknown0.25.2
Resolution: Fixed
Status: assignedclosed

This should be fixed. See #10771

Note: See TracTickets for help on using tickets.