Opened 9 years ago

Closed 9 years ago

#9700 closed Bug Report - Crash (Unverified)

0.24 Crash after Recording Finished with memory Corruption (glibc)

Reported by: flybrick@… Owned by: danielk
Priority: minor Milestone: unknown
Component: MythTV - Recording Version: 0.24-fixes
Severity: medium Keywords:
Cc: Ticket locked: no

Description (last modified by Raymond Wagner)

I am running a dedicated mythbackend with 4 PVR-150 cards. I am seeing a repeated crashes of the backend after running for several days. It appears that the series of events leading up to the event is that at least 1 PVR-150 card becomes unavailable for a period of time, then a recording finished, then scheduling starts, then a malloc memory corruption error is generated.

I am attaching the mythbackend.log with the events leading up to the crash and the crash dump itself. The crash occurs about 1/2 way through the log at "2011-03-28 23:01:00.737"

Attachments (2)

crashlog_20110328.txt.gz (14.8 KB) - added by flybrick@… 9 years ago.
mythbackend.log for crash
version_info (676 bytes) - added by Raymond Wagner 9 years ago.

Download all attachments as: .zip

Change History (8)

Changed 9 years ago by flybrick@…

Attachment: crashlog_20110328.txt.gz added

mythbackend.log for crash

comment:1 Changed 9 years ago by robertm

Status: newinfoneeded_new

In order to debug something like this, we will need a backtrace of the crash per the manual:

http://www.mythtv.org/wiki/Debugging

comment:2 Changed 9 years ago by Raymond Wagner

Description: modified (diff)

Changed 9 years ago by Raymond Wagner

Attachment: version_info added

comment:3 Changed 9 years ago by flybrick@…

I am currently rebuilding with "--compile-type=profile" so as to generate a better backtrace from a core dump. It tends to take several days to a week to crash.

Unfortunately, I had already pulled the latest fixes/0.24, not the one that crashed here.

In the meantime, I was able to extract a C++ mangled-name backtrace from the last crash, and tracing back through the code, it looks like it crashed in I am currently rebuilding with "--compile-type=profile" so as to generate a better backtrace from a core dump. It tends to take several days to a week to crash.

Unfortunately, I had already pulled the latest fixes/0.24, not the one that crashed here.

In the meantime, I was able to extract a C++ mangled-name backtrace from the last crash.

It looks like the crash is somehow entered in the method "void DeviceReadBuffer::fill_ringbuffer(void)" in DeviceReadBuffer?.cpp, at one of the calls to lock.lock() (~line 256 or 304).

From there, it just continues through libQtCore until glibc detects the memory corruption. If it really is a memory corruption issue, it is quite possible it didn't occur anywhere in the call chain, but at some other unrelated point.

2011-03-28 23:01:00.737 scheduler: Finished recording: Castle "Law & Murder": channel 1013
*** glibc detected *** /usr/bin/mythbackend: malloc(): memory corruption: 0x09fc5c39 ***
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0x22cd591]
/lib/tls/i686/cmov/libc.so.6(+0x6e395)[0x22d0395]
/lib/tls/i686/cmov/libc.so.6(__libc_malloc+0xec)[0x22d202c]
/usr/lib/libQtCore.so.4(_Z7qMallocj+0x1d)[0x7c5db8d]
/usr/lib/libQtCore.so.4(_ZN10QByteArrayC1Eic+0x36)[0x7c65b66]
/usr/lib/libQtCore.so.4(_Z15qt_error_stringi+0x59)[0x7c5a749]
/usr/lib/libQtCore.so.4(+0x655ff)[0x7c625ff]
/usr/lib/libQtCore.so.4(+0x65780)[0x7c62780]
/usr/lib/libQtCore.so.4(_ZN6QMutex4lockEv+0x132)[0x7c5e052]
/usr/lib/libmythtv-0.24.so.0(_ZN16DeviceReadBuffer15fill_ringbufferEv+0x2c)[0x12a449c]
/lib/tls/i686/cmov/libc.so.6(clone+0x5e)[0x232fa4e]

comment:4 Changed 9 years ago by Raymond Wagner

Errors like that typically indicate some form of hardware or system instability outside of the program itself. Are you experiencing anomalous faults in any other programs? Have you tried running memtest or similar software?

comment:5 in reply to:  4 Changed 9 years ago by John Werner <flybrick@…>

Replying to wagnerrp:

Errors like that typically indicate some form of hardware or system instability outside of the program itself. Are you experiencing anomalous faults in any other programs? Have you tried running memtest or similar software?

I've been wondering the same thing. The machine only runs mythbackend. I will try to find sometime when it is not scheduled to record something and boot-up into memtest and see what happens.

[After running MythTv? for 4+ years, its amazing how much you end up recording.]

comment:6 Changed 9 years ago by danielk

Resolution: Unverified
Status: infoneeded_newclosed

I rewrote the DRB buffer handling in the mythtv-rec2 branch, but I'm pretty certain the existing code was not writing past the end of the buffers. So I believe memory overwriting bug is probably at some distance from where the backtrace is from and so this debugging is not useful in finding it.

A valgrind log would change my mind or some repeats..

Note: See TracTickets for help on using tickets.