FE crash w/ XvMC on 720x480p mpg

[gary@…]

(tons of info coming) Version is actually 19-fixes

I've been having random crashes on the FE when my 2 year old watches her shows. All these shows are converted tivo recordings that have been re-encoded with TMPGEnc to 720x480p. The shows were imported into myth via the rebuilddatabase scripts, and 'mythcommflag --rebuild' run on them.

Finally, I remembered to set ulimit -c so I can see a core dump, as the "-v playback" logs appear useless (not completely useless, as they demonstrate a possible issue: When showing a 480p mpg, myth appears to detect it as 480i (interlaced.) I don't have this problem with 720p mpgs.

Attached, please find mythfrontend.log.gz (rather than truncate the log and possibly lose information you might want, I've left the entire thing intact. Search for "2006-05-26 17:18:32.848" to find the first log entry for the showing that resulted in the crash.

mythfrontend --version: Library API version: 0.19.20060121-2 Source code version: 9856 Options compiled in:

linux release using_v4l using_alsa using_ivtv using_lirc using_dvb using_dvb_eit using_x11 using_xv using_dvdnav using_xrandr using_xvmc using_opengl_vsync using_frontend using_backend

BT from the core dump: (gdb) bt #0 0x00002aaaab24c6c1 in VideoBuffers::AddInheritence? ()

from /usr/lib/libmythtv-0.19.so.0

#1 0x00002aaaab293446 in VideoOutputXv::DrawSlice? ()

from /usr/lib/libmythtv-0.19.so.0

#2 0x00002aaaab1e87b4 in render_slice_xvmc ()

from /usr/lib/libmythtv-0.19.so.0

#3 0x00002aaaabbb569b in ff_draw_horiz_band ()

from /usr/lib/libmythavcodec-0.19.so.0

#4 0x00002aaaabc5c059 in mpeg1_encode_mb ()

from /usr/lib/libmythavcodec-0.19.so.0

#5 0x00002aaaabc5e5de in ff_mpeg1_find_frame_end ()

from /usr/lib/libmythavcodec-0.19.so.0

#6 0x00002aaaabba6573 in avcodec_decode_video ()

from /usr/lib/libmythavcodec-0.19.so.0

#7 0x00002aaaab1f63cc in AvFormatDecoder::GetFrame? ()

from /usr/lib/libmythtv-0.19.so.0

#8 0x00002aaaab1c21f2 in NuppelVideoPlayer::GetFrameNormal? ()

from /usr/lib/libmythtv-0.19.so.0

#9 0x00002aaaab1c28c0 in NuppelVideoPlayer::GetFrame? ()

from /usr/lib/libmythtv-0.19.so.0

#10 0x00002aaaab1c73b4 in NuppelVideoPlayer::StartPlaying? ()

from /usr/lib/libmythtv-0.19.so.0

#11 0x00002aaaab1623e9 in SpawnDecode? () from /usr/lib/libmythtv-0.19.so.0 ---Type <return> to continue, or q <return> to quit--- #12 0x00002aaaade7d14a in start_thread () from /lib/tls/libpthread.so.0 #13 0x00002aaaae499943 in clone () from /lib/tls/libc.so.6

(Log is attached)

If there's any other information I can give to help resolve this recurring issue, please let me know (I also monitor the -dev, -users, and -commit lists)

[gary@…]

FE log

[Isaac Richards]

Need to compile in debug mode and generate a backtrace.

[gary@…]

recompiling now with changed configure switch (--compile-type=debug). Any bets on it not crashing now? Will re-open the bug once I get a bt with debug build

[gary@…]

Finally! I didn't think it was going to crash at all with a debug compile... but it's finally repeated. Attached is mythfrontend.log.2.gz (If its useful)

Version:

server mythtv # DISPLAY=:0 /usr/bin/mythfrontend --version Library API version: 0.19.20060121-2 Source code version: 10051 Options compiled in:

linux debug using_v4l using_alsa using_ivtv using_lirc using_dvb using_dvb_eit using_x11 using_xv using_dvdnav using_xrandr using_xvmc using_opengl_vsync using_frontend using_backend

Core BT:

(gdb) bt #0 0x00002aaaab31c58c in MythDeque?<VideoFrame_*>::find (this=0x2aaab5008c00,

item=0x2aaab7b94170) at mythdeque.h:47

#1 0x00002aaaab31a0c5 in MythDeque?<VideoFrame_*>::contains (

this=0x2aaab5008c00, item=0x2aaab7b94170) at mythdeque.h:62

#2 0x00002aaaab315a5f in VideoBuffers::AddInheritence? (this=0x2aaab5008ba8,

frame=0x2aaab7b94310) at videobuffers.cpp:875

#3 0x00002aaaab34cb4a in VideoOutputXv::DrawSlice? (this=0x2aaab5008ab0,

frame=0x2aaab7b94310, x=0, y=0, w=720, h=16) at videoout_xv.cpp:2554

#4 0x00002aaaab281ed8 in NuppelVideoPlayer::DrawSlice? (this=0x1243680,

frame=0x2aaab7b94310, x=0, y=0, w=720, h=16) at NuppelVideoPlayer?.cpp:924

#5 0x00002aaaab2b2b76 in render_slice_xvmc (s=0x23c8d60, src=0x2aaac0dc3e90,

offset=0x42805420, y=0, type=3, height=16) at avformatdecoder.cpp:1314

#6 0x00002aaaabc9dcde in ff_draw_horiz_band (s=0x42804ae0, y=1115704000,

h=-1188050864) at mpegvideo.c:4111

#7 0x00002aaaabd5ddd2 in mpeg_decode_slice (s1=0x42804ae0, mb_y=16,

buf=0x42805520, buf_size=1115704000) at mpeg12.c:2665

#8 0x00002aaaabd5efb7 in mpeg_decode_frame (avctx=0x23c8d60, data=0x42804ac0,

data_size=0x428057dc, buf=0x2aaabe26f110 "", buf_size=4363) at mpeg12.c:3227

#9 0x00002aaaabc8becf in avcodec_decode_video (avctx=0x23c8d60,

picture=0x42805880, got_picture_ptr=0x428057dc, buf=0x2aaabe26f110 "", buf_size=4363) at utils.c:947

#10 0x00002aaaab2b98b8 in AvFormatDecoder::GetFrame? (this=0x1611860,

onlyvideo=0) at avformatdecoder.cpp:2518

#11 0x00002aaaab28235c in NuppelVideoPlayer::GetFrameNormal? (this=0x1243680,

onlyvideo=0) at NuppelVideoPlayer?.cpp:979

#12 0x00002aaaab282de7 in NuppelVideoPlayer::GetFrame? (this=0x1243680,

onlyvideo=0, unsafe=false) at NuppelVideoPlayer?.cpp:1057

#13 0x00002aaaab28c104 in NuppelVideoPlayer::StartPlaying? (this=0x1243680)

at NuppelVideoPlayer?.cpp:2642

#14 0x00002aaaab240349 in SpawnDecode? (param=0x1243680) at tv_play.cpp:190 #15 0x00002aaaadfc414a in start_thread () from /lib/tls/libpthread.so.0 #16 0x00002aaaae5e0943 in clone () from /lib/tls/libc.so.6 #17 0x0000000000000000 in ?? ()

Some info in frame0

(gdb) frame 0 #0 0x00002aaaab31c58c in MythDeque?<VideoFrame_*>::find (this=0x2aaab5008c00,

item=0x2aaab7b94170) at mythdeque.h:47

47 if (*it == item) (gdb) print it $11 = {_M_cur = 0x50, _M_first = 0x50, _M_last = 0x250,

_M_node = 0x2aaabbf8e950}

(gdb) print item $12 = (VideoFrame? * const) 0x2aaab7b94170 (gdb) print *item $13 = {codec = FMT_XVMC_IDCT_MPEG2, buf = 0x2aaab920bbb0 "�\021�\035",

height = 480, width = 720, bpp = -1, size = 32, frameNumber = 14084, timecode = 469936, priv = {0x2aaab6778dd0 "�\037", 0x2aaab6778df0 "F\005",

0x0, 0x0}, qscale_table = 0x0, qstride = 0, interlaced_frame = 0,

top_field_first = 0, forcekey = 0}

(gdb)

Some info in frame1

(gdb) frame 1 #1 0x00002aaaab31a0c5 in MythDeque?<VideoFrame_*>::contains (

this=0x2aaab5008c00, item=0x2aaab7b94170) at mythdeque.h:62

62 { return find(item) != deque<T>::end(); } (gdb) print item $6 = (VideoFrame? * const) 0x2aaab7b94170 (gdb) print *item $7 = {codec = FMT_XVMC_IDCT_MPEG2, buf = 0x2aaab920bbb0 "�\021�\035",

height = 480, width = 720, bpp = -1, size = 32, frameNumber = 14084, timecode = 469936, priv = {0x2aaab6778dd0 "�\037", 0x2aaab6778df0 "F\005",

0x0, 0x0}, qscale_table = 0x0, qstride = 0, interlaced_frame = 0,

top_field_first = 0, forcekey = 0}

(gdb)

[gary@…]

frontend log from -v playback

[danielk]

Can you try to reproduce this with SVN head?

There have been some chnages in this code, but I'll look at this since the changes weren't intended to prevent a crash.

[anonymous]

got another crash this afternoon with the same 19-fixes branch as above (same binary) backtrace has the same functions. Looks like it crashed at the same point (listing below)

44          const_iterator find(T const item) const
45          {
46              for (const_iterator it = deque<T>::begin(); 
                     it != deque<T>::end(); ++it)
47                  if (*it == item)   // SIGSEG ON THIS LINE
48                      return it;
49              return deque<T>::end();
50          }

The following might also help (done from frame 0):

(gdb) print *it
You can't do that without a process to debug.
(gdb) print it
$1 = {_M_cur = 0x39f6ee8, _M_first = 0x39f6cf0, _M_last = 0x39f6ef0,
  _M_node = 0xaf0630}
(gdb) print item
$2 = (VideoFrame * const) 0x3763b18
(gdb) print it._M_cur
$3 = (VideoFrame **) 0x39f6ee8
(gdb) print *(it._M_cur)
Cannot access memory at address 0x39f6ee8
(gdb)

[gary@…]

repeated on trunk

Library API version: 0.19.20060631-1
Source code version: 10111:10113
Options compiled in:
 linux debug using_lmsensors using_v4l using_alsa using_ivtv using_lirc using_dvb using_x11 using_xv using_xrandr using_xvmc using_xvmc_pbuffer using_opengl_vsync using_opengl using_frontend using_backend

Backtrace:

#0  0x00002aaaab45a136 in MythDeque<VideoFrame_*>::find (this=0x220d940,
    item=0x4175ff0) at mythdeque.h:47
#1  0x00002aaaab457ca7 in MythDeque<VideoFrame_*>::contains (this=0x220d940,
    item=0x4175ff0) at mythdeque.h:62
#2  0x00002aaaab45365f in VideoBuffers::AddInheritence (this=0x220d8e8,
    frame=0x4175ea0) at videobuffers.cpp:883
#3  0x00002aaaab48c310 in VideoOutputXv::DrawSlice (this=0x220d7b0,
    frame=0x4175ea0, x=0, y=0, w=720, h=16) at videoout_xv.cpp:2578
#4  0x00002aaaab396d02 in NuppelVideoPlayer::DrawSlice (this=0x4f6b620,
    frame=0x4175ea0, x=0, y=0, w=720, h=16) at NuppelVideoPlayer.cpp:1130
#5  0x00002aaaab3e0aa2 in render_slice_xvmc (s=0x5130660, src=0x2aaab77f3340,
    offset=0x4207d020, y=0, type=3, height=16) at avformatdecoder.cpp:1592
#6  0x00002aaaabdb3f18 in ff_draw_horiz_band (s=0x4207c6e0, y=1107805888,
    h=30367472) at mpegvideo.c:4215
#7  0x00002aaaabe7e58d in mpeg_decode_slice (s1=0x4207c6e0, mb_y=16,
    buf=0x4207d128, buf_size=1107805888) at mpeg12.c:2658
#8  0x00002aaaabe7f8dd in mpeg_decode_frame (avctx=0x5130660, data=0x4207c6c0,
    data_size=0x4207d2ac, buf=0x2aaab8276090 "", buf_size=1604)
    at mpeg12.c:3253
#9  0x00002aaaabda1774 in avcodec_decode_video (avctx=0x5130660,
    picture=0x4207d2b0, got_picture_ptr=0x4207d2ac, buf=0x2aaab8276090 "",
    buf_size=1604) at utils.c:953
#10 0x00002aaaab3e7b80 in AvFormatDecoder::GetFrame (this=0x17f0c70,
---Type <return> to continue, or q <return> to quit---q
onlyvideo=0Quit
) at avformatdecoder.cpp:2908
#11 0x00002aaaab3971ac in NuppelVideoPlayer::GetFrameNormal (this=0x4f6b620,
    onlyvideo=0) at NuppelVideoPlayer.cpp:1185
#12 0x00002aaaab397c40 in NuppelVideoPlayer::GetFrame (this=0x4f6b620,
    onlyvideo=0, unsafe=false) at NuppelVideoPlayer.cpp:1263
#13 0x00002aaaab3a2147 in NuppelVideoPlayer::StartPlaying (this=0x4f6b620)
    at NuppelVideoPlayer.cpp:3269
#14 0x00002aaaab347129 in SpawnDecode (param=0x4f6b620) at tv_play.cpp:251

I'll keep the core around if you want more information...

[danielk]

possible fix

[danielk]

Can you try the attached?

It removes the inheritence lock and uses the global lock instead,

I think will fix the segfault you are seeing.

[matt-bulk-mythtv@…]

I reported a similar problem to Gary, and danielk suggested it was a duplicate of this one, so I tried the patch, but am still getting crashes (quite regularly). I've included my backtrace below, although I'm not sure how to embed it into a textarea - sorry :(

Note that the video I was playing was heavily corrupted (lots of MPEG-2 errors due to a very bad DVB signal). I can supply the video file if necessary (although it's large). Also, if you need any more info from the core, please let me know.

This was a version compiled from 0.19-fixes (SVN 10146) with the patch on this ticket applied.

Core was generated by `mythfrontend -l /home/mythtv/mythfrontend.log'.
Program terminated with signal 11, Segmentation fault.
warning: current_sos: Can't read pathname for load map: Input/output error

(gdb) bt
#0  0xb62610ce in mallopt () from /lib/tls/libc.so.6
#1  0xb62608fb in mallopt () from /lib/tls/libc.so.6
#2  0xb625fc43 in malloc () from /lib/tls/libc.so.6
#3  0xb63e12fe in operator new () from /usr/lib/libstdc++.so.5
#4  0xb63cd609 in std::__default_alloc_template<true, 0>::allocate () from /usr/lib/libstdc++.so.5
#5  0xb7ae6a11 in std::__simple_alloc<VideoFrame_*, std::__default_alloc_template<true, 0> >::allocate (__n=128)
    at stl_alloc.h:232
#6  0xb7ae69de in std::_Deque_alloc_base<VideoFrame_*, std::allocator<VideoFrame_*>, true>::_M_allocate_node (
    this=0xb219ec60) at stl_deque.h:409
#7  0xb7adf7a2 in std::_Deque_base<VideoFrame_*, std::allocator<VideoFrame_*> >::_M_create_nodes (this=0xb219ec60, 
    __nstart=0x81ec6bc, __nfinish=0x81ec6c0) at stl_deque.h:534
#8  0xb7add8b5 in std::_Deque_base<VideoFrame_*, std::allocator<VideoFrame_*> >::_M_initialize_map (this=0xb219ec60, 
    __num_elements=0) at stl_deque.h:511
#9  0xb7add774 in _Deque_base (this=0xb219ec60, __a=@0xb219ea70, __num_elements=0) at stl_deque.h:457
#10 0xb7add71c in deque (this=0xb219ec60, __a=@0xb219ea70) at stl_deque.h:690
#11 0xb7add408 in MythDeque (this=0xb219ec60) at videoout_xv.cpp:1972
#12 0xb7ad72da in VideoBuffers::AddInheritence (this=0x9026a2c, frame=0x8e34ca4) at videobuffers.cpp:874
#13 0xb7b10046 in VideoOutputXv::DrawSlice (this=0x9026958, frame=0x8e34ca4, x=0, y=0, w=704, h=0) at videoout_xv.cpp:2545
#14 0xb7a38c97 in NuppelVideoPlayer::DrawSlice (this=0x81fa9c8, frame=0x8e34ca4, x=0, y=0, w=704, h=0)
    at NuppelVideoPlayer.cpp:924
#15 0xb7a6ef51 in render_slice_xvmc (s=0x81e0af0, src=0x91d8380, offset=0xb219f060, y=0, type=3, height=0)
    at avformatdecoder.cpp:1314
#16 0xb71e10f9 in ff_draw_horiz_band (s=0x1010100, y=0, h=0) at mpegvideo.c:4111
#17 0xb7412df9 in XVMC_VLD_decode_slice (s=0x9024c20, mb_y=0, 
    buffer=0x94488c6 "3?;?p?\214?G7?\210s?\a??\216O???[?w&?\t?wN\236\236?N\002ӽ8\026??\234\016s?\234\033?N\205\235?\234\005.\rt?\177\203]:\f#I׮\235\v1?I?N\bB?2n\235?\231:\235\027?N?!???#d\226\223N\207??e8\031???p\035?wN\f??N\2071\237UӠ??׬?t???u:B1?\220E?F?\232t~aq5?????a4", buf_size=17812) at xvmcvldvideo.c:192
#18 0xb73325ad in mpeg_decode_slice (s1=0x9024c20, mb_y=0, buf=0xb219f188, buf_size=16843008) at mpeg12.c:2551
#19 0xb7333cea in mpeg_decode_frame (avctx=0x81e0af0, data=0xb219f450, data_size=0xb219f3ac, buf=0x94488b0 "", 
    buf_size=17834) at mpeg12.c:3227
#20 0xb71d74e4 in avcodec_decode_video (avctx=0x81e0af0, picture=0x1010100, got_picture_ptr=0xb219f3ac, 
    buf=0x1010100 <Address 0x1010100 out of bounds>, buf_size=17834) at utils.c:947
#21 0xb7a767c9 in AvFormatDecoder::GetFrame (this=0x8db8748, onlyvideo=0) at avformatdecoder.cpp:2518
#22 0xb7a39114 in NuppelVideoPlayer::GetFrameNormal (this=0x81fa9c8, onlyvideo=0) at NuppelVideoPlayer.cpp:979
#23 0xb7a39d9c in NuppelVideoPlayer::GetFrame (this=0x81fa9c8, onlyvideo=0, unsafe=false) at NuppelVideoPlayer.cpp:1057
#24 0xb7a43e99 in NuppelVideoPlayer::StartPlaying (this=0x81fa9c8) at NuppelVideoPlayer.cpp:2673
#25 0xb79f10c9 in SpawnDecode (param=0x81fa9c8) at tv_play.cpp:190
---Type <return> to continue, or q <return> to quit---
#26 0xb640db63 in start_thread () from /lib/tls/libpthread.so.0
#27 0xb62c618a in clone () from /lib/tls/libc.so.6 

[danielk]

Gary & Matt, I'm going to need a full backtrace of all the threads.

[danielk]

A more radical fix (please attach a full backtrace with this patch applied)

[matt-bulk-mythtv@…]

Here's a set of backtraces for all threads without the new fix applied; I'll test the new fix tomorrow. Thanks for your help!

[matt-bulk-mythtv@…]

backtrace of all threads

[anonymous]

Note, I had to snip some of the non-ascii data in tbe backtrace.

[danielk]

Backtrace is not from SVN-head + patch + it appears to be an unrelated problem.

Matt, I believe you may have different problem. Unlike Gary's patch yours shows an invalid pointer being passed to avcodec_decode_video(), and it looks like it may be an memory in ffmpeg problem. I've e-mailed you instructions for verifying this.

Gary, your problem appears to be a timing/locking problem and I still need your "thread apply all bt" backtrace to figure out what is going on with this ticket. Reopen w/attached bt..

[anonymous]

1871-v2.patch appears to have fixed the problem at this point. I've been running continuous playback of playlists on mythtv for about 48 hours (while within gdb), and while I haven't had the crash mentioned in this bug, mythtv will certainly crash in the next 24 hours from exhausting memory. (I believe there's another ticket out there for a memory leak.)

Daniel, is there something else I should do/try in regards to this ticket?

[gary@…]

err.. sorry - forgot to put my email in the above ticket.

[danielk]

(In [10241]) Refs #1871. Reworks video buffer locking to avoid two display and decoding thread from accessing buffer dequeques at the same time when tracking inter-frame dependencies.

[danielk]

(In [10242]) Closes #1871. Backport of video buffer locking fix. This reworks video buffer locking to avoid two display and decoding thread from accessing buffer dequeques at the same time when tracking inter-frame dependencies.