Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#5971 closed defect (fixed)

stream reading eats up all memory in all myth apps

Reported by: Hans-Peter Oeri <hp@…> Owned by: Janne Grunau
Priority: major Milestone: 0.22
Component: Video Playback Version: head
Severity: medium Keywords: libmythtv libavformat memory AVFormatDecoder
Cc: Ticket locked: no

Description

Recently, I noticed a server halting. mythbackend on preview generation took up all memory. Upon further inspection, both mythtranscode and mythcommflag do the same.

mythcommflag on a 20MB sample stream eats up 120MB of memory - for full recordings, the system runs OOM. Of course, I tried to "rebuild" seektables, but the bug seems to be "before" that.

From valgrind's/massif's output, it seems that from AVFormatDecoder::OpenFile on, av_malloc'ed and av_mallocz'ed memory is never freed.

Attachments (4)

valgrind.two (128.7 KB) - added by Hans-Peter Oeri <hp@…> 11 years ago.
valgrind log file
massif.two.ms (195.8 KB) - added by Hans-Peter Oeri <hp@…> 11 years ago.
massif log file (ms_printed)
massif.new.ms (123.8 KB) - added by Hans-Peter Oeri <hp@…> 11 years ago.
with eliminated VPS stream; massif log file (ms_printed)
gdb.txt (109.0 KB) - added by Hans-Peter Oeri <hp@…> 11 years ago.
log of av_(malloc|free|realloc), truncated

Download all attachments as: .zip

Change History (17)

Changed 11 years ago by Hans-Peter Oeri <hp@…>

Attachment: valgrind.two added

valgrind log file

Changed 11 years ago by Hans-Peter Oeri <hp@…>

Attachment: massif.two.ms added

massif log file (ms_printed)

comment:1 Changed 11 years ago by Hans-Peter Oeri <hp@…>

Unfortunately, I don't see through the whole allocing/freeing businness in libavformat.

I found, however, that removing a special "stream" from the mpeg ts changes the memory behaviour:

I have the following streams in the file: 0xA0 => MPEG2VIDEO 0xA1 => MP2 0xA3 => AC3 0xA5 => Teletext (TTX) 0xA9 => VPS

It seems, myth recognizes 0xA9 as "second" video stream, which it isn't (http://en.wikipedia.org/wiki/Video_Program_System). Removing 0xa9 from the file (mpegdemux -r / project-x) is a work-around for me.

As VPS is not interpreted by mythtv anyway, and there is no information in VPS to keep, such streams may be safely discarded.

Changed 11 years ago by Hans-Peter Oeri <hp@…>

Attachment: massif.new.ms added

with eliminated VPS stream; massif log file (ms_printed)

comment:2 Changed 11 years ago by Hans-Peter Oeri <hp@…>

If I eliminate the VPS stream but keep the TS format, everything works fine (see massif.new log)

comment:3 Changed 11 years ago by Hans-Peter Oeri <hp@…>

Correction... it seems that all recordings from ORF (Austria) are somehow buggy. I don't know what, the usual tools don't display anything unusual, but a plain conversion through project-x (with all streams intact) solves the problem, too.

Still, bringing down the whole machine is bad(tm). AVFormatDecoder should probably just bail out, if something goes awry.

As a workaround:

  • userjob to go over all recordings with project-x;
  • configure apache to block all preview images (of recordings which might be not yet "cleansed")

comment:4 Changed 11 years ago by Hans-Peter Oeri <hp@…>

Learning to cope with threaded programs ;) I use mythtranscode --mpeg2 --video --infile to really limit the surroundings.

The first ever call to ac_read_frame_internal at libs/libavformat/utils.c:2162 never returns...

comment:5 Changed 11 years ago by stephen.hocking@…

Just a note to confirm that I've seen this too - on Ubuntu 8.04 64bit - it's weird to see mythbackend eating up to 21GB mem, and also Mac OS/X intel, 10.5.5 with MythFrontend. In both cases, svn 19244.

comment:6 Changed 11 years ago by Hans-Peter Oeri <hp@…>

I made gdb log all calls to av_malloc and av_free (and av_realloc)... It seems, that after some startup work, some kind of loop allocating 192/64 byte sizes is entered. Those blocks are never freed (at least until the program reaches its ulimit and gets killed).

See gdb.text attached (truncated).

Changed 11 years ago by Hans-Peter Oeri <hp@…>

Attachment: gdb.txt added

log of av_(malloc|free|realloc), truncated

comment:7 Changed 11 years ago by Hans-Peter Oeri <hp@…>

Working on a less complex sample of 20MB, and with --verbose all, I get the following:

2008-12-07 15:48:41.384 Transcoding from /home/hpoeri/Desktop/trial20.mpg to out20.mpg
2008-12-07 15:48:41.393 Opening /home/hpoeri/Desktop/trial20.mpg
2008-12-07 15:48:41.829 ac-tex damaged at 19 2
2008-12-07 15:48:41.829 Warning MVs not available
2008-12-07 15:48:41.830 concealing 1156 DC, 1156 AC, 1156 MV errors
2008-12-07 15:48:41.857 Could not find codec parameters (Audio: mp3, s16)
2008-12-07 15:48:41.857 Could not find codec parameters (Video: 0x0000)
2008-12-07 15:48:42.233 Input #0, mpegts, from '/home/hpoeri/Desktop/trial20.mpg':
2008-12-07 15:48:42.233   Duration: N/A, bitrate: N/A
2008-12-07 15:48:42.253     Stream #0.0[0x1fa]: Video: mpeg2video, yuv420p, 544x576 [PAR 24:17 DAR 4:3], 3900 kb/s, 25.00 tb(r)
2008-12-07 15:48:42.253     Stream #0.1[0x1fb](ger): Audio: mp3, s16
2008-12-07 15:48:42.253     Stream #0.2[0x1fd](ger): Data: 0x0000
2008-12-07 15:48:42.253     Stream #0.3[0x200]: Video: 0x0000, 90000.00 tb(r)
2008-12-07 15:48:42.254 Skipping invalid audio stream: 1
2008-12-07 15:48:42.254 Skipping unsupported codec 2 on stream 2
2008-12-07 15:48:42.700 Found end of file without finding  any frames
2008-12-07 15:48:42.701 Transcoding /home/hpoeri/Desktop/trial20.mpg failed

Which would seem coherent with the no-freeing noticed above. Although the sample file is playable with all relevant players I have, myth's libavformat seems to eat up memory while searching for a frame (or frame boundaries?)

Unfortunately, I don't know anything about mpegts contents, yet.

However, myth really should give up before EOF or OOM ;)

comment:8 Changed 11 years ago by Dibblah

Owner: changed from Isaac Richards to janneg
Status: newassigned

This looks like it's closely related to #5016 - This time with an audio channel rather than video.

comment:9 Changed 11 years ago by Janne Grunau

Owner: changed from janneg to Janne Grunau
Status: assignedaccepted

comment:10 Changed 11 years ago by Janne Grunau

Milestone: unknown0.22
Priority: blockermajor

comment:11 Changed 11 years ago by Janne Grunau

Resolution: fixed
Status: acceptedclosed

(In [19279]) enhance detection of VBI streams in mpeg ts. Fixes #5971

comment:12 Changed 11 years ago by Janne Grunau

(In [19295]) ignore PRIVATE_DATA streams for now. Refs #5971 They cause huge memleaks with CODEC_ID_PROBE added in [19173]. ffmpeg TS demux ignores them too and we should correct the stream type it contains useful data.

comment:13 Changed 11 years ago by stephen.hocking@…

I can confirm that a memory leak matching this still occurs as of svn 19289 - both Mac OS/X & Linux 64bit.

Note: See TracTickets for help on using tickets.