Opened 17 years ago

Closed 17 years ago

Last modified 17 years ago

#3464 closed patch (fixed)

MythTranscode segfaults on resize when src=dst then src!=dst

Reported by: anonymous Owned by: cpinkham
Priority: minor Milestone: unknown
Component: mythtv Version: unknown
Severity: medium Keywords: mythtranscode segfault resize
Cc: Ticket locked: no

Description

There is a bug in mythtranscode which generates a segfault when resizing the frame. The problem only occurs when the source file changes resolutions and only after the source resolution is the same as the destination and then changes again.

In my test case initial source resolution was 544x480 with a "noresize" profile. The source resolution changed to 528x480 (source is mpeg2 ts) and mythtranscode would segfault every time in libmythavcodec.so img_resample. If you look at programs/mythtranscode/transcode.cpp

            if ((video_width == newWidth) && (video_height == newHeight))
            {
                frame.buf = lastDecode->buf;
            }
            else
            {
                avpicture_fill(&imageIn, lastDecode->buf, PIX_FMT_YUV420P,
                               video_width, video_height);
                avpicture_fill(&imageOut, frame.buf, PIX_FMT_YUV420P,
                               newWidth, newHeight);
             ...

You can see that when the frame dimensions are the same, the frame buffer is simply passed through to the recorder. However, when the dimensions are different, img_resample writes to frame.buf. Unfortunately frame.buf still points to the previous frame's lastDecode->buf which is invalid. The attached patch sets the frame.buf pointer to the proper scratch buffer (newFrame).

The other unrelated line change is in the newFrame allocator. I don't see why the size of the buffer needs to be recalculated considering it was calculated previously.

Attachments (1)

transcode_resize_fix.diff (1.2 KB) - added by bmayland@… 17 years ago.
3 line patch

Download all attachments as: .zip

Change History (5)

Changed 17 years ago by bmayland@…

Attachment: transcode_resize_fix.diff added

3 line patch

comment:1 Changed 17 years ago by bmayland@…

I should have searched better first. This looks like it might be the solution to tickets #2360, #3183, #3274 as well.

comment:2 Changed 17 years ago by cpinkham

Owner: changed from Isaac Richards to cpinkham

comment:3 Changed 17 years ago by cpinkham

Resolution: fixed
Status: newclosed

(In [13467]) Reset frame.buf when necessary in the transcoder. This should fix an issue with segfaults when the dimensions of the input video change over the course of the video.

Also reuse the calculated frame size instead of recalculating when allocating the frame buffer.

Patch by Bryan Mayland.

Closes #3464.

comment:4 Changed 17 years ago by cpinkham

(In [14548]) Carry over [13467] from trunk. Here is the original commit log:

Reset frame.buf when necessary in the transcoder. This should fix an issue with segfaults when the dimensions of the input video change over the course of the video.

Also reuse the calculated frame size instead of recalculating when allocating the frame buffer.

Patch by Bryan Mayland.

References #3464.

Note: See TracTickets for help on using tickets.