Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#12531 closed Bug Report - General (fixed)

vdpau segfaults when video dimensions not multiple of 16

Reported by: yiannividalis@… Owned by: JYA
Priority: minor Milestone: 0.28
Component: MythTV - Video Playback Version: Master Head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

Hi, I am running latest master (g218306f), but the problem was present at g7126654 too, I just now managed to create the samples.

It happens: on my pc monitor (1920x1080x60Hz), nvidia GTX 750, 352.41 driver. Vlc plays video fine (with vdpau selected).

It doesn't have a problem on my tv (which can switch resolutions and refresh rates, if it is relevant) with nvidia 9500gt, driver 340.76.

I will attach two samples of the same 20sec video, one that segfaults and another one that doesn't.

Please let me know if I should provide any more info.

MythTV Version : v0.28-pre-3156-g218306f-dirty
MythTV Branch : master
Network Protocol : 87
Library API : 0.28.20151024-1
QT Version : 5.4.2
Options compiled in:
 linux profile using_alsa using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_bindings_php using_dvb using_frontend using_vbox using_ceton using_hdpvr using_libcrypto using_libfftw3 using_libxml2 using_lirc using_mheg using_opengl using_opengl_video using_qtwebkit using_qtscript using_qtdbus using_taglib using_v4l2 using_x11 using_xrandr using_xv using_profiletype using_bindings_perl using_bindings_python using_bindings_php using_freetype2 using_mythtranscode using_opengl using_vaapi using_vdpau using_ffmpeg_threads using_mheg using_libass using_libxml2

Attachments (3)

sample1920x798-segfault.mkv (396.4 KB) - added by yiannividalis@… 4 years ago.
video that crashes mythavtest and frontend
sample1920x800-ok.mkv (390.0 KB) - added by yiannividalis@… 4 years ago.
video that plays ok
gdb.txt (43.4 KB) - added by yiannividalis@… 4 years ago.
compile is profile, not debug

Download all attachments as: .zip

Change History (9)

Changed 4 years ago by yiannividalis@…

Attachment: sample1920x798-segfault.mkv added

video that crashes mythavtest and frontend

Changed 4 years ago by yiannividalis@…

Attachment: sample1920x800-ok.mkv added

video that plays ok

comment:1 Changed 4 years ago by JYA

Please provide a backtrace.

Thank you for the report.

Changed 4 years ago by yiannividalis@…

Attachment: gdb.txt added

compile is profile, not debug

comment:2 Changed 4 years ago by yiannividalis@…

I attached the gdb.txt file but it's with compile-type=profile, not debug. If it doesn't help, let me know and I'll recompile tomorrow.

Thanks for the quick response.

comment:3 Changed 4 years ago by yiannividalis@…

la /usr/lib64/vdpau/
σύνολο 900K
drwxr-xr-x   2 root root 4,0K Οκτ   9 17:01 .
drwxr-xr-x 204 root root 216K Οκτ  31 09:53 ..
lrwxrwxrwx   1 root root   25 Οκτ   9 17:01 libvdpau_nvidia.so.1 -> libvdpau_nvidia.so.352.41
-rwxr-xr-x   1 root root 623K Οκτ   9 17:01 libvdpau_nvidia.so.352.41
lrwxrwxrwx   1 root root   23 Ιούν 18 23:30 libvdpau_trace.so.1 -> libvdpau_trace.so.1.0.0
-rwxr-xr-x   1 root root  51K Ιούν 18 23:30 libvdpau_trace.so.1.0.0

comment:4 Changed 4 years ago by Jean-Yves Avenard <jyavenard@…>

Resolution: fixed
Status: newclosed

In ce3eb5e4a0b745d7abe57979fd38f219ea5ed2ac/mythtv:

Get around libvdpau crashing with some resolutions.

Two problems at hand, one is ours being that the pitch for U plane was wrong (twice as big as it should have been).
There's still appear to be a problem with libvdpau causing an out of bound read when using resolution not multiple of 16. The issue is bypassed here as we allocate more than necessary and have both the U and V plans pointing to the same buffer, giving us plenty of leeway.

Fixes #12531

comment:5 Changed 4 years ago by Jean-Yves Avenard <jyavenard@…>

In ce3eb5e4a0b745d7abe57979fd38f219ea5ed2ac/mythtv:

Get around libvdpau crashing with some resolutions.

Two problems at hand, one is ours being that the pitch for U plane was wrong (twice as big as it should have been).
There's still appear to be a problem with libvdpau causing an out of bound read when using resolution not multiple of 16. The issue is bypassed here as we allocate more than necessary and have both the U and V plans pointing to the same buffer, giving us plenty of leeway.

Fixes #12531

comment:6 Changed 4 years ago by paulh

Milestone: unknown0.28
Note: See TracTickets for help on using tickets.