Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#13176 closed Patch - Bug Fix (fixed)

AVsync is broken at high playback speeds in some streams

Reported by: Brad Martin <mythtv@…> Owned by: Peter Bennett
Priority: minor Milestone: 30.0
Component: MythTV - Video Playback Version: 0.28.1
Severity: medium Keywords:
Cc: Ticket locked: no

Description

At playback speeds of 1.5x or greater, I have found that the video can often surge way ahead of the audio, by tens of seconds or more. This often happens during a commercial break, where I was in sync prior to the commercial, but completely lose sync upon the transition. It may eventually sync again when the show comes back on. FWIW I have a frame-doubling interlacer configured, which may be implicated as well.

I have tracked this down to a couple of causes:

  • The VideoSync? class is not being kept up to date with the frame interval and frame-doubling state at any given time as the stream and playback speed change.
  • The VideoSync? is attempting to clamp the maximum sync adjustment to a small multiple of the wrong frame interval due to the above.
  • The avsync_predictor is continually dropping frames, because it is failing to account for extra time due to the repeat_pict setting on some video frames, as well as any dynamically-applied sync delay.

Attached is a patch against the fixes/0.28 branch that improves these issues for me.

Attachments (2)

0001-mythplayer-Fix-issues-with-vsync.patch (17.9 KB) - added by Brad Martin <mythtv@…> 3 years ago.
20171120_#13176_mythplayer-Fix-issues-with-vsync2.patch (16.8 KB) - added by Peter Bennett 3 years ago.
Patch fixed for applying to master branch

Download all attachments as: .zip

Change History (18)

Changed 3 years ago by Brad Martin <mythtv@…>

comment:1 Changed 3 years ago by Stuart Auchterlonie

Milestone: needs_triage30.0
Owner: changed from JYA to Peter Bennett
Status: newassigned

comment:2 Changed 3 years ago by Peter Bennett

Status: assignedinfoneeded

Can you supply a sample video which is subject to the video surging ahead of the audio? I just watched part of a 1080i recording which has AC3 sound at 2x speed including a commercial and I did not see any occasion where video and audio were out of sync. Perhaps your commercial break is at a different resolution from the program. I would like to compare playback before and after this fix.

comment:3 Changed 3 years ago by Brad Martin <mythtv@…>

I don't think it's the resolution, I think it's the framerate. The show I debugged with (NBC Nightly News) had a repeat_pict of zero during the actual program, but positive values during different commercials.

I'll crop and upload a verified sample when I get a chance.

comment:4 Changed 3 years ago by Brad Martin <mythtv@…>

Status: infoneededassigned

At this link you can find a snippet of a transport stream from an NBC Nightly News recording, which shows the problem as it goes into commercial. I also included a cellphone video of how this clip plays on my machine without the patch.

https://drive.google.com/open?id=1BE4NP_liMhVYlsq4E-EBVAFv2nVT1Tvl

comment:5 Changed 3 years ago by Brad Martin <mythtv@…>

I just added another clip at that same location from SNL (not during commercial).

comment:6 Changed 3 years ago by Peter Bennett

I tested this patch with VAAPI, VDPAU and Raspberry Pi. It works well on the test videos, and I could not find any negative impact.

The first test video (NBC Nightly News) audio gets jerky at 2.0 stretch. That problem is fixed if you increase the Audio Read Ahead (available in master, see #13172).

I did not find any video of my own that exhibits the avsync problem when speeded up. I have a recording of that same Nightly News broadcast and it is fine speeded up without the patch, even when going into commercials. The problem must be dependent on the individual broadcast station.

Last edited 3 years ago by Peter Bennett (previous) (diff)

comment:7 Changed 3 years ago by Mark Spieth

Can you list the formats res + interlacing, deinterlacers used by this video. This would allow a better decision to be made. What it could mean also is CPU decoding power limited if the stream is HD or 4k 60fps progressive. The video stream after decoding is bursty. I havent analysed this case yet though.

Maybe for testing it might be good to have both modes available with a settings change.

This might be too difficult though from a practical pov.

comment:8 Changed 3 years ago by mythtv@…

The problem is not jerkiness, rather the video is surging well ahead of the audio, as you can see in the mkv recording. I have been fighting this issue regularly for six months or longer, so it is neither a new problem nor is it exclusive to highly specific videos. The problem happens on two different machines for me, both running the same myth version. Both of my machines seem to be using "usleep+busy wait" as the sync method, however a brief test using RTC timing showed no difference. Both machines are running Ubuntu 16.04 with NVIDIA graphics. The first machine is using the VDPAU High Quality playback profile. The primary deinterlacer is "Advanced (2x HW)", and the fallback is "Advanced (1x HW)". The second machine (the one the mkv was captured on) is using the OpenGL High Quality profile, with "Greedy HighMotion? (2x)" and "Greedy HighMotion?" deinterlacers.

I don't believe a CPU limitation could explain the video surging *ahead* of the audio in this manner.

comment:9 Changed 3 years ago by Peter Bennett

Maybe it is not clear from my comment - I think the patch is good and worthwhile installing.

comment:10 Changed 3 years ago by Mark Spieth

Hmmm.

Increased audio buffer works though. Then your patch must be making the playback smoother or adding its own latency to compensate for the A/V timestamp separation.

It would be interesting to see how hardware sync from dri which I believe causes less issues with sync especially tearing without requiring double/triple buffering by the video card.

Is the stream 1080p60?

comment:11 in reply to:  9 ; Changed 3 years ago by Mark Spieth

Replying to pbennett:

Maybe it is not clear from my comment - I think the patch is good and worthwhile installing.

OK commit it. Im comfortable with the level of testing you have done. I will install and trial ASAP.

comment:12 in reply to:  11 Changed 3 years ago by Peter Bennett

Replying to markspieth:

OK commit it. Im comfortable with the level of testing you have done. I will install and trial ASAP.

I will wait a couple of days in case anybody else want to test. The code looks reasonable. I may think of other test cases in the meantime and I will test it with a few other videos to make sure I see no adverse impact. Let me know if you see any problems.

The test videos are 1280x720 60fps MPEG2, Audio is AC-3, 6 channels.

Changed 3 years ago by Peter Bennett

Patch fixed for applying to master branch

comment:13 Changed 3 years ago by Peter Bennett

The original patch has a conflict with master. This is the same patch but fixed for master.

comment:14 Changed 3 years ago by Stuart Auchterlonie

:shipit:

comment:15 Changed 3 years ago by Brad Martin <mythtv@…>

Resolution: fixed
Status: assignedclosed

In 67a11c1599f2d4e33d4e0859dfa30ed537f8563d/mythtv:

mythplayer: Fix issues with vsync with time stretch

Vsync was broken with various combinations of:

  • 2x deinterlacing
  • High audio stretch factors (1.8x+)
  • Video streams using repeat_pict

This was due to a combination of:

  • avsync_predictor failing to predict the correct frame interval
  • The vsync implementation trying to clamp the frame delay to a small multiple of the wrong frame interval, due to getting out sync with the interval and interlace setting at any given moment.

With this change, the vsync method no longer tries to independently
track the frame interval and interlace setting; instead, this is
passed on a frame-by-frame basis.

Fixes #13176

Signed-off-by: Peter Bennett <pbennett@…>

comment:16 Changed 3 years ago by Peter Bennett

Owner: changed from Peter Bennett to Peter Bennett
Note: See TracTickets for help on using tickets.