Predictively skip frames to smooth out timestretch

[jppoet@…]

Timestretch can be a little jerky because Myth just tries to decode/display the frames faster, but they still have to fit within the refresh.

This patch tries to predictively drop frames at a steady rate, to match the timestretch speed. While it does not make playback silky smooth, it does make it much less jerky.

[jppoet@…]

Predictively drop frames based on playback speed

[jpoet]

Use a table for show:skip values

[jpoet]

Update patch for r23788.

Uses a table to define the ratio of shown frames to skipped frames. Using math does not produce expected results.

New version ignores interlaced content, since 30 frame per second material can be display at 2.0x speed on a 60Hz display smoothly without skipping frames.

Still not perfectly smooth past 1.5x play speed, but much better than without the patch.

[jpoet]

Much smoother at non 1.25x,1.5x speeds

[jpoet]

New version is much smoother at non 1.25x/1.5x speeds. 1.25x and 1.5x are still the smoothest.

[Mark Spieth]

vsync fix + predictive skip at any rate

[Mark Spieth]

there are 2 problems with existing smooth playback of mismatched framerate vs refreshrate.

1. vsync for gl and drm have an incorrect equation in WaitForFrame?

2. if frameinterval < refreshrate, it will never work correctly. this si the problem addressed by johns patches though it was only for the case where source material rate = refresh rate, otherwise it introduces blockyness.

the latest patch makes playback really smooth for frameinterval >= refreshinterval. tested with playback,timestmap. divergence is now very stable at any play speed. vsync.cpp part only.

the frame skip is untested (NuppelVideoPlayer?.*) but should work. needs extra testing as my systems cant exceed this limit due to fast refresh speeds.

[Mark Spieth]

smoothsync with better start and unpause

[Mark Spieth]

patch 4 has the previous improvements plus:

1. better unpause and start playing response. This almost eliminates the frame doubling required at start of play or after seek/unpause. Reduces the zero write latency when audio is paused, and discards any audio packets which come before the first video packet.

2. faster avsync convergence. convergence is now adjusted by a frame interval period.

item 1 can be split from the patch if desired. audiooutputbase.cpp and avformatdecoder.cpp,h

the vsync patch can also be split as it is also independent on the other changes in NVP.

[Mark Spieth]

latest patch has accurate avsync estimation algorithm and doesnt require avsync_avg because of it.

fixed audio timecode generation. wont be relevant as done differently in hdaudio branch but important for avsync.

result is very smooth until the cpu has trouble keeping up.

feedback appreciated.

[managementboy@…]

Tried to apply, but I get some hunks that don't. Used with trunk 24479...

[robertm]

Please keep questions about use of patches off of trac... Do not want to have to lock this ticket.

[Mark Spieth]

same as 5 but made from lower in patch stack

[Mark Spieth]

for post hdaudio

[Mark Spieth]

new patch 8 with #8500 patch included for hdaudio. may fix delayed seek issue.

[Mark Spieth]

updated for r25575

[Mark Spieth]

updated patch for r25575.

[jpoet]

Been using version 11 of Mark Spieth's patch for a few days now. It works very well.

I am not qualified to review the actual code, though. Mark Kendall do you still want to own this, or does someone like tralph want to take it?

[Mark Spieth]

split logic for smoothsync to simplify approval.

audio (11) is a logging fix and a line I think is important for locking purposes. most went in #8500.

vsync-opt (11) is the vsync stuff which the player uses.

main (12) one uses both audio and vsync opt.

[stuartm]

Milestone 0.25 deleted

[Mark Spieth]

update after r26895

[Mark Spieth]

updated patch after r26895 which had a conflict.

[jpoet]

version 13 against 0.24-fixes r27341

[jpoet]

Version 11 against 0.24-fixes r27341

[jpoet]

Version 11 against 0.24-fixes r27341

[jpoet]

New patches added for 0.24-fixes r27341

[tralph]

Mark, could you please explain why the audio patch is needed?

[Mark Spieth]

from memory, when upmix state is changed, reset is necessary so reset_active lock is used. independent of the smoothsync patch. there were other parts which were committed by jean-yves a while back I think.

the other is a logging change as the args were bogus.

[Mark Spieth]

the main purpose of audio patch was for accurate audio timestamps. see patch 8 for historical purposes.

[JYA]

Replying to markspieth:

from memory, when upmix state is changed, reset is necessary so reset_active lock is used. independent of the smoothsync patch. there were other parts which were committed by jean-yves a while back I think.

the other is a logging change as the args were bogus.

Mark, ToggleUpmix? calls Reconfigure, which calls reset_active.Clear() very early on. So calling reset_active.Ref() as well would be equivalent to doing: reset_active.head++; immediately followed by: reset_active.head = 0;

I can't imagine why this audio patch would make a difference whatsoever.

[jpoet]

Updated for 206bde055d2d440a730270bf15c34dcf42992d8f

[jpoet]

Updated for 206bde055d2d440a730270bf15c34dcf42992d8f

[jpoet]

Updated patches for trunk 206bde055d2d440a730270bf15c34dcf42992d8f

[tralph]

Remaining extra code for smooth sync feature - needs review - updated for 49065221b40b3bc928bb372f57bb63b26077a871

[tralph]

Mark, I've committed the bulk of the changes and removed some dead code. I'd like to get everything wrapped up but need some questions answered.

vsync change questions

1. For the hardware sync methods you changed the value of 'n' to a new equation. Could you explain this? I have no way of testing these changes.

2. Also you are returning an 'int' for the WaitForFrame? but it's only effective for the hardware vsync methods. Could also explain why this is needed?

3. Which hardware vsync methods have you tested? Only DRM works anymore.

4. Why do you avoid the negative trigger accumulation when doing predictive skip in vsync.cpp. Specifically the line, "if (ret_val < -m_frame_interval && m_frame_interval >= m_refresh_interval)"

mythplayer questions

1. We used to allow +/- 1.5 frame intervals of AV-sync slip before adding compensation to the next trigger. Why did you change it +/- 1 refreshrate interval?

2. Why do you use the avsync_used variable instead of avsync_avg for the slip adjustment check?

3. You removed the 'lastsync = true' from the +/- 1 refreshrate interval block. Is this because you want the changes to occur on every new frame. It seems the old code would only make adjustments every other frame because of this and maybe that shouldn't have been the case.

4. Why is the prevtc != 0 check needed?

5. It seems you've added a new function to retrieve currentaudiotime and you update the variable in more instances. What's the reason for this?

I believe that covers everything. Thanks again for the patch. The bulk that has been committed makes a substantial difference for time stretch of 60fps material.

[tralph]

updated for 956b4f277fa23d8d8fc98b82f0911da642b18305

[tralph]

Closed by the following commits:

49510a8d421e22b2dab6

8fd44c8f6b0c7c2053a6

2fe9124b538524c3399e

956b4f277fa23d8d8fc9

3b102319c7ffa89fc116

[tralph]

combinded and refeshed patch for 0.24-fixes users

[Taylor Ralph]

vsync: refactor the time of day and nexttrigger calculations to simplify the code.

Patch supplied by Mark Spieth.

Refs #7964.

Changeset: 45b8f89034549ef64522bdac2d22097dde342b7f

[Taylor Ralph]

mythplayer: add LOC to VERBOSE calls for improved logging and make formatting improvements.

Refs #7964.

Changeset: 0db106f66ffb6fa238e9bd990cf871571ae40ce8

[Taylor Ralph]

mythplayer: add most of the smooth sync code.

Adds most of the smooth sync code which provides predictive frames drops when time stretch needs to playback faster than the displays refesh rate. Some limited testing shows playback at 1.5x for 60fps material is much smoother now.

Many smaller changes have been left out until further review.

Patch by Mark Spieth.

Refs #7964.

Changeset: 49065221b40b3bc928bb372f57bb63b26077a871

[Taylor Ralph]

Add AVSyncGetAudiotime() helper function to address video only playback.

Also update currentaudiotime in a few additional places.

Patch by Mark Spieth.

Refs #7964.

Changeset: 49510a8d421e22b2dab6c216fbfaa268e990557b

[Taylor Ralph]

Change avsync_adjustment to use frame_interval instead of refreshrate when audio is way behind.

Patch by Mark Spieth.

Refs #7964.

Changeset: 8fd44c8f6b0c7c2053a61c2590c781961023c8bb

[Taylor Ralph]

Avoid performing an avsync_adjustment on the very first frame when prevtc == 0.

Patch Mark Spieth.

Refs #7964.

Changeset: 2fe9124b538524c3399eefc99e23009aff818a42

[Taylor Ralph]

Comments by Mark Spieth: "Due to the fact that refresh rate is the fundamental output timing rate, not the input rate which can be anything. The av_sync adjustment needs to be with respect to the resfresh rate and not frame interval. However, no real proof that this is any better or worse but just IMO more logically correct and consistent."

"avsync_used is the minimum of the delayed estimate and the current value of avsync (with the delay from last vsync event included). This improves lock time as the smallest value is the better one to use just in case the instantaneous value deviates from the steady state. Also speeds up initial lock time as avdel is 0 initially."

Refs #7964.

Changeset: 956b4f277fa23d8d8fc98b82f0911da642b18305

[Taylor Ralph]

Add vsync_delay_clock method to compensate for hardware vsync phase issues.

This also deprecates the use of KeepPhase? in favor of handling this in AVSync.

Patch by Mark Spieth.

Closes #7964.

Changeset: 3b102319c7ffa89fc1169079151ac6670909f79c