Opened 14 years ago

Closed 11 years ago

#843 closed task (fixed)

Implement accelerated Picture-in-Picture support

Reported by: danielk Owned by: skamithi
Priority: minor Milestone: 0.22
Component: mythtv Version: head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

After I get #827 (FF-REW >3x) working again I plan to fix Picture-in-Picture support.

The code for this is currently disabled in tv_play, and may need some reworking to work with the new LiveTV code.

Attachments (8)

pipplayer.3.diff (76.2 KB) - added by skamithi 13 years ago.
pipplayer.2.diff created using the latest mythtv-vid update.
pipplayer.4.diff (77.3 KB) - added by skamithi 13 years ago.
some bug fixes. still have to fix tv:swappip and tv::toggleactivewindow.. also need to check main xvideo surface if it can support the pip.
pipplayer.5.diff (78.7 KB) - added by skamithi 13 years ago.
removed my attempt to control pip fps. needs more work. turn on deinterlacing for pip. set override filter list to "linearblend".(not sure if this is the right thing). tv::toggleactivewindow enabled..only works when the main window is live tv.
pipplayer.6.diff (84.3 KB) - added by skamithi 13 years ago.
more bug fixes. toggleactivewindow and swappip should be fairly stable now. haven't tried all the different key actions to see what happens..added VideoOutput::IsResolutionSupported? . ran to check that main XV surface supports the specified resolution. also resolved some issues where pipplayer didn't die cleanly.
pipplayer.7.diff (86.6 KB) - added by skamithi 13 years ago.
2nd attempt to control fps of pip..cuts fps in 1/2. see code changes in nvp::avsync. works for recorded shows. doesn't work well for live tv pip. problem with livetv.. it doesn't report framesplayed consistently until i ffw/rew or hit pause on the pip. trying to resolve this issue.
pipplayer.diff (88.8 KB) - added by skamithi 13 years ago.
playbackbox preview set to play 30sec of the show. if using nullvideoout and pip is live tv, increase prebuffer to 20 from 4. this helps the pip fps reduction work for software scaled live tv pip. couple of bug fixes. added some user notifications (pip failure, toggle active window change, toggle denial if main window is a recording)
pipplayer.2.diff (88.8 KB) - added by skamithi 13 years ago.
force usleep videosync for pips. other videosync methods seem to lock up the pip. fixed aspect issue where there is black bar on the top and bottom if aspect ratio is forced. will do more testing tomorrow on my nvidia box. most dev done on ati PC.
pipplayer.8.diff (15.8 KB) - added by skamithi 13 years ago.
get chromakey osd to go over pip. code untested. also has a couple of bug fixes.

Download all attachments as: .zip

Change History (101)

comment:1 Changed 14 years ago by danielk

(In [8364]) References #843. This re-enables PiP.

This does not re-enable TV::SwapPIP(), this function as written was broken before and is still broken. It only works if the two recorders share the swapped channels (same lineup) or a extra recorders are available that support the swapped channels.

comment:2 Changed 14 years ago by danielk

(In [8365]) References #843. Initial SwapPIP() support.

This is just the needed tv_play changes. The tv_rec change should be ready tomorrow. This works, but the recording quality is not so good if using a pip aware recorder.

comment:3 Changed 14 years ago by danielk

(In [8368]) References #843.

tv_rec portion of SwapPIP(). This just makes the recordings full size.

comment:4 Changed 14 years ago by danielk

(In [8373]) References #843. Reverts [8367] and part of [8365], disables SwapPIP().

comment:5 Changed 14 years ago by danielk

(In [8500]) References #843.

This makes the recorder switching internal to tv_play so that we handle the LiveTVChain completely within tv_play. This will allow PiP recorder swapping to be implemented safely.

comment:6 Changed 14 years ago by danielk

(In [8501]) References #843.

Re-enables PiP swapping.

comment:7 Changed 14 years ago by danielk

Milestone: 0.190.20

The remaining problem is simply the lack of hardware accellerated PiP so I'm moving this ticket to 0.20, since HW accllerated PiP doesn't really have to be in 0.19.

But once I fix my remaining 'needed to fix before 0.19' tickets, I'll revisit this.

comment:8 Changed 13 years ago by danielk

Summary: Fix Picture-in-Picture supportImplement accelerated Picture-in-Picture support

comment:9 Changed 13 years ago by danielk

Milestone: 0.200.21

comment:10 Changed 13 years ago by danielk

Owner: changed from danielk to skamithi

Stanley is looking at this...

Changed 13 years ago by skamithi

Attachment: pipplayer.3.diff added

pipplayer.2.diff created using the latest mythtv-vid update.

comment:11 Changed 13 years ago by skamithi

objectives for this ticket..developed by danielk. Includes some quotes from danielk.:

1st/ PIP Code consolidation

There are three "PiP" implementations, the regular PiP, the video window in the LiveTV EPG and the preview video window in the recording browser. This means there is code duplication galore and the preview video code is S-C-A-R-Y.

2nd/ As Is, PiP only supports one PiP window, seeing what is

on multiple tuners at once would be nice. a properly encapsulated PiP player would be assist with this.

3rd/ As Is the PiP window doesn't properly handle things if the

framerate of the PiP video and the main video don't match.

4rd/ Acceleration

  • There is a limit to how many XvMC or XVideo ports you can open, you need to be able to fall back to another output method, at minimum to the software scaling and RGB conversion we use now.
  • Some XvMC/XVideo drivers don't allow you to shrink the playback window beyond a certain point, so these drivers need to be detected and worked around (or config option).
  • Some XvMC/XVideo implementations may advertise more scalers than they really support, so these need to be detected and worked around (or config option).
  • If the XvMC/XVideo uses chromakeying, you need a different key color for each video.
  • If the XvMC/XVideo don't use chromakeying, you need to open a seperate X window for the PiP window and ensure that it stays on top, in the right place, and doesn't steal focus.
  • Deinterlacing.. it would be really nice to be able to de-interlace the PiP window. How this is done depends on the acceleration method, but in many cases this should be easy since you are shrinking the video and can just avoid sampling the even or odd rows.

"

Changed 13 years ago by skamithi

Attachment: pipplayer.4.diff added

some bug fixes. still have to fix tv:swappip and tv::toggleactivewindow.. also need to check main xvideo surface if it can support the pip.

Changed 13 years ago by skamithi

Attachment: pipplayer.5.diff added

removed my attempt to control pip fps. needs more work. turn on deinterlacing for pip. set override filter list to "linearblend".(not sure if this is the right thing). tv::toggleactivewindow enabled..only works when the main window is live tv.

Changed 13 years ago by skamithi

Attachment: pipplayer.6.diff added

more bug fixes. toggleactivewindow and swappip should be fairly stable now. haven't tried all the different key actions to see what happens..added VideoOutput::IsResolutionSupported? . ran to check that main XV surface supports the specified resolution. also resolved some issues where pipplayer didn't die cleanly.

Changed 13 years ago by skamithi

Attachment: pipplayer.7.diff added

2nd attempt to control fps of pip..cuts fps in 1/2. see code changes in nvp::avsync. works for recorded shows. doesn't work well for live tv pip. problem with livetv.. it doesn't report framesplayed consistently until i ffw/rew or hit pause on the pip. trying to resolve this issue.

Changed 13 years ago by skamithi

Attachment: pipplayer.diff added

playbackbox preview set to play 30sec of the show. if using nullvideoout and pip is live tv, increase prebuffer to 20 from 4. this helps the pip fps reduction work for software scaled live tv pip. couple of bug fixes. added some user notifications (pip failure, toggle active window change, toggle denial if main window is a recording)

Changed 13 years ago by skamithi

Attachment: pipplayer.2.diff added

force usleep videosync for pips. other videosync methods seem to lock up the pip. fixed aspect issue where there is black bar on the top and bottom if aspect ratio is forced. will do more testing tomorrow on my nvidia box. most dev done on ati PC.

comment:12 Changed 13 years ago by skamithi

when using xvmc videoutput and chromakey osd for the main window..accel pip will generate lots of decoding error. last issue to resolve before i commit.

comment:13 Changed 13 years ago by skamithi

pip on xvmc screen using a fx5200 wants to use Xv..but bestsupportedcodec function was returning the wrong codec type. resolved this.

testing on the nvidia system and wide screen tv revealed the following problems. a) pip for playbackbox is cut short on the left side. aspect ratio issues i believe. b) swappip when xvmc is using the main window doesn't work. swap doesn't happen cleanly. works fine with the software scaled pip. c) pip when using xvmc and chromaosd still resides on top of the osd. not sure if there is something i'm doing wrong or this is the way it should be.

comment:14 Changed 13 years ago by skamithi

(In [11805]) Refs #843. accel pip support. still some issues to contend with. see the ticket for the latest update. opengl pip support is been worked on in #2649.

comment:15 Changed 13 years ago by skamithi

Status: newassigned

swappip should work now.

comment:16 Changed 13 years ago by skamithi

togglepipview may be really slow cause it tries to use xshm. next commit will have this fix, plus resolve the problem where the chromaosd is below the pip.

Changed 13 years ago by skamithi

Attachment: pipplayer.8.diff added

get chromakey osd to go over pip. code untested. also has a couple of bug fixes.

comment:17 Changed 13 years ago by skamithi

(In [11833]) Refs #843. prevent XV PIP if OSD is chromakey and the surface doesn't support chromakey. fixed a few bugs. still tracking down a problem where live tv pip hangs on a channel change.

comment:18 Changed 13 years ago by skamithi

(In [12205]) Refs #843. update the pip sizing function. thx for mark kendall for the patch. switch activenvp back to the main nvp if the pip nvp is active, and the OSD menu is accessed. new function to determine if a software PIP is allowed. for now, opengl or XV accelerated main surfaces will be supported.

comment:19 Changed 13 years ago by danielk

(In [12770]) Refs #843. Refs #2287. This temporarily reverts out the accelerated PiP.

This is causing various problems with PiP. I'm going to make some simplifying changes to tv_play video player handling then apply this back in smaller chunks which should be easier to debug.

comment:20 Changed 12 years ago by skamithi

(In [13728]) Refs #843. Create mythtv-pip branch from mythtv-vid for developing/testing new PiP code.

comment:21 Changed 12 years ago by skamithi

(In [13729]) Refs #843. remove pipplayer.cpp from mythtv-vid for now. PiP development moved to mythtv-pip branch for now.

comment:22 Changed 12 years ago by skamithi

add the playercontext class and change necessary code to use it. entire patch created by danielk, it'll make swapping of the pip and main picture easier to manage. just testing it and will commit it in chunks.

comment:23 Changed 12 years ago by skamithi

(In [13850]) Refs #843. change code to use playercontext class. still buggy and testing. PiP has been disabled while this testing continues.

comment:24 Changed 12 years ago by skamithi

(In [13919]) Refs #843. mythtv-pip: move tv state variable to playercontext. this should hopefully make managing TV state changes for multiple playercontexts easier. live tv should now work again. will continue testing/fixing bugs or Todos for 2-3 weeks before working on stage2 - getting accel pip for Live TV.

comment:25 Changed 12 years ago by skamithi

(In [14015]) Refs #843. mythtv-pip : move some dialog boxes out of NVP to prevent hang after applying playercontext code.

comment:26 Changed 12 years ago by skamithi

(In [14117]) Refs #843. mythtv-pip update. apply r14012:14115 from mythtv-vid

comment:27 Changed 12 years ago by skamithi

(In [14118]) Refs #843. implement current PIP features using playercontext code. next step is to implement hw accel pip.

comment:28 Changed 12 years ago by skamithi

(In [14207]) Refs #843. mythtv-pip: fix some bugs relating to switching cards and changing channels. also made some cosmetic changes to some functions.

comment:29 Changed 12 years ago by skamithi

(In [14279]) Refs #843. mythtv-pip. make previous channel code work better in a playercontext environment. also includes other minor fixes (mainly typos).

comment:30 Changed 12 years ago by skamithi

(In [14375]) Refs #843. started work on hw accel PIP. allow live tv PIP startup when watching a pre recorded show. hw accel PIP will resize based on PIP aspect. made attempt to reduce PIP fps and apply onefield filter on PIP. beginning work on restricting when hw accel PIP will be used. can only test XVideo, XVMC and OpenGL. fixed issue where live tv PIP hangs when the show in the PIP changes. also with recorded show PIP it should just exit when the show is done. because of using player contexts some features such has prompting the user before the PIP show quits can be easily done. don't know effects of code on IVTV or mac acceleration.

comment:31 Changed 12 years ago by skamithi

(In [14391]) Refs #843. after testing with xv, xvmc-blit and opengl renderers, tv pip will use nullvideo. except for xvmc-blit. for opengl with softblend renderer and xvmc-blit renderer OSD will be below PIP. best renderer for PIP so far is opengl with opengl2 osd. going to test xvmc-opengl renderer, then will work on adding hw-accel pip to playbackbox code. still got some user notification stuff to work on with the TV PIP.

comment:32 Changed 12 years ago by skamithi

(In [14460]) Refs #843 merge r14287:14457 from mythtv-vid.

comment:33 Changed 12 years ago by skamithi

(In [14481]) Refs #843. merge r14458:14479 from mythtv-vid

comment:34 Changed 12 years ago by skamithi

(In [14482]) Refs #843. implement playbackbox PIP using player contexts. currently playbackbox PIP uses software scaling. still checking if opengl or other renderers would be good options as well. fixed some compile issues and osd lockups. still not worked out a PIP solution for xvmc-opengl renderer.

comment:35 Changed 12 years ago by skamithi

(In [14484]) Refs #843. fix typo when checking if null video NVP should be used or not for the PIP.

comment:36 Changed 12 years ago by skamithi

(In [14508]) Refs #843. allow HW accel playback preview. option is disabled by default. video renderer of playback preview is determined by CPU profiles. also fixed a couple of bugs.

comment:37 Changed 12 years ago by skamithi

(In [14511]) Refs #843. when using the embedded playbackbox (#2426) allow user to start recording as a PIP.

comment:38 Changed 12 years ago by skamithi

(In [14565]) Refs #843. disable attempts to have a pip solution for xvmc-blit and xvmc-opengl at this point. also changed preview video timers in playbackbox to only play the preview video of selected show once and then stop. this should spare the log from filling up by repeatedly showing the preview window if one leaves mythtv in the playbackbox window for several hours. preview will play for 1 minute.

comment:39 Changed 12 years ago by skamithi

(In [14573]) Refs #843 mythtv-pip update: merge r14457:14572 from mythtv-vid.

comment:40 Changed 12 years ago by skamithi

(In [14577]) Refs #843. fix PIP positioning problems with the "start a recording as a pip feature". also resolve some compile problems after last merge from mythtv-vid. will merge current mythtv-pip branch back to mythtv-vid later today. will continue working on picture by picture and xvmc pip support in mythtv-pip.

comment:41 Changed 12 years ago by skamithi

(In [14580]) Refs #843. Refs #2287. merge mythtv-pip to mythtv-vid.

comment:42 Changed 12 years ago by skamithi

(In [14591]) Refs #2287. Refs #843 fix use of the player context in mythcommflag.

comment:43 Changed 12 years ago by skamithi

(In [14620]) Refs #843. fix issue where playback box preview video doesn't start when playbackbox window is first opened.

comment:44 Changed 12 years ago by skamithi

(In [14621]) Refs #843. merge r14581:r14620 from mythtv-vid to mythtv-pip

comment:45 Changed 12 years ago by skamithi

(In [14651]) Refs #843. Stage 1 of picture by picture support. activate pbp through picture-by-picture option in the embedded playbackbox. live-tv support is next, plus add capability to flip between PBP and PIP and to swap PBP. pbp will not be allowed if both pictures cannot use hw-accel renderer.

comment:46 Changed 12 years ago by skamithi

(In [14653]) Refs #843. mythtv-vid: fix slowness in playbackbox with preview video enabled after mythtv-pip merge. problem reported by mark kendall. also resolves problem where preview pixmap is enabled whether it is turned on or not.

comment:47 Changed 12 years ago by skamithi

(In [14662]) Refs #843. merge r14621:r14661 from mythtv-vid to mythtv-pip

comment:48 Changed 12 years ago by skamithi

(In [14663]) Refs #843. stage 2 of Picture by picture support. allowing swapping of PBP windows. this maps to SWAPPIP the same action as swapping PIPs. Code difference between the 2 is minimal so decided to keep the action mapped to the same keystroke. Changing active PBP window (TOGGLEACTIVEWINDOW) enables the sound for that PBP window, and disables the sound for the inactive window.

comment:49 Changed 12 years ago by skamithi

(In [14665]) Refs #843. add livetv support for picture by picture. will start working on xvmc PiP solutions.

comment:50 Changed 12 years ago by skamithi

(In [14667]) Refs #843. make PIP use usleep video sync. for some reason opengl vsync causes hw accel preview video to hang.

comment:51 Changed 12 years ago by skamithi

(In [14669]) Refs #843. fix attempt to reduce fps by half for hw accel pip. was causing playback preview video hangs.

comment:52 Changed 12 years ago by skamithi

(In [14672]) Refs #843. allow Xv and opengl pip to still persist after switching tv tuners on the main window. also done some minor cosmetic changes with player context definitions.

comment:53 Changed 12 years ago by skamithi

(In [14676]) Refs #843. draw a border around software scaled tv PIP when it is the active window. also did some minor cosmetic changes with showpip function.

comment:54 Changed 12 years ago by skamithi

(In [14677]) Refs #843. allow pip to switch tv tuners during live tv channel change without crashing mythtv.

comment:55 Changed 12 years ago by skamithi

(In [14678]) Refs #843. Refs #2287. change method for embedding TV video in guidegrid and playbackbox. Allows opengl/xv/xvmc-blit rendered TV windows to be embedded. xvmc-opengl doesn't work yet cause xvmc-opengl playback is broken. talking to danielk, there are some thread safety issues with embedded video playback to deal with.

comment:56 Changed 12 years ago by skamithi

(In [14696]) Refs #843 Refs #2287. mythtv-vid: fix bug with playercontext code that was causing hang during live tv window destruction.

comment:57 Changed 12 years ago by skamithi

(In [14710]) Refs #843 Refs #2287. remove playercontext code in mythcommflag. causing intermittent commflagging failures. may fix and re-add later.

comment:58 Changed 12 years ago by skamithi

(In [14719]) Refs #843 merge r14622:14718 from mythtv-vid to mythtv-pip

comment:59 Changed 12 years ago by skamithi

(In [14740]) Refs #843. add picture-by-picture support for opengl renderer. made some cosmetic changes to a couple of functions and consolidated some playercontext code.

comment:60 Changed 12 years ago by skamithi

(In [14775]) Refs #2287 Refs #843. mythtv-vid: playercontext code: fix crash while watching live tv.

comment:61 Changed 12 years ago by skamithi

(In [14786]) Refs #843. mythtv-pip: fix starting/stopping pbp from osd menu, prevent screen saver restore after pbp transition and prevent embedding when pbp is activated. disable sound activation for secondary pbp when changing active window. enabling sound on 2nd pic causes sync issues.

comment:62 Changed 12 years ago by skamithi

(In [14833]) Refs #843 merge 14719:14832 from mythtv-vid to mythtv-pip

comment:63 Changed 12 years ago by skamithi

(In [14961]) Refs #843. couple of pip fixes in playercontext code. force onefield filter on pip frames before ShowPIP is run on it, prevent hw accel pip in playbackbox from clearing bookmark,get playbackbox pip to restart after exiting show and you in the watch recording window.

comment:64 Changed 12 years ago by skamithi

(In [14970]) Refs #843. merges r14663:r14969 from mythtv-vid to mythtv-pip.

comment:65 Changed 12 years ago by skamithi

(In [15162]) Refs #843. merge r14970:15159 from mythtv-vid to mythtv-pip

comment:66 Changed 12 years ago by skamithi

(In [15459]) Refs #843. merge r15160:15457 from mythtv-vid to mythtv-pip.

comment:67 Changed 12 years ago by skamithi

(In [15469]) Refs #843. add support for changing between PIP and PBP on the fly. refactor TV::ProcessKeyPress?() as suggested by danielk in the original playercontext patch. will test latest update for a couple of weeks then merge mythtv-pip back into mythtv-vid.

comment:68 Changed 12 years ago by skamithi

(In [15508]) Refs #843. fix osd size issue when using opengl PBP with opengl osd.

comment:69 Changed 12 years ago by skamithi

(In [15532]) Refs #2287 Refs #843. disable opengl vsync functions when the video is embedded. helps prevent some hangs during video resizing when opengl vsync is enabled.

comment:70 Changed 12 years ago by skamithi

(In [15536]) Refs #843 Refs #843. merge r15458:15535 from mythtv-vid to mythtv-pip

comment:71 Changed 12 years ago by skamithi

(In [15587]) Refs #843. provide colour border around opengl PIP when it is the active window.

comment:72 Changed 12 years ago by skamithi

(In [15588]) Refs #843 Ref #2287. remove some extra calls to openglcontext->MakeCurrent?()

comment:73 Changed 12 years ago by skamithi

(In [15589]) Refs #843. rename pip state variable in videoout* functions.

comment:74 Changed 12 years ago by skamithi

(In [15590]) Refs #843. fix problem where opengl pip border continues to be drawn when restarted and resolve some pip osd menu display issues when using PBP.

comment:75 Changed 12 years ago by skamithi

(In [15601]) Refs #843. Refs #2287. last merge from mythtv-pip to mythtv-vid before deleting mythtv-pip.

comment:76 Changed 12 years ago by skamithi

(In [15625]) Refs #843. delete mythtv-pip branch.

comment:77 Changed 12 years ago by skamithi

(In [15725]) Refs #843: fix bug that causes dpms not to function when exiting the tv player when in Picture by picture mode.

comment:78 Changed 12 years ago by danielk

Milestone: 0.210.22

comment:79 Changed 12 years ago by skamithi

(In [16320]) Refs #843: mythtv-vid: [15725] didn't solve the DPMS issue when pip is deactivated. this solution seems to work better.

comment:80 Changed 12 years ago by skamithi

(In [16402]) Refs #843. Refs #2287. mythtv-vid: part of [13617] was causing a hang when calling EPG or embedded playbackbox. modified the code to prevent hang. also fix bookmark setting issue when swapping PiP/PbP.

comment:81 Changed 12 years ago by skamithi

(In [16575]) Refs #843. mythtv-vid. update the program's flags after stopping the preview player. also apply stop preview player action to any keypress.

comment:82 Changed 12 years ago by skamithi

(In [16632]) Refs #843. mythtv-vid: resolve hang when exiting picture by picture. also make sure audio is not activated for picture by picture.

comment:83 Changed 12 years ago by skamithi

(In [16633]) Refs #843. mythtv-vid: start preview player at the beginning and not at the bookmark.

comment:84 Changed 12 years ago by skamithi

(In [16727]) Refs #843 Refs #2287. mythtv-vid: adding some verbose messages regarding picture by picture and the pip state variable.also resolve some minor errors in the playercontext code

comment:85 Changed 12 years ago by skamithi

(In [16832]) Refs #843. resolve crash in ttfont.cpp when using picture by picture and the XV renderer. also change location where bookmarking is enabled when using picture by picture

comment:86 Changed 11 years ago by skamithi

(In [17386]) Refs #843, Refs #2287. mythtv-vid: changes made to the preview video for recorded shows in mythtv-vid branch don't work after qt4 update . working on fixing this. for now disabling the feature.

comment:87 Changed 11 years ago by skamithi

(In [17393]) Refs #843, Refs #2287. mythtv-vid: fixes preview video changes after qt4 upgrade.

comment:88 Changed 11 years ago by skamithi

(In [17760]) Refs #843. Refs #2287. mythtv-vid: make opengl PBP behave like the Xv PBP. OSD does not overlay over both pictures. OSD stays within only the active picture.

comment:89 Changed 11 years ago by skamithi

(In [17766]) Refs #843, Refs #2287. mythtv-vid. fix swapping between PIP & PBP after applying [17760]

comment:90 Changed 11 years ago by skamithi

(In [17787]) Refs #843, Refs #2287. mythtv-vid. enable 2% vertical scaling for the hardware rendered preview player and software scaled PiP.

comment:91 Changed 11 years ago by skamithi

(In [17904]) Refs #2287, Refs #843. fix some wrong logic introduced by me during a mythtv-vid merge [ 17848].

comment:92 Changed 11 years ago by skamithi

(In [17994]) Refs #843, Refs #2287. mythtv-vid: remove code I put in that was supposed to prevent resizing of opengl rendered picture during tv playback startup. It doesn't work. currently if you start EPG at tv playback you'll see nothing in the resized tv window and when you exit the EPG the picture is not resized again properly. Looking for a different solution that will work.

comment:93 Changed 11 years ago by skamithi

Resolution: fixed
Status: acceptedclosed

updates to PIP have been synced in trunk now.

Note: See TracTickets for help on using tickets.