Opened 13 years ago

Closed 11 years ago

#3361 closed task (fixed)

Improve Preview Generation

Reported by: danielk Owned by: danielk
Priority: minor Milestone: unknown
Component: mythtv Version: head
Severity: low Keywords:
Cc: Stuart Auchterlonie Ticket locked: no

Description (last modified by danielk)

TODO's

  • Implement FILO ordering for preview generation
  • Break preview generation out into a seperate process, to avoid segfaults bringing down the whole app
  • Allow user control over where a preview is generated (backend, frontend, both) and limit the number of previews generation processes in flight at any point in time.

Attachments (5)

3361-v1.patch (42.2 KB) - added by danielk 12 years ago.
Runs preview generation in separate process to isolate ffmpeg segfaults, so far remote previews are only partially supported.
3361-v2.patch (52.2 KB) - added by danielk 12 years ago.
Adds priority queue for preview generation and a maximum number of previews to generate at once (2). When the same preview is requested again it goes to the top of the queue..
3361-v3.patch (58.3 KB) - added by danielk 12 years ago.
Updated patch, starts extending the protocol to allow for different sized previews
3361-v4.patch (67.3 KB) - added by danielk 12 years ago.
Makes remote previews and screen grabs work…
mythtv-find_file_for_preview_gen.patch (2.1 KB) - added by sphery <mtdean@…> 12 years ago.

Download all attachments as: .zip

Change History (14)

comment:1 Changed 13 years ago by danielk

Description: modified (diff)

comment:2 Changed 13 years ago by Stuart Auchterlonie

Cc: Stuart Auchterlonie added

comment:3 Changed 12 years ago by danielk

(In [13835]) Refs #3361. Exit more quickly after the 5th failed attempt to generate preview, and also only try five times when not using threaded preview.

Changed 12 years ago by danielk

Attachment: 3361-v1.patch added

Runs preview generation in separate process to isolate ffmpeg segfaults, so far remote previews are only partially supported.

Changed 12 years ago by danielk

Attachment: 3361-v2.patch added

Adds priority queue for preview generation and a maximum number of previews to generate at once (2). When the same preview is requested again it goes to the top of the queue..

Changed 12 years ago by danielk

Attachment: 3361-v3.patch added

Updated patch, starts extending the protocol to allow for different sized previews

Changed 12 years ago by danielk

Attachment: 3361-v4.patch added

Makes remote previews and screen grabs work...

comment:4 Changed 12 years ago by danielk

(In [15027]) Refs #3361. Don't try to load nonexistant icons..

I noticed this when working on the preview generation. ChannelInfo::LoadIcon?() was attempting to load icons even if the iconpath was empty. This results in a lot of unneeded chatter between the backend and frontend. I only noticed this because I thought a storage group helper function I had added was failing to find valid files, but it turned out to be an existing problem.

comment:5 Changed 12 years ago by danielk

(In [15028]) Refs #3361. Adds a little error condition debugging and error resiliance to the remotefile FileTransfer? handling in RemoteFile? class.

comment:6 Changed 12 years ago by danielk

(In [15029]) Refs #3361. Improves Preview Generation.

  • Implements a priority queue for preview generation and limits the total number of in-flight preview generation threads.
  • Runs preview generator in a separate process so that an ffmpeg crash can't take down the backend or frontend we're using.

Currently the number of preview generation threads is limited to two by the 'previewGeneratorMaxRunning' variable. I haven't yet implemented the third todo, adding user control over various aspects of the preview generation processing.

Note: This also extends the screen shot code to work with remote backends, and fixes a race condition that caused this to crash sometimes. However unless both the backend and frontend are updated to this revision a remote 'screen shot' will just return a copy of the preview image.

Changed 12 years ago by sphery <mtdean@…>

comment:7 Changed 12 years ago by sphery <mtdean@…>

The attached patch, mythtv-find_file_for_preview_gen.patch , modifies the Storage Group changes committed in [15029] based on the suggestion from Chris Pinkham at http://www.gossamer-threads.com/lists/mythtv/dev/303576#303576 . I think the changes are as he suggested and fit in well with the preview generator changes in [15029], but I'm happy to make additional changes as necessary.

Without the patch, going to the MythWeb Recorded Programs page takes forever and results in a huge CPU usage spike (mythbackend was running at 100% on my Athlon XP 2400+ (load average > 6) for so long that I just restarted it; on Chris Petersen's quad-core (Core 2?) backend, it was taking 400% and recovered after a few minutes). Individual preview generation jobs seem to work without too much issue before the patch; however, requesting a large number of previews simultaneously overwhelms mythbackend because of the recursive, recursive searches.

comment:8 Changed 12 years ago by danielk

(In [15145]) Refs #3361. Reworks the preview generator's use of storage groups so as to avoid recursive searches and hence use less CPU. This also implements some cleanups in storagegroup.{cpp,h} after running them by Chris Pinkham.

comment:9 Changed 11 years ago by danielk

Resolution: fixed
Status: newclosed

Closing, the only remaining issue has it's own ticket #3353.

Note: See TracTickets for help on using tickets.