Opened 11 years ago

Closed 9 years ago

#5818 closed enhancement (duplicate)

Patch for 'filemarkup table is inefficient'

Reported by: simonwalls@… Owned by: cpinkham
Priority: minor Milestone: unknown
Component: MythTV - General Version: head
Severity: low Keywords: videos
Cc: Ticket locked: no

Description

This is 'the start' of a patch for the minor issue in Ticket #5109 with the filemarkup table containing the full pathname of the file for every single mark. Particularly when MARK_GOP_BYFRAME is in use, eg. for mpeg-2's, there are many thousands of marks for a movie file.

This patch changes the usage of filemarkup.filename to be the intid of the video (as found in videometadata.intid). It is against 0.21-fixes SVN 17450.

It does not purport to be complete and suitable for inclusion, since to do it properly, a field name change would be required in the database schema (filemarkup.filename -> filemarkup.intid) complete with upgrade of the schema number etc etc in the appropriate file(s) eg. mythvideo/dbcheck.cpp. It might be the case, although I don't think so at this time, that some other plugins require changes.

However, the basics work despite those limitations and is in use on my main system. I'm raising the ticket to get the patch recorded and available to others, particularly the original submitter of Ticket #5109.

When I have more time, I can look at ensuring this patch is complete (all other plugins do or don't require mods) and then including the date of last rebuild of filemarkup in the videometadata table, to aid the other latent problem I wrote of in Ticket #5109.

My database went from 63Mb to 74Mb with 33 mpeg2 files (including 12 movies) added to the filemarkup table (162,000 rows in table). Without the patch an almost identical size increase occurred with only 3 movies indexed.

A script to rebuild the seektables of all .mpg videos using mythtranscode (with mpeg2 option) is included. (The original contents of filemarkup are not dropped, they are retained in the table but will not be found, since the access to the filemarkup table now looks for the numeric 'intid' from the videometadata table, in that column).

Attachments (4)

SWpatch_filemarkup_111008 (8.0 KB) - added by simonwalls@… 11 years ago.
Patch to 0.21-fixes SVN 17450 to improve efficiency of filemarkup table
ticket_rebuild_seeks_script.sh (988 bytes) - added by simonwalls@… 11 years ago.
Shell script to run mythtranscode for all .mpg files in a directory to rebuild their seektables (using --mpeg2 option)
SWpatch_filemarkup_121008 (9.5 KB) - added by simonwalls@… 11 years ago.
Improved patch to 0.21-fixes SVN 17450
SWpatch_filemarkup_131008 (9.5 KB) - added by simonwalls@… 11 years ago.
Commented out the debug-output statements

Download all attachments as: .zip

Change History (10)

Changed 11 years ago by simonwalls@…

Attachment: SWpatch_filemarkup_111008 added

Patch to 0.21-fixes SVN 17450 to improve efficiency of filemarkup table

Changed 11 years ago by simonwalls@…

Shell script to run mythtranscode for all .mpg files in a directory to rebuild their seektables (using --mpeg2 option)

Changed 11 years ago by simonwalls@…

Attachment: SWpatch_filemarkup_121008 added

Improved patch to 0.21-fixes SVN 17450

Changed 11 years ago by simonwalls@…

Attachment: SWpatch_filemarkup_131008 added

Commented out the debug-output statements

comment:1 Changed 10 years ago by cpinkham

Milestone: 0.22unknown
Owner: changed from Isaac Richards to cpinkham
Status: newassigned

The conversion from filename -> int will be done as part of some larger changes post-0.22, so I'm changing the milestone for this ticket.

comment:2 Changed 10 years ago by cpinkham

Milestone: unknown0.23

comment:3 Changed 10 years ago by stuartm

Component: mythtvPlugin - MythVideo

comment:4 Changed 10 years ago by cpinkham

Milestone: 0.230.24
Version: 0.21-fixeshead

Moving some things to 0.24 since we're almost at the 0.23 feature freeze.

comment:5 Changed 9 years ago by robertm

Component: Plugin - MythVideoMythTV - General

Moving to general since it pertains more directly to the DB schema updates Chris has pending at this point.

comment:6 Changed 9 years ago by cpinkham

Milestone: 0.24unknown
Resolution: duplicate
Status: assignedclosed

This feature will be accomplished as part of a larger file and seek table database reorg that is in the works. There is no need to keep this ticket open so I'm closing it to help cleanup Trac. Marking ticket as 'duplicate' even though it's not because it's not really a "won't fix" or any other category.

Note: See TracTickets for help on using tickets.