Opened 15 years ago

Closed 14 years ago

#2544 closed patch (fixed)

Move mythfilldatabase cleanup tasks to the housekeeper

Reported by: sphery <mtdean@…> Owned by: cpinkham
Priority: minor Milestone: unknown
Component: mythtv Version: 0.20
Severity: medium Keywords:
Cc: Ticket locked: no


The attached patches move cleanup functionality from mythfilldatabase to the housekeeper. Previously, we made the assumption that all users would run mythfilldatabase. However, now that EIT data has become a valid alternative for some, many users do not need to run mythfilldatabase. Currently, they must run mythfilldatabase at least occasionally to clean expired recording rules, the oldrecorded table, and program listings. After these patches, running mythfilldatabase will no longer be necessary for EIT-only users and database cleanup will be performed on a daily basis.

The patch mythtv-cleanup_recording_rules_and_oldrecorded_in_housekeeper.patch moves the non-program-listings cleanup tasks from mythfilldatabase to the housekeeper. It's a very straightforward patch making use of the new DailyCleanup tag.

The patch mythtv-cleanup_program_listings_through_housekeeper.patch modifies the housekeeper to ensure program listings data get cleaned up as part of the housekeeper's DailyCleanup. Rather than move the cleanup code to the housekeeper, the code was left in mythfilldatabase and mythfilldatabase was modified to allow running a "--do-cleanup-only" task which simply performs the cleanup tasks. Then, the housekeeper was modified to run mythfilldatabase with the "--do-cleanup-only" argument if MythFillEnabled is 0 (if Myth doesn't automatically run mythfilldatabase) using the already in-place MFDThread code modified to allow specifying mythfilldatabase program and arguments through an MFDStruct argument (also allows specifying log since it seemed wrong to specify 2 out of 3 and in case we decide to forego logging of cleanup tasks). The reasoning behind leaving the listings data cleanup in mythfilldatabase is that the program that populates the tables should clean them up. The "--do-cleanup-only" argument is unadvertised (it's "--help" output is #if'ed out and can be either left as is, enabled, or removed) since its functionality is a normal part of mythfilldatabase's data grabbing and advertising it is likely to cause people to think they need to run it manually.

Because mythfilldatabase supports a "--no-delete" argument, which allows keeping listings data for a week (rather than expiring all listings data whose starttime was on or before the previous day), the housekeeper calls "mythfilldatabase --do-cleanup-only --no-delete". The code in mythfilldatabase that calls clearOldDBEntries() was left intact. That way, users who run mythfilldatabase on their own may decide to allow "immediate" expiration of listings data or may specify "--no-delete". (This is why the housekeeper checks to see if it even needs to run a "--do-cleanup-only" task.)

In addition, mythtv-cleanup_program_listings_through_housekeeper.patch also changes the housekeeper to find mythfilldatabase the same way we find other external myth apps--using GetInstallPrefix?() and also modifies the setting label, default value, and help text, removing the focus on the path. (This additional change was put in the same patch because of the amount of conflicts a separate patch would create.)

After one patch is applied, one of the *-after* patches should be used to apply the second patch. Because of conflicts the patch without the "after" won't apply after the other "after"-less patch. You'll get some fuzz, but they apply correctly.

Attachments (6)

mythtv-cleanup_recording_rules_and_oldrecorded_in_housekeeper.patch (4.7 KB) - added by sphery <mtdean@…> 15 years ago.
mythtv-cleanup_program_listings_through_housekeeper.patch (7.2 KB) - added by sphery <mtdean@…> 15 years ago.
mythtv-cleanup_recording_rules_and_oldrecorded_in_housekeeper-after_prog_list.patch (4.7 KB) - added by sphery <mtdean@…> 15 years ago.
mythtv-cleanup_program_listings_through_housekeeper-after_rec_rules.patch (7.3 KB) - added by sphery <mtdean@…> 15 years ago.
mythtv-cleanup_program_listings_through_housekeeper-post_11619.patch (1.8 KB) - added by sphery <mtdean@…> 14 years ago.
mythtv-programdata-cleanup_message.patch (478 bytes) - added by sphery <mtdean@…> 14 years ago.

Download all attachments as: .zip

Change History (9)

Changed 15 years ago by sphery <mtdean@…>

Changed 15 years ago by sphery <mtdean@…>

Changed 15 years ago by sphery <mtdean@…>

Changed 15 years ago by sphery <mtdean@…>

comment:1 Changed 15 years ago by cpinkham

Owner: changed from Isaac Richards to cpinkham

comment:2 Changed 14 years ago by sphery <mtdean@…>

After Daniel moved all of the mythfilldatabase functionality around ([11619]), the previously supplied patches are no longer valid. The updated patch, mythtv-cleanup_program_listings_through_housekeeper-post_11619.patch , should be used now, instead.

The new patch is significantly simpler than the previous patches, since the program listings table cleanup is now in libmythtv. There's no longer a modification to the way mythfilldatabase is run or the need for the MFDStruct for passing arguments to be used for calling mythfilldatabse in the MFDThread. Also, there's no need for adding a "--do-cleanup-only" argument to mythfilldatabase since mythfilldatabase now simply coordinates the population/cleanup of program listings--we can completely bypass mythfilldatabase for the cleanup.

Also, the new patch does not break out the cleanup of program listings data from cleanup of recording rules and cleanup of oldrecorded. Cleanup of listings, recording rules, and oldrecord remains together in clearOldDBEntries() (which was simply moved from filldata.cpp to programdata.cpp in Daniel's commit). After this patch is committed, I'll see about moving them (as a pedantic change--since they're not actually program listings data) to a more appropriate location (will discuss this more in IRC).

Finally, because there is no longer a need to change how mythfilldatabase is run, changing how mythfilldatabase is found no longer conflicts with these changes. Therefore, I created another ticket, #2622, to change how mythfilldatabase is found.

I'm also attaching a patch, mythtv-programdata-cleanup_message.patch, which prefixes the message, "Keeping 7 days of data." with, "Listings Cleanup:". Without this change, the message, output by clearOldDBEntries() and written to the backend log when cleanup is performed by the housekeeper, would likely scare users, who may think they will lose recordings and other data that's older than 7 days.

Changed 14 years ago by sphery <mtdean@…>

Changed 14 years ago by sphery <mtdean@…>

comment:3 Changed 14 years ago by cpinkham

Resolution: fixed
Status: newclosed

(In [12189]) Call ProgramData::clearOldDBEntries() from within the housekeeper once a day so that people who use only EIT data don't have to run mythfilldatabase just to cleanup old data in the database.

Patch by Michael T. Dean.

Closes #2544.

Note: See TracTickets for help on using tickets.