Opened 16 years ago
Closed 15 years ago
Last modified 15 years ago
#5746 closed patch (fixed)
With multiple disk of different sizes, place new recording on partition which will have the lowest priority files deleted
Reported by: | Owned by: | cpinkham | |
---|---|---|---|
Priority: | minor | Milestone: | unknown |
Component: | mythtv | Version: | 0.21-fixes |
Severity: | low | Keywords: | mutple disk, expire, |
Cc: | Rob Smith, stuartm | Ticket locked: | no |
Description
When using multiple disks of different sizes the new recording is not always placed on the disk which has the lowest priority files to be expired.
This patch adds a new pass to the placement algorithm which simulates expiring the old files in the users required order. When a partion is found with enough space that partition is selected for the new file.
pass 1: As before - find any partition with enough space to fit the file pass 2: simulate expiration until a partition is found with enough space pass 3: (old pass 2) - find any partition that will take the file.
I've tested this patch for over a month (it takes that long) with 1 TB and 1 500 GB disk.
Attachments (2)
Change History (9)
Changed 16 years ago by
Attachment: | file_placement.patch added |
---|
comment:1 Changed 15 years ago by
Cc: | Rob Smith added |
---|
Changed 15 years ago by
Attachment: | file_placement.2.patch added |
---|
comment:2 Changed 15 years ago by
Updated patch "file_placement.2.patch":
- Made estimated placement size of files 1/3 bigger. This fixes a minor bug in all 3 passes where a file system may be selected as having enough free space but because the auto-expire algorithm assumes 1/3 more space over what is required and would expire a file.
- When I file can not be found in pass 2 it is ignored instead of aborting.
- verbosity changed to VB_FILE on some printouts and each pass indicated. Run mythbackend with "-v file" when reporting problems.
- fixed memory leak: freed some memory allocated in pass 2 as a result of calling GetAllExpiring?
comment:3 Changed 15 years ago by
I am pleased to report good preliminary results in the first couple of days of testing the newest version of John Fulton's patch. All recent "Expired program" entries in System Status|Log Entries are for either deleted or expire-after-x recordings, and the Deleted recording group has been whittled down to only a few entries. This is consistent with expectations for a setup, like mine, always running at maximum capacity.
comment:4 Changed 15 years ago by
Owner: | changed from Isaac Richards to cpinkham |
---|---|
Status: | new → assigned |
This doesn't appear to take into account filesystems which are not accessible on the master backend, it only does a CheckFile?() looking for the file. That needs to be taken into account before it can be committed. I also would want this to be an option. I see 2 main balancing ideas behind Storage Groups. The current mode is to balance I/O load across filesystems, that takes precendence over disk space. This mode will make disk space and expirable recordings take precedence over I/O (weight) on a filesystem. I think that there should be an option to control these two modes. I think this second mode should set the starting weight of local filesystems to 0 and might want to sort the directories by free space first then by weight.
If you want to discuss further, reply on the mailing list. I'm all for adding a feature like this, but don't want it to take away from current "balance the I/O load" functionality for those that still prefer the current mode (like myself).
comment:5 Changed 15 years ago by
Cc: | stuartm added |
---|
comment:6 Changed 15 years ago by
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
(In [21225]) Add some more intelligence to the Storage Scheduler.
When using the new "Balance Free Space" method, the Storage Scheduler will now perform an intermediate step if it can not find a directory with enough free space to hold the recording. Previously, the scheduler would go through the directory list again and put the recording on the first acceptable directory in the list and hope that something could be expired from that directory. The new middle pass will go through the directory list again looking at what can be expired from each directory to find out the best place to put the recording based on what recording(s) may have to be expired to make room for the new recording.
One warning.... The corner case of full drives and multiple recordings starting at once will still not be handled effectly by this 2nd pass. The 2nd recording will probably be put on the same filesystem as the first since "free space" is the deciding factor. It may be that the "free space"-only method may work better to use a fuzzy comparison and then take weight into account which would help balance out multiple recordings in this scenario.
This patch also does 2 things regarding the default value of the StorageScheduler? setting:
1) The default method on the setup screen is now 'Balanced Free Space'. 2) A DB upgrade inserts a setting into the database to set the method
to 'Combination' so that users upgrading from previous versions will see the same scheduling logic they have been seeing. Users can change the value in mythtv-setup anytime and restart the master backend for the new value to take effect.
Closes #5746 using modified patch originally by John R Fulton.
patch to scheduler.cpp for mythbackend