Opened 13 years ago

Closed 13 years ago

Last modified 13 years ago

#2194 closed enhancement (fixed)

Patch to randomise mythfilldatabase times

Reported by: willu.mailingLists@… Owned by: cpinkham
Priority: minor Milestone: 0.21
Component: mythtv Version: head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

Currently myth has a housekeeper thread that can run mythfilldatabase(mfd). It carefully takes a pair of times in the preferences, and is supposed to be run between those times. And it is - it is run as soon as the housekeeping thread runs between those times. For someone who keeps their my box on all the time, this leads to mfd running in the first 5 minutes of the 'available' time period every day.

The Australian Community TV Guide was seeing load spikes in the first five minutes of popular hours.

This patch modifies the housekeeper thread so that it spreads its load. It will randomise the time it calls mfd. In particular, there is a small probability that mfd will be called each time the housekeeping thread wakes up. If mfd hasn't been called by the last 30 mins of the available window then it is called. The probability of being called is adjusted so that times are spread over the entire time range.

Attachments (3)

randomfill.patch (1.1 KB) - added by willu.mailingLists@… 13 years ago.
patch to randomise mythfilldatabase download times
housekeeper.diff (1.4 KB) - added by willu.mailingLists@… 13 years ago.
An polished version of the patch with suggestions from the mailing list incorporated
housekeeper.2.diff (1.8 KB) - added by willu.mailingLists@… 13 years ago.
Add some sanity checking and break some of the longer lines

Download all attachments as: .zip

Change History (11)

Changed 13 years ago by willu.mailingLists@…

Attachment: randomfill.patch added

patch to randomise mythfilldatabase download times

comment:1 Changed 13 years ago by cpinkham

Milestone: 0.21
Owner: changed from Isaac Richards to cpinkham
Version: head

comment:2 Changed 13 years ago by cpinkham

Doesn't this patch have the adverse side effect that jobs we want to run once a day could potentially run every 12 hours because of the following line:

longEnough = ((period * oneday) - oneday/2);

That alone would be a bad thing, but is probably prevented by what appears to be flawed logic in the random time check:

((random()%((maxhour-minhour+1)*6)) == 0)

So if maxhour is 24 and minhour is 0, then you're checking if the random number is evenly divisible by (24 - 0 + 1) * 6 which works out to 150. Those would seem to be very very low odds which would mean we'd always be running in the last half hour of our window when the "(hour == maxhour && minute > 30)" check kicked in. Doesn't seem very randomized.

Can you check the logs to see when your mythfilldatabase runs were actually occuring to show this is really random and not semi-random, but mostly in the last 30 minutes of the window?

comment:3 Changed 13 years ago by cpinkham

One more comment. The (hour == maxhour && minute > 30) check would seem to fail if maxhour is 24 since the time will never be 24:30 or beyond.

comment:4 Changed 13 years ago by willu.mailingLists@…

Just a quick pointer to my answer, and the ensuing thread on the mailing list.

Summary: Chris' objections are answered (and a slightly improved patch included), but then he says he wants to re-think the whole way the housekeeping system works.

Changed 13 years ago by willu.mailingLists@…

Attachment: housekeeper.diff added

An polished version of the patch with suggestions from the mailing list incorporated

Changed 13 years ago by willu.mailingLists@…

Attachment: housekeeper.2.diff added

Add some sanity checking and break some of the longer lines

comment:5 in reply to:  description Changed 13 years ago by anonymous

I have run the patched housekeeper.cpp and have a comment.

I admit I applied the patch without fully reviewing it and thought that I would test the a config that spanned midnight.

After noticing that mfdb did not run for a couple of days I looked at the patch properly and noticed that if starttime > endtime that the housekeeping thread will not run.

There are a couple of options that I see to handle this:

  • The help text should probably let the user know this when configuring the start and stop times.
  • The start and stop time should be set to 0 and 23 in this case
  • The patch updated to allow the time window to span midnight.

Anyway, I will fix my start stop times and keep running the patch.

Cheers.

comment:6 Changed 13 years ago by willu.mailingLists@…

The issue with a config that spans midnight is an issue with the current code that the proposed patch doesn't fix. It is a separate bug, not a regression caused by this patch. Of course, that doesn't mean it shouldn't be fixed, just that we don't want to hold up this patch waiting on a fix. See this thread for more info.

comment:7 Changed 13 years ago by cpinkham

Resolution: fixed
Status: newclosed

(In [12784]) Patch by William Uther to randomize the mythfilldatabase runtimes within the desired start/end run window. I know there were other issues and ideas brought up when this patch was first submitted, but I don't see any reason to hold up this patch while those are thought about or put on the back burner. This patch seems to work well and should help spread out the load on the grabber sources.

Closes #2194.

comment:8 Changed 13 years ago by cpinkham

(In [12785]) Fix the max value and help text on the MythFillMinHour? and MythFillMaxHour? spinbox settings to reflect the fact that the max hour is actually inclusive, so for instance if you specify 13, it means that mythfilldatabase could be run anytime before 13:59.

I fixed the .ts files here by search-and-replace so it wouldn't force a retranslation of these strings. The really should be retranslated in order to be totally accurate, but I didn't want to force it.

References #2194.

Note: See TracTickets for help on using tickets.