EPG slow in latest SVN

[anonymous]

In the latest SVN (14490) after the mythvid branch merge, using BOB de-interlacing causes the EPG to be painfully slow when watching LiveTV. Any movement in the EPG (single line, PGUP,PGDN, or jump to channel) can take upwards of 5-10 seconds to complete. Turning off de-interlacing or selecting one of the other methods makes movement in the EPG almost instantaneous or within a second. The machine is not *extremely* beefy, but can handle normal SD content w/ BOB or HD content w/ XvMC just fine and never used to have this problem prior to the merge.

[fracmak]

I've been experiencing the same thing, except in reverse, 1080i with bob deint works just fine with the EPG, cpu usage is around 30-40% for mythfrontend, and like 1% for Xorg, but 720p video with no filters takes up the same cpu for mythfrontend, but Xorg takes up the rest of the cpu, ~60%. There's not much noticeable delay on moving around the EPG until you decide to scroll the EPG, then you start to get a 5-10 second delay for mythtv responding to keyboard buttons.

[danielk]

I can't reproduce this, can you provide me the "mythfrontend -v playback" log + tell me what kind of hardware you are using?

[fracmak]

mythfrontend -v playback

[fracmak]

I've attached the output, the hardware it's running on is an AMD64 3500, using a GeForce? FX 5200 on an AV8 Deluxe motherboard running Fedora 7. Playing video is just fine, but once I pull up the EPG, cpu usage spikes and I get a large number of dropped frames and myth stops responding to keyboard input. Again, 1080i video I don't have this problem, only 720p.

[danielk]

fracmak, I still can't reproduce after trying to replicate everything I see in your log. The only really bad thing I see in there is that you are using busy loops for timing; when I turned that on here CPU usage shot up, as expected.

Try running this as root before starting up playback:

echo 1024 > /proc/sys/dev/rtc/max-user-freq

Let me know if this fixes the problem; if it does add it to your startup scripts.

Also you do know that you are NOT using XvMC, right? If playback of 720p only worked with XvMC before I wouldn't expect it to work with plain XVideo now.

[bhuffman@…]

I'll post my logs tonight when I get back home. Perhaps they can shed some light. I'm also using usleep w/ busy wait. I don't have the layout that you have for the RTC. Fedora Core 6 uses this: /proc/sys/dev/hpet/max-user-freq and I recall that when doing that "echo 1024" into it, myth still wouldn't use RTC.

[fracmak]

I recently upgraded to Fedora Core 7 from Fedora Core 3 and have been having a lot of difficulty getting the RTC to work. RTC was working in Fedora Core 3, I did notice that hpet has taken over the role of RTC, and was able to set it's frequency to 1024, but myth still isn't able to use it.

But I've had 720p working without XVMC ever since the switch to libmpeg and using proc-opts for my AMD64 chip, it's only in the EPG that things really come to a screaching halt.

[fracmak]

oh, on a side note, and this might be easier to fix, I don't mind the high cpu usage/dropped frames in the EPG if I could actually control the EPG. At the moment keyboard commands take anywhere from 15-20 seconds to be recognized. If somehow we could force keyboard commands to be processed when dropping frames, that might fix my dilemma.

[bhuffman@…]

I'm attaching output from mythfrontend -v playback on my machine. I have a P4 2GHz (older model) and an Nvidia 5200 AGP. Granted the frontend could use a hardware upgrade, but certainly playing SD content while scrolling through the EPG is much slower after the mythvid merge.

[danielk]

Shane suggested it might be the channel sorting that is taking a long time..

[danielk]

Can you try the attached patch?

I'm removing bobdeint from the description as I do not think it has anything to do with the slow EPG. If the attached patch doesn't help can you try the Program Guide from the Scheduling menu and tell me if that is slow for you too?

[bhuffman@…]

FYI - I've already tried using the Program Guide from the Scheduling menu (without this patch) and it is pretty speedy. Also, when switching interlace methods from BOB to (I think) Linear Blend, I didn't see the slow-down. It could simply be that BOB requires more CPU than the other methods and I'm right on the edge.

[bhuffman@…]

Sorry - meant to also say that I will try the patch this evening when I get home. I didn't mean to dismiss that point.

[bhuffman@…]

This patch did not solve the problem for me. This is definitely associated with livetv and the ability for the frontend to handle the playback in the window as well as intercept and act on the key strokes b/c: 1) The EPG in the scheduling menu is fast, 2) If I pause the liveTV and then use the EPG, the problem does not exhibit itself and 3) This is not necessarily tied to just moving around in the EPG. After scrolling around a little, even hitting "exit" can take up to 10 seconds to process.

[danielk]

(In [14611]) Refs #3986. Refs #3687. This fixes a performance regression caused by [14448] which fixed the channel sorting problem reported in #3687. It does not fix the performance regression reported in #3986, but does lower the CPU usage slightly for me and significantly for Shane.

[danielk]

bhuffman, do you think you could narrow this down to a changeset for me? I'm kind of perplexed by this regression at the moment.

Also, if you could run mythfrontend in gdb and hit Ctrl-C during the pause and send me the backtraces of all the running threads that might help me narrow down where things are getting stuck.

[bhuffman@…]

I will work on this over the weekend. Thanks.

[bhuffman@…]

I'm attaching the gdb.txt output. I captured this by running mythfrontend inside of gdb with the standard options and then hitting CTRL-C while it was pausing between a PGUP (might have been PGDN). To determine the exact changeset will take more time. I'll work on that over the weekend. Let me know if this helps at all.

[danielk]

bhuffman, it looks like VideoOutputXv::Show() is holding the X11 lock, and DrawUnusedRects?() is waiting on it. Show() is only holding the lock for an XSync() which really shouldn't take any time at all when using XVideo or OpenGL rendering.

I'm attaching a testing patch for you that will time how long this call is taking. If it's taking much longer than about 2 million cycles per frame this may be the source of the problem. Of course, this backtrace may just be a fluke.

[danielk]

testing patch

[bhuffman@…]

I'm attaching the output of mythfrontend after applying this patch and watching TV and moving around in the EPG. I hope this output is what you intended.

[fracmak]

Sorry for taking so long to apply the patch. I've been trying to fix the RTC issues before proceeding. CPU usage has gone down with RTC, but the issues are still there. The regex caching patch helped make things better though. I've also been noticing an occasional audio dropout (every 30-60 seconds) as well which might be related to this (kernel 2.6.22.9, alsa 1.0.14) . Now when I enter the EPG, cpu usage doesn't spike, so there's usually always 30% idle cpu, but it still hangs quite a bit. If I alt tab out of mythtv, the EPG doesn't redraw for 3-4 seconds. I've attached the gdb.txt file when it's hung, and I added the timing patch and have attached the results of that too. Hope this helps.

[danielk]

fracmak, if you comment remove the like "X11S(XSync(XJ_disp, False));" at the bottom of VideoOutputXv::Show(FrameScanType? scan) in videout_xv.cpp does this make your slowdown problem go away?

[bhuffman@…]

Were you saying to comment out that line? That makes the video go crazy although it does seem to make the EPG move reasonably fast.

[anonymous]

Are you by any chance using the r8169 networking driver?

[bhuffman@…]

Nope. E100

[bhuffman@…]

Might have been the nvidia driver. Here's a "snip" from the changelog in the latest driver. I'll notify if it's resolved once I install this.

"# Fixed problems scrolling ARGB X drawables in Qt."

[fracmak]

I just tried the new NVidia driver and it didn't fix the problem. Is there any possibility of having a setting that'll disable the live preview in the EPG?

[bhuffman@…]

Didn't fix it for me either. In fact, it broke XvMC, so I'm back the previous driver.

[fracmak]

Been doing some more research and discovered that if I add in the following to the device section of the xorg.conf the cpu usage of Xorg drops significantly when watcing HDTV recordings

Option “UseEvents?” “True”

Unfortunately it still doesn't solve the EPG problem, keyboard commands still have to 3-4 second delay before being processed. I'm wondering if it were possible to disable the preview window, if everything would work just fine (I really wouldn't miss the video preview window)

[fracmak]

In playing around with turning off the video preview of the EPG, I might have stumbled across something that might help solve this. In guidegrid.cpp, I commented out the update() call on line 757 in function GuideGrid::repaintVideoTimeout(void), thinking it would stop the drawing of the video preview. It didn't have the desired effect, the video was still being shown, but suddenly the EPG became extremely responsive and I've so far seen no ill effects. Hopefully this will lead to an explaination as to what's the underlying problem is.

[Kane]

I can confirm that bob de-interlacing is causing the issue. I use recent svn, and turning bob on (really necessary as picture quality is poor without it) makes epg really slow.

I have three frontends, sempron cpu and 1gt ram seems to have unusable epg. One frontend has 2gt ram and athlon X2 5000+ cpu, and there epg is little better, but still slow, and sometimes it freezes for 5-10 secs.

Without bob, epg is just ok. Also without preview picture, started from manage recordings-menu.

[anonymous]

I have the same problem as well with the guide and I have de-interlacing turned off. I have two cards Kworld ATSC-110 and an ATI TV Wonder. When viewing SD and I bring up the guide, the guide works without a problem even with de-inter set to linear blend. When I switch to the HD card and bring the guide up, it's very slow with either de-inter on or off as it sometimes takes 2 minutes before the guide will move.

[Kane]

This is strange. I updated few weeks ago from 15728 to 16487 and problem seemed to disappear. Today I updated 16487 to 16588 and guide is unusable again! Then I tried to downgrade back to 16487, but it didnot change guide slowness. So, there is something in the system what makes it slow, and it does not happen all time. I also tried to restart mysql server, but no help. If I set video scan from "detect" to "progressive" before I go to guide, it works flawlessly. So it has something to do with deinterlacing.

[Kane]

I chaged deinterlacer from bob to greedy 2x and this fixes problem.

[james.sumners@…]

After configuring my inittab to directly login my mythtv user and start X I began experiencing this problem. Since kernel >=2.6.25 requires PAM authentication to enable realtime priority threads, instead of merely setting the SUID bit, this prevented mythfrontend from entering realtime priority mode. After installing GDM and configuring it for autologin the problem is fixed.

Note: this is with the 0.21 release without -fixes.

[darkstar6262@…]

This problem has also affected me. I believe it is caused in some way by the bob deinterlacer, as it doesn't occur with any other deinterlacer (or with none). Specifics:

Ubuntu 8.04 running MythTV 0.21.0+fixes16838, realtime threads not enabled
Celeron D 3.0gHz w/ 1Gb RAM
PC Chips P23G motherboard
Geforce 6200 AGP running with nvidia binary drivers and DVI output @ 1920x1080, 60hz

With bob deinterlacing enabled, video output is crisp and CPU output is about 60 to 70% when just watching video. When in the guide, CPU increases to about 80%, but the guide is very unresponsive. Keyboard events register after ~15 to 30 seconds or longer. This is using XV output with SD.

I've managed to modify the source such that deinterlacing is disabled when entering the guide, and re-enabled when leaving it. This corrects the problem for me, making the guide perfectly responsive with little impact to the video. Upon restoring full video view, deinterlacing is restored to its previous setting.

I can provide patches if interested, but I don't believe this to be a good long-term solution as it just works around the problem. But it allows me to use bob with full functionality (which is a lower-CPU impact filter compared to the others).

I've tried the two patches suggested above (disabling the X11S call in videoout_xv.cpp and commenting out the call to update() in guidegrid.cpp). The first causes the video display to flash violently (and does not correct the guide problem), while the second helps, but does not completely fix, the problem -- it allows me to move freely using the arrow keys, but page-up and page-down do not respond unless I move to another window and back (effectively forcing a refresh).

Again, none of this is a problem with any other deinterlacing setting. This problem is particularly interesting to me since bob is the only decent looking deinterlacer that I can use without shaky/stuttery output due to lack of CPU processing power.

[darkstar6262@…]

Patch to disable deinterlacing while the guide is active

[darkstar6262@…]

I've just attached my patch that disables the deinterlacing while the guide is active. I've been using it for about a month now without any issues, and I can confirm that it makes the guide very responsive, regardless of which deinterlacer is being used. I have seen no ill effects as a result of the patch.

[danielk]

(In [18172]) Fixes #3986. Lowers CPU usage in EPG by temporarily disabling deinterlacing while displaying EPG.

This is a bit of a hack, and should no longer be needed after mythtv-vid merge.

[matt.doran@…]

Any possibility of merging this to 0.21-fixes?

It's one of the few pain points with mythtv for me ... and my WAF really takes a hit with non-responsive guide. :)

[matt.doran@…]

I just applied this patch to my 0.21-fixes and the guide speedup works beautifully .... unfortunately it introduces another problem.

When you leave the guide (i.e press escape), the deinterlacing doesn't re-enable. :(

[danielk]

(In [18180]) Refs #3986. Save m_scan_locked in addition to the m_scan state for the EPG speedup hack, otherwise the autodetection state is lost and MythTV won't be able to respond to scan changes in the stream.

[matt.doran@…]

Thanks Daniel.... this works great on 0.21-fixes.

Using bob previously made the guide painfully slow. The wife is very impressed by this fix! :)

I think this fix would be a good candidate for backporting.

[dan.rinkevich@…]

Will this be incorporated into a changeset of 0.21-fixes?

[anonymous]

For those of you running Ubuntu Hardy: I've backported the fix: ​https://bugs.launchpad.net/mythtv/+bug/229949/comments/28

Danielk told me that if it's been tested successfully, he'll commit it to -fixes.

[laga@…]

The fix also works in -fixes: ​http://ubuntuforums.org/showpost.php?p=5827190&postcount=7

I'm attaching a patch

[anonymous]

patch against -fixes

[laga]

Re-opening as danielk wanted to commit this to -fixes.

[dan.rinkevich@…]

This fixes the issue when I go into the program guide after the show is playing. I am running Mythbuntu 8.04 weekly 0.21-fixes with the backported fix (i.e. I added deb ​http://ppa.launchpad.net/laga/ubuntu hardy main to /etc/apt/sources.list). Nicely done!

Unfortunately, if I set the guide to automtically display when entering "Watch TV" (i.e check the "Show program guide when entering LiveTV" box in Utilities/Setup? >> Setup >> TV Settings >> Program Guide), it still hangs when starting "Watch TV". Does going into the program guide this way perhaps ignore this fix?

[danielk]

(In [18547]) Fixes #3986. Backport hack that disables bobdeint in th EPG video preview.

[danielk]

Dan R, it's possible that that entry method bypasses this hack. The real fix for this, Stanley's work in mythtv-vid, will not be backported to 0.21-fixes, but will hopefully be ready for the 0.22 release.