Memory leak in DVB DummyVideo code

[anonymous]

I originally posted this in #3317 but that ticket got closed without resolving my issue. For well over 6 months now, I've noticed that the backend slowly increases its virtual memory footprint. Which eventually leads to slowing the entire machine down (more-so on machines with small amounts of RAM and not enough swap.

Digging into it with pmap, I found lots of these:

pmap -d `pidof mythbackend`
...
a10de000       4 ----- 00000000a10de000 000:00000   [ anon ]
a10df000    8188 rwx-- 00000000a10df000 000:00000   [ anon ]
...

(That's 8MB of anonymous memory for those playing along). And I noticed that the number of these went up everytime I changed a channel.

An 8MB Anonymous page is just what you see when libpthread allocates a new stack when creating a thread, but after several channel changes, I was seeing lots more of these anonymous-memory entries than running threads. So I surmized that we weren't reclaiming the stack when a thread terminated.

I wrote a utility (included) to track all thread creation and what memory it was associated with (just a wrapper around libpthread) and was able to use it to map pthread_create calls to stack allocations.

Using this, I determined that each time dvbrecorder.cpp::StartDummyVideo? was called, the number of anonymous memory entries would increase, and these were never being reclaimed.

Simply adding a:

pthread_detach(_video_thread);

right after the pthread_create call (this could alternatively be done using attributes passed to pthread_create I suppose) allows the pthread stacks to be reclaimed once the thread terminates, and now myth no longer increases it's virtual memory footprint by 8MB every time a channel-change occurs.

[anonymous]

Patch to fix the issue

[anonymous]

results of pmap after ~12 channel changes before the patch

[anonymous]

results of pmap after ~12 channel changes after the patch

[anonymous]

library wrapper to keep track of stack-allocation in pthreads

[anonymous]

I should mention that this affects 0.20-stable as well as SVN head. And that there are probably a couple other places like this, this is just the most frequent (and thus most annoying) one.

[Stuart Auchterlonie]

I looked into this as part of #3317.

What i found is that the code already does a pthread_join when the thread finishes. A pthread_join is supposed to do the same cleanup of a thread as would the completion of a detached thread.

Stuart

[danielk]

Potential fix

[danielk]

Alternate potential fix

[danielk]

I've attached two potential fixes, the first '3692-v1.patch' is very simple, but I'm not terribly confident it will work. The second should work, it makes the dummy video thread starting and stopping more like other joined MythTV threads, but is a little more involved.

[anonymous]

Neither one of these patches fixes the issue.

I added some debugging code and found that while StopDummyVideo?() gets called, we never get into the while() loop that calls pthread_join() This is because of this:

Index: dvbrecorder.cpp
===================================================================
--- libs/libmythtv/dvbrecorder.cpp     (revision 13823)
+++ libs/libmythtv/dvbrecorder.cpp     (working copy)
@@ -1137,7 +1137,7 @@
     }
 
     if (StreamID::IsVideo(info->streamType))
-        _stop_dummy = true;
+        StopDummyVideo();
 
     if (StreamID::IsAudio(info->streamType))
         GetTimeStamp(tspacket);

Which when added (without any of your patches applied) fixes the problem.

[anonymous]

Well, I've done some testing and this seems to slow down channel changing in a way that my 1st post does not. It is spending ~ 0.5 sec or so waiting for the pthread_join to complete. Using the 'pthread_detach' method does not show this lag.

[Stuart Auchterlonie]

I'm tempted to make the dummy video thread a detached thread, as I don't believe we really need to wait around for the result of the thread. It could quite happily be left to die off all by itself.

[danielk]

(In [14438]) Fixes #3692. Fixes a memory leak in DVB Radio support in trunk.