mythtranscode doesnt die after lossless transcoding to cut out ads

[oobe.trouble@…]

Hi i use mythtranscode to edit my recordings after i make cut points i use a lossless transcoding profile im not sure if it has the same effect with a lossy transcode profile but when the transcode is finished i can see the mythtranscode proccess never dies

in mythweb i can see the job is finished but when i check with ps there are many idle unused mythtranscode proccess hanging around this isnt a big deal if im only transcoding one thing but if i am doing a lot of recordings all at once as i do every few weeks to make more space i end up with a huge amount of zombie like idle mythtranscode proccess's some of which use about 1.3% of system memory

if this issue is already reported and has a ticket im sorry but i couldnt find it if you need to know more info for me to provide just let me know

it should be fairly easy to reproduce on any system and may of been overlooked since not all people edit out commercials

[lofty69@…]

This issue is also effecting my 0.22 backend and is stopping the backend from shutting down. It worked fine in 0.21 Fixes

[J.Pilk@…]

I use mythtranscode like this on a routine basis, starting it from the 'Watch recordings' screen. It doesn't remain visible as active in 'atop' when it completes, but it *is* deleting the old file despite having this disabled in mythsetup - unless the preview file is deleted while the transcode is in progress. fc10 0.22 fixes from ATrpms. Was OK in 0.21 fixes.

[sphery]

(In [22631]) Fix a copy/paste typo from [17881] which used the wrong variable name, resulting in deleting the original pre-transcode recording file if any previews were present, regardless of the value of SaveTranscoding?. The typo was made during the huge QString-safety fix in #5311.

Refs #7315. (Even though this issue is unrelated to the issue on that ticket, a comment on that ticket mentions this issue.)

[sphery]

(In [22632]) Fix a copy/paste typo from [17881] which used the wrong variable name, resulting in deleting the original pre-transcode recording file if any previews were present, regardless of the value of SaveTranscoding?. Backports [22631] from trunk.

Refs #7315. Thanks to John Pilkington for reporting the issue.

[sphery]

Perhaps related to #7135?

[arizonamythtv@…]

I'm also running into the same problem with mythfilldatabase not ending at least in the processes even though the db and FE show it as completed. The BE still thinks mythfilldb is still running thus it'll stop updating the listings from schedules direct. As a band-aide, I added a cron job to kill mythfilldb every morning in case it has hung on me. Not sure if it's related to this.

[J.Pilk@…]

First a 'thank you' to Mike for picking up my report about not keeping the old file, and committing the patch so quickly. Sorry it was OT but I thought it was perhaps related.

Second, confirmation that I, too, see several inactive instances of mythtranscode in ps; they don't show in the normal atop screen because they aren't using cpu.

[robwebb5@…]

I see this issue too (release-0.22-fixes 22714) and previous 0.22-fixes versions.

At a guess, mythtv is launching a shell to launch the transcode command - while the transcode is running the following jobs are active:

rob@mediapc:~$ ps -Af | grep trans
mythtv   13708  3209  0 23:37 ?        00:00:00 sh -c /usr/bin/mythtranscode -j 10 -V 3 -p autodetect -l
mythtv   13709 13708 55 23:37 ?        00:00:12 /usr/bin/mythtranscode -j 10 -V 3 -p autodetect -l

After the transcode job is finished you are left with:

rob@mediapc:~$ ps -Af | grep trans
mythtv   13726     1  0 23:39 ?        00:00:00 /usr/bin/mythtranscode -j 10 -V 3 -p autodetect -l

Note that the parent process of the transcode job has changed from 13708 (the shell that mythtv launched) to 1, which would indicate the parent process has been killed.

Leaping into wild conjecture here, mythtv launches the shell job to perform the transcode job and keeps track of the pid that it spawned. When the transcode job is done mythtv reaps that pid, but it is reaping the pid of the shell, not the pid of the transcode command that the shell spawned.

[mythtv@…]

I too have seen this problem. I only do a lossless transcode (no commercial cuts).
Running the mythtranscode job manually will also leave the mythtranscode process running after it says it has finished.
The work around for me was to turn OFF the 'Delete Files Slowly' option and then mythtranscode does exit when it finishes.

release-0.22-fixes updated last night to latest revision.

Darryl

[J.Pilk@…]

2009-11-04 10:14:35.223 mythbackend version: branches/release-0-22-fixes/mythtv/ [22680] www.mythtv.org (from ATrpms)

It does die properly now. I still have the slow delete enabled. And the 'Keep file after transcoding' (yes) setting works. I haven't tried the 'no' setting yet.

[lofty69@…]

Working for me too as of [22679] (ATRPMs)

[sphery]

Seems to have been fixed with recent changes--2 out of 6 users who reported seeing the issue have said so. If still seeing the issue, please comment on the ticket and I'll reopen.

[mythtv@…]

I'm still having this problem. I went into mythtv-setup and only changed the 'Delete Files Slowly' option and turned it back on. Then I picked a recorded program with MythWeb and made it transcode again.

The transcode process is still running even though the log says it has finished. The initial shell process has disappeared.

The version is

Please include all output in bug reports. MythTV Version : 22715 MythTV Branch : branches/release-0-22-fixes Network Protocol : 50 Library API : 0.22.20091023-1 QT Version : 4.5.1 Options compiled in:

linux release using_oss using_alsa using_backend using_dvb using_frontend using_hdpvr using_iptv using_mheg using_opengl_video using_opengl_vsync using_qtwebkit using_x11 using_xrandr using_xv using_bindings_perl using_bindings_python using_opengl using_ffmpeg_threads using_live using_mheg

Note: I built mythTV direct from the branches/release-0-22-fixes repository, not using ATRPMS. It was a clean MythTV install on a freshly installed Centos 5.4. (centosplusPAE kernel)

Darryl

[sphery]

2 of 6 does not a quorum make, it seems

[J.Pilk@…]

Details for comparison:

'ps -AH | grep transcode' now gives no output after backend announcement that transcoding has finished. Entered from 'Watch recordings' screen, i, Job Options, Begin transcoding, Default

Linux version 2.6.27.37-170.2.104.fc10.x86_64

mythfrontend --version Please include all output in bug reports. MythTV Version : 22680 MythTV Branch : branches/release-0-22-fixes/mythtv/ Network Protocol : 50 Library API : 0.22.20091023-1 QT Version : 4.5.2 Options compiled in:

linux release using_oss using_alsa using_pulse using_arts using_jack using_backend using_dvb using_firewire using_frontend using_hdhomerun using_hdpvr using_iptv using_ivtv using_joystick_menu using_libfftw3 using_lirc using_mheg using_opengl_video using_opengl_vsync using_qtwebkit using_v4l using_x11 using_xrandr using_xv using_xvmc using_xvmc_vld using_xvmcw using_bindings_perl using_bindings_python using_opengl using_vdpau using_ffmpeg_threads using_libavc_5_3 using_live using_mheg

HTH

[J.Pilk@…]

The build installed earlier, which had the problem, was from ATrpms 'testing'

mythtv-0.22-217_trunk_r22570.fc10.x86_64.rpm

The 'ok' build that I have now, with --version output shown above, is from

mythtv-0.22-218_trunk_r22679.fc10.x86_64.rpm

[J.Pilk@…]

I apologise for my bitty postings here: trac was responding very slowly and I couldn't edit.

I wonder if the 'unlinking immediately' in the fragment of backend log below is relevant to this ticket? It seems to happen consistently.

2009-11-06 10:29:57.997 Transcode Completed 2009-11-06 10:29:58.925 Transcoding /store/myth/recs/1005_20091106080500.mpg done 2009-11-06 10:29:59.012 Error Truncating '/store/myth/recs/1005_20091106080500.mpg.png' could not determine size of the file, unlinking immediately. 2009-11-06 10:29:59.172 transcode: Transcode Finished: Peppa Pig "Pig tales": Autodetect (825.5 MB) 2009-11-06 10:29:59.172 JobQueue?: Transcode Finished: Peppa Pig "Pig tales": Autodetect (825.5 MB)

[mythtv@…]

I've done some basic debugging of the mythtranscode code.

In main.cpp, int slowDelete(QString filename)

I've traced the execution all the way through until it gets to

 if (inBackground) 
        exit(0);
 else 
   VERBOSE(VB_IMPORTANT, QString("Finished truncating %1.").arg(filename));

inBackground is set to true and so it calls the exit(0) statement and does not return. I assume this is when this process should end?

The code forks another process to do the truncate loop and completes the truncate successfully. The file is no longer on the disk.

The PID that the fork() call returns is the one that is still running. The original process has ended.

Hope this helps.

Darryl

[joe@…]

I've been having this problem as well. It appears to be related to libmythdb and mythtranscode forking with an active DB connection. Below is the last few lines of strace before mythtranscode hangs:

ftruncate(17, 3449716)                  = 0 (These 2 lines are slow-deleting
nanosleep({0, 500000000}, NULL)         = 0  the original file)
write(13, "0"..., 1)                    = 1  
futex(0x11d917c, FUTEX_WAIT_PRIVATE, 1, NULL

Attaching gdb to the same process provides the following:

#0  0x00007f3398a24d19 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#1  0x00007f33995976f9 in QWaitCondition::wait () from /usr/lib/qt4/libQtCore.so.4
#2  0x00007f3399596757 in QThread::wait () from /usr/lib/qt4/libQtCore.so.4
#3  0x00007f339e29894c in MythSocketThread::ShutdownReadyReadThread (this=0x11d8fa0) at mythsocketthread.cpp:64
#4  0x00007f3397d53a1d in exit () from /lib/libc.so.6

ShutdownReadyReadThread? is as follows, line 64 is 'wait();':

void MythSocketThread::ShutdownReadyReadThread(void)
{
    {
        QMutexLocker locker(&m_readyread_lock);
        m_readyread_run = false;
    }

    WakeReadyReadThread();

    wait(); // waits for thread to exit

    CloseReadyReadPipe();
}

Now, I'm not sure what is exactly the right way to fix this, close the DB before fork? After fork? What's the right way to close it anyway?

Hopefully this provides enough hints for someone else to nail it down.

-Joe

[sphery]

This issue only occurs if you enable "Delete files slowly" in mythtv-setup and do not enable "Save original files after transcoding (globally)".

I think the proper fix is to just remove all of the slow delete code in mythtranscode and either send a message to the backend to have it delete the file or implement a solution similar to #6376 .

Until it's fixed, users can either enable, "Save original files after transcoding (globally)" (which will only affect transcodes) or disable "Delete files slowly" (which will affect mythbackend as well as mythtranscode).

[sphery]

(In [23886]) Fixes #7315. Remove mythtranscode's internal copy of the slow-delete code and instead send a request to the backend to delete the file.

[oobe.trouble@…]

Hi i tried the patch on .23-fixes and re-enbled delete files slowly then transcoded some recordings works fine thanks a lot for you great work.

[sphery]

(In [24053]) Fix slow-delete with mythtranscode. Backports [23886] from trunk. Refs #7315.