Opened 10 years ago

Closed 10 years ago

#6680 closed enhancement (fixed)

User Job for exporting to MythVideo

Reported by: Raymond Wagner <raymond@…> Owned by: Raymond Wagner
Priority: minor Milestone: 0.23
Component: Contributed Scripts & Apps Version: head
Severity: low Keywords:
Cc: Ticket locked: no

Description

This is a User Job implementation of the same functionality found in #6158, to get around a few of the limitations of the aforementioned patch. Namely, it blocks the interface, and assumes local file access.

This script requires updates to the Python bindings found in #6678. It also requires the TTVDB grabber in #6242 to get the season and episode number. While it currently works for me, it is unfinished and likely buggy.

The script is run just like the official jobs, just add a userjob for 'mythvidexport.py %JOBID%', and limit execution to the backend storing your video directory.

Path and Title are currently statically set to: '<base>/TV/<series>/Season <season>/<season>x<episode> - <episode title>' and '<series> - <season>x<episode> - <episode title>' These will be user adjustable from settings in the database in a future revision. Script will use 'Videos' StorageGroup? path if available, or will otherwise use 'VideoStartupDir?' as the base path.

Attachments (14)

mythvidexport.py (6.9 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.2.py (8.9 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.3.py (12.5 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.4.py (12.7 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.5.py (12.9 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.6.py (13.6 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.7.py (13.9 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.8.py (13.9 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.9.py (13.8 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
Fixes a mistake with the formatting fields from the database
mythvidexport.10.py (17.8 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.11.py (18.0 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
This is identical to the previous, except it functions without #6885.
mythvidexport.12.py (19.1 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.13.py (18.9 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
mythvidexport.14.py (16.4 KB) - added by Raymond Wagner <raymond@…> 10 years ago.
update for #7264

Download all attachments as: .zip

Change History (40)

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.py added

comment:1 Changed 10 years ago by Raymond Wagner <raymond@…>

Script now takes the option of pulling all data from TTVDB (--ttvdb), or pulling none, instead grabbing the episode number from the command line (--episode=S00E00). Default behavior remains pulling episode number from TTVDB, and metadata from the listings provider.

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.2.py added

comment:2 Changed 10 years ago by Raymond Wagner <raymond@…>

Added custom formatting for filename and title. It *should* work, but I won't have anything to test it on until later tonight. Strings are stored in settings values 'mythvideo.exportfmt' and 'mythvideo.titlefmt'.

Available strings are: %TITLE%, %SUBTITLE%, %SEASON%, %EPISODE%, %YEAR%, %DIRECTOR%, %HOSTNAME%, %STORAGEGROUP%, %GENRE%

Defaults are:

'TV/%TITLE%/Season %SEASON%/%SEASON%x%EPISODE% - %SUBTITLE%'

and

'%SEASON%x%EPISODE% - %SUBTITLE%'

I'm happy with the formatting I use, so it's a bit difficult thinking of what other people might want. Any suggestions on other strings to add?

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.3.py added

comment:3 Changed 10 years ago by Raymond Wagner <raymond@…>

Patch 4 fixes some bugs in 3, as well as changes the default strings to those of #6158, after concerns were brought up about batch updaters not understanding the default naming scheme.

A new option '--noepisode', specifies not to look for an episode number.

Also, the script is now set up to attempt a copy over scp (public key auth required) if the file is stored on a remote backend and file access is not available.

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.4.py added

comment:4 Changed 10 years ago by Raymond Wagner <raymond@…>

Script will now use an optional TTVDB.py config file if found in ~/.mythtv/ttvdb.conf. Also, file transfers are now done completely internally. If the file cannot be found on the local filesystem, it is pulled over the backend data socket.

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.5.py added

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.6.py added

comment:5 Changed 10 years ago by Raymond Wagner <raymond@…>

With ttvdb.py now in trunk, the job will always grab episode data from thetvdb.com, and will exit if failed (this means TV show exports only for now). Also, with the 'title' field in videometadata being split out to 'title','subtitle','season', and 'episode', the reason for separate 'fileformat' and 'titleformat' no longer exists. Field names have been updated to match those in #6158.

Support for image grabs has also been added for Coverfiles, Banners, and Fanart, but is only functional if the storagegroup exists. Use of the paths in the settings table is not supported.

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.7.py added

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.8.py added

comment:6 Changed 10 years ago by Raymond Wagner <raymond@…>

Will now automatically generate a 'folder.jpg' for the folder if one does not exist.

%SEASONPAD% has been added upon request as a forced two digit season. To maintain consistency, %EPISODE% now has no padding, and a %EPISODEPAD% has been added to replace the old behavior.

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.9.py added

Fixes a mistake with the formatting fields from the database

comment:7 Changed 10 years ago by stuartm

Component: MythTV - GeneralContributed Scripts & Apps
Owner: changed from Isaac Richards to robertm
Status: newassigned
Version: unknownhead

comment:8 in reply to:  7 Changed 10 years ago by anonymous

Requires http://svn.mythtv.org/trac/ticket/6885 Also -- needs to add content checking... TTvdb does not alway return year, so this script will fail

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.10.py added

comment:9 Changed 10 years ago by Raymond Wagner <raymond@…>

This one is a big rewrite. The script should now work with all forms of recorded content. Currently, it understands three types: TV, Movies, and Generic. The difference between a TV show and a movie is whether a subtitle exists in the program data. TV shows use the ttvdb grabber, movies use the tmdb grabber, and generic is anything that fails to be recognized by those respective grabbers.

That means... yet one more change to the formatting fields in the database (hopefully the last). The script now uses the following fields and defaults
mythvideo.TVexportfmt -- Television/%TITLE%/Season %SEASON%/%TITLE% - S%SEASON%E%EPISODPAD% - %SUBTITLE% mythvideo.MOVIEexportfmt -- Movies/%TITLE% mythvideo.GENERICexportfmt -- Videos/%TITLE%

'--tformat', '--mformat', and '--gformat' will override the default formatting for a one time use, or if not given a recording to work with, they will update the database to the specified format and exit.

'--listingonly' disables the grabbers, instead using the program data from your listing source. Note that 'TV' mode still requires the ttvdb grabber to find the season and episode number, and in failing to do so, will still drop back to 'GENERIC' mode.

As anon mentioned, this requires #6885.

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.11.py added

This is identical to the previous, except it functions without #6885.

comment:10 Changed 10 years ago by anonymous

You should add divide be zero checking around

remt = rem/rate

ZeroDivisionError?: integer division or modulo by zero

comment:11 Changed 10 years ago by paulh

Milestone: 0.220.23

To late to get this in for 0.22

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.12.py added

comment:12 Changed 10 years ago by Raymond Wagner <raymond@…>

Patch 12 relaxes the ttvdb image URL search string, to work better with any changes with the grabber script. Also adds a '-s/--simulation' option similar to JAMU and MiroBridge?, and calls mythcommflag on the copied file so fast forwarding in mythvideo works properly.

To the anonymous commenter, it may be worth adding a proper error to report as such, but if you have not transferred any data in the previous four seconds, there is something wrong and the script SHOULD fail.

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.13.py added

comment:13 Changed 10 years ago by Raymond Wagner <raymond@…>

Updates script to use recent changes in MythVideo? grabber format. Also updates image saves to follow standard MythVideo? naming.

comment:14 Changed 10 years ago by dave@…

Raymond...

It appears the latest python bindings broke this again... Traceback (most recent call last):

File "/usr/local/bin/mythvidexport.py", line 554, in <module>

main()

File "/usr/local/bin/mythvidexport.py", line 535, in main

export = VIDEO(opts,int(args[0]))

File "/usr/local/bin/mythvidexport.py", line 89, in init

self.get_source()

File "/usr/local/bin/mythvidexport.py", line 104, in get_source

self.source = mythinst.getCheckfile(self.prog)

File "/usr/lib/python2.6/dist-packages/MythTV/MythTV.py", line 350, in getCheckfile

res = self.backendCommand(BACKEND_SEP.join(['QUERY_CHECKFILE','1',program.toString()])).split(BACKEND_SEP)

File "/usr/lib/python2.6/dist-packages/MythTV/MythTV.py", line 145, in backendCommand

self.socket.send(command)

UnicodeEncodeError?: 'ascii' codec can't encode character u'\xe9' in position 97: ordinal not in range(128)

comment:15 Changed 10 years ago by Raymond Wagner <raymond@…>

It's got nothing to do with the bindings, but rather a reoccurring nuisance Doug Vaughan and I have dealing with in relation to UTF8. Follow step 5 at 'http://www.mythtv.org/wiki/Jamu#This_stupid_thing_does_not_work.21'. If that doesn't solve the problem, please run the following in a python shell and paste the result somewhere.

import MythTV
myth = MythTV.MythTV()
a = myth.getRecording(<chanid>,<starttime>)
print a.__dict__

Replace chanid and starttime with the recording that has failed.

comment:16 Changed 10 years ago by dave@…

It was caused be Ren\xe9e Zellweger. I edited the data to remove the accented "e".

comment:17 Changed 10 years ago by anonymous

I am getting the following error message when trying to use mythtvidexport.13.py

Traceback (most recent call last):

File ".mythtv/scripts/mythvidexport.py", line 554, in <module>

main()

File ".mythtv/scripts/mythvidexport.py", line 533, in main

export = VIDEO(opts,0)

File ".mythtv/scripts/mythvidexport.py", line 89, in init

self.get_source()

File ".mythtv/scripts/mythvidexport.py", line 103, in get_source

self.srchost = self.prog.hostname

AttributeError?: 'NoneType?' object has no attribute 'hostname'

comment:18 in reply to:  17 Changed 10 years ago by anonymous

Replying to anonymous:

I am getting the following error message when trying to use mythtvidexport.13.py

Traceback (most recent call last):

File ".mythtv/scripts/mythvidexport.py", line 554, in <module>

main()

File ".mythtv/scripts/mythvidexport.py", line 533, in main

export = VIDEO(opts,0)

File ".mythtv/scripts/mythvidexport.py", line 89, in init

self.get_source()

File ".mythtv/scripts/mythvidexport.py", line 103, in get_source

self.srchost = self.prog.hostname

AttributeError?: 'NoneType?' object has no attribute 'hostname'

ok, maybe ignore that. i typed too soon and i think that recording no longer exists.

on a recording that DOES exist, I get:

Traceback (most recent call last):

File ".mythtv/scripts/mythvidexport.py", line 554, in <module>

main()

File ".mythtv/scripts/mythvidexport.py", line 533, in main

export = VIDEO(opts,0)

File ".mythtv/scripts/mythvidexport.py", line 89, in init

self.get_source()

File ".mythtv/scripts/mythvidexport.py", line 104, in get_source

self.source = mythinst.getCheckfile(self.prog)

File "/usr/lib/python2.6/dist-packages/MythTV/MythTV.py", line 350, in getCheckfile

res = self.backendCommand(BACKEND_SEP.join(['QUERY_CHECKFILE','1',program.toString()])).split(BACKEND_SEP)

File "/usr/lib/python2.6/dist-packages/MythTV/MythTV.py", line 968, in toString

string += BACKEND_SEP + locale.format("%0.6f" %self.stars)

TypeError?: format() takes at least 2 arguments (1 given)

comment:19 Changed 10 years ago by anonymous

Sorry for the spam. This appears to be an issue with the MythTV.py package (at least in my Mythbuntu 9.10 distro). I fixed line 968 of MythTV.py to be

string += BACKEND_SEP + locale.formate("%0.6f", self.stars)

and mythvidexport successfully runs now. Unfortunately, it dumped the output directly into my base video directory titled as <title>.mpg; but that may just be a case of my not having set up the options correctly?

comment:20 Changed 10 years ago by robertm

Discussion belongs on mailing lists, NOT TICKETS.

comment:21 Changed 10 years ago by Raymond Wagner <raymond@…>

Version 13 and further are for trunk, and the new grabber format that trunk uses. They will not work properly on 0.22-fixes. I apologize for not making that clear.

comment:22 Changed 10 years ago by joelm

my apologies for gumming up the tracker as well. (anonymous from above)

Changed 10 years ago by Raymond Wagner <raymond@…>

Attachment: mythvidexport.14.py added

update for #7264

comment:23 Changed 10 years ago by robertm

Owner: changed from robertm to Raymond Wagner

comment:24 Changed 10 years ago by cpinkham

mythvidexport.14.py was committed to trunk a while back, but not referenced in the commit. Ticket is being left open for now for tracking issues.

comment:25 Changed 10 years ago by robertm

Milestone: 0.230.24

Since this is already committed, pushing to .24.

comment:26 Changed 10 years ago by Raymond Wagner

Milestone: 0.240.23
Resolution: fixed
Status: assignedclosed

Closing complete in preparation for 0.23. Future bugs or issues can open a new ticket, or be voiced in the mailing list.

Note: See TracTickets for help on using tickets.