Python Bindings Reorganization

[Raymond Wagner]

This tracking ticket is for an internal reorganization of the python bindings. The first goal is to split the existing files into smaller, more manageable files.

• static.py - formerly MythStatic?.py, same purpose
• exceptions.py - MythError? class and subclasses
• logging.py - MythLog? class
• msearch.py - generic UPnP M-Search implementation, split out from the database autodetect code
• utility.py - decorators, utility classes, modified socket
• altdict.py - altered dictionary-like classes
• connections.py - base connection classes for database, backend, frontend, and xml
• database.py - managed database data access, and connection cache
• system.py - classes for running external programs and metadata grabbers
• mythproto.py - backend connection cache, event handler, and file transfers
• dataheap.py - collection of database access and manipulation classes
• methodheap.py - collection of methods for use with database, backend, frontend, and xml connections

This reorganization is intended to be largely transparent to the upper level user. There are two significant changes that will be visible externally.

1. To facilitate the reorganization and cut down on dependency issues, several methods have been rewritten as classmethods. Previous methods such as 'Program().getRecorded()' will now be performed as 'Recorded.fromProgram()'.
2. Several methods that previously returned lists will now return generators. While this means their output will always be an iterable, and checks for 'None' are unnecessary, it also means if you want to check length, or use the data multiple times, you will need to use 'list()' first. Methods providing iterables will now return instantly, as processing will be performed as data is requested. Additionally, errors may be raised during use of the iterable, rather than only at its creation.

Some other changes that will affect lower level users are as follows:

1. The use of classmethods instead of overloaded constructors has been followed through in a number of other places.
   1. The FEConnection and XMLConnection have .fromUPNP() methods that return generators of objects from a UPNP search.
   2. DBData subclasses no longer accept raw input. That is now provided by a .fromRaw() method.
   3. Guide and Program no longer have a etree input, with that functionality now being in .fromEtree() instead.
   4. getAllEntries() for DBData subclasses is now a classmethod, meaning it does not have to be called from an instantiated object.
2. MythLog? no longer accepts a file input, but now offers _setfile and _setfileobject classmethods, accepting a file path and opened file object, respectively. These will, by default, close the previous log file, unless that is 'stdout'. As before, the file is used globally by all instances of MythLog?.
3. The DictData? class, and all subsequent subclasses, are now based directly on 'dict'. DictData?._data no longer exists, and there may be other minor changes to behavior when used by subclasses.
4. The DBDataRef and DBDataCRef classes have been cleaned up and are now based directly on 'list'. Existing markup and cast information for Recorded and Video should behave similarly externally, but subclassing the two will be different (and hopefully more consistent).
5. The Exception classes are no longer exact duplicates. The individual error codes can only be used against their related exception type. All Exceptions can still be caught using the base MythError? class.

These changes have been tested against JAMU and MiroBridge?, with only minor patching needed, and any other python code in the repository should be unaffected.

[Raymond Wagner]

current version, to be committed in a few days

[Raymond Wagner]

Commited in [25147]. Leaving open for regression tracking.

[Raymond Wagner]

(In [25148]) Update mirobridge for changes in python bindings. Fix abort when deleting video with no title. Patch by Doug Vaughan. Refs #8577.

[Raymond Wagner]

(In [25149]) Update JAMU for changes in python bindings. Patch by Doug Vaughan. Refs #8577.

[Raymond Wagner]

(In [25150]) Add missed dependency include. Refs #8577.

[Raymond Wagner]

(In [25199]) Fix missing import. Refs #8577.

[Raymond Wagner]

(In [25200]) Rework DBConnection as a connection pool. Each opened cursor now receives its own connection, with a default pool size of 2. When the pool is exhausted, additional connections will be spawned as needed. This allows the database connection to be considered thread safe.

The pool size can be managed with the methods:

DBConnection.setDefaultSize(size) # classmethod DBConnection.resizePool(size)

The consequence of this change is that all cursors must be manually closed with:

cursor.close()

Any cursor not closed will result in a leaked connection. This includes cursors lost during error handling. In order to facilitate this, connections and cursors now offer a context manager for the 'with' statement:

db = MythDB() # connection serving as context manager with db as cursor:

cursor.execute(...)

# cursor serving as context manager with db.cursor() as cursor2:

cursor2.execute(...) # nested use is allowed with db as cursor3:

cursor3.execute(...)

In this syntax, the cursor will be automatically closed, and the connection released, as the interpreter exits the context, regardless of exception state.

Refs #8577.

[Raymond Wagner]

(In [25213]) Forcefully closes temporary database connections when cursor is closed. A crude fix until I can get the garbage collector to do it properly. Fixes problem introduced in [25200]. Refs #8577.

[Raymond Wagner]

(In [25219]) Adds importMetadata methods for Video and Recorded. Accepts one VideoMetadata? record, created by the VideoGrabber? class. Also includes several FileTransfer? related fixes. Refs #8577.

[Raymond Wagner]

(In [25260]) Add missing dependancy following changes reorganization. Refs #8577.

[Raymond Wagner]

(In [25274]) This commit tweaks the DBData[Write] classes, and adds several classes

for working with MythMusic database tables.

DBData classes now accept a '_key' parameter. This parameter is a list

providing the database fields to be used to link an object to an entry in the database. This replaces the previous '_where' and '_setwheredat' fields.

The '_key' parameter will set the other two values as:

_where = ' AND '.join(_key) _setwheredat = ','.join(_key)+','

If more control is needed, these values can still be set manually.

DBDataWrite.create() now returns a fully usable object, rather than the

last rowid. This method now only works properly if the unique identifier defined by the '_where' parameter is defined by the user. The old behavior of 'create()' is now found in the '_create()' method, if more control is needed than the new behavior provides.

DBDataWriteAI is a new class is a modified DBDataWrite specifically for

use with tables that use an Auto-Incrementing id field as the unique identifier.

DBData._evalwheredat() is now the only thing that should be setting the

'_wheredat' parameter, and so must be run to tie an object to an entry in the database. That means this method can be overridden to provide any code that must be run after an object is fully mapped, replacing '_postinit()'. '_evalwheredat()' now accepts an optional tuple, to allow '_wheredat' to be set directly, rather than pulling from data in the object.

Basic support for access to MythMusic data has been added, with the

following classes:

Song Album Artist MusicPlaylist? MusicDirectory?

These classes all have several classmethods, allowing them to be

initialized off of each other.

Also added is a MythMusic class offering a 'searchMusic' method.

Refs #8577

[Raymond Wagner]

(In [25301]) Fix support for Python 2.5.x following [25200]. Refs #8577.

[Raymond Wagner]

Any major foul-ups should have been caught by now, so I'm closing it. Any future bugs can start a new ticket.