Exception using python bindings

[tom@…]

With recent versions of the python MySQLdb library you can get an exception when the MythTV python bindings do executemany:

Traceback (most recent call last):
  File "./mythcommflag-silence", line 250, in <module>
    main()
  File "./mythcommflag-silence", line 217, in main
    rec.update()
  File "/usr/lib/python2.7/site-packages/MythTV/database.py", line 429, in update
    self._push()
  File "/usr/lib/python2.7/site-packages/MythTV/dataheap.py", line 354, in _push
    self.markup.commit()
  File "/usr/lib/python2.7/site-packages/MythTV/database.py", line 624, in commit
    ','.join(['?' for a in fields])), data)
  File "/usr/lib/python2.7/site-packages/MythTV/_conn_mysqldb.py", line 96, in executemany
    raise MythDBError(MythDBError.DB_RAW, e.args)
MythTV.exceptions.MythDBError: MySQL error: sequence item 0: expected string, bytearray found

The problem is that executemany calls back to the logged cursor with the query expressed as a bytearray that has been encoded using the database encoding. That then throws an exception because you can't use join on a bytearray.

I have opened ​https://github.com/MythTV/mythtv/pull/153 with a patch to decode back to a string in this case.

[Roger Siddons]

Thanks for the patch.

Is this solely due to an upstream update in mysqlclient-python ? If so, are version numbers of the problematic/last-known-good available ?

[tom@…]

Yes this was triggered by upgrading from Fedora 25 to 26 which broke the commercial flagger I was using (based on ​https://www.mythtv.org/wiki/Commercial_detection_with_silences#silence.py) as it was failing to update the recording markup and bombing out.

The actual change in mysqlclient-python was from 1.3.7 to 1.3.12 by the looks of it.

[tom@…]

Looks like ​https://github.com/PyMySQL/mysqlclient-python/commit/49e401b3bc2c123b36949baeb409b91dcbdf85a6 is the trigger as that first appears in 1.3.8 and is a major rewrite of executemany.

It doesn't actually change the conversion of the query to a bytearray but it does change executemany to call back to self.execute (and hence get caught by Myth's LoggedCursor? derived class) rather than the self._query internal method than underlies execute.

[Roger Siddons]

Thanks for the details.

Ubuntu 16.04 & 17.04 use python-mysqldb 1.3.7.1; 17.10 uses 1.3.10 so will likely be affected also.

I've verified this patch has no adverse affects on an (old) 0.29 with 16.04. So I see no reason why it shouldn't be committed/backported immediately.

I'm a long way off master, up to my neck in broken code atm and not comfortable with pull requests so if some kindly soul feels inclined to do the honours, please do so.

[Peter Bennett]

I will commit this around September 26th, giving folks a week to review it first.

[Mark Spieth]

reviewed. LGTM

[Tom Hughes <tom@…>]

In b5201fd40bb4259361f5e689606ff5a1cf66dba2/mythtv:

Fixes #13123 Exception using python bindings

Update log_query to handle bytearray queries

Merge remote-tracking branch 'tomhughes/bytearray'

Signed-off-by: Peter Bennett <pbennett@…>

[Tom Hughes <tom@…>]

In b2a3174c0b66de83f8c931f157d4e5e709cb517e/mythtv:

Fixes #13123 Exception using python bindings

Update log_query to handle bytearray queries

Merge remote-tracking branch 'tomhughes/bytearray'

Signed-off-by: Peter Bennett <pbennett@…>
(cherry picked from commit b5201fd40bb4259361f5e689606ff5a1cf66dba2)