Opened 14 years ago

Closed 13 years ago

#1919 closed patch (fixed)

Schema Change for MythMusic/Web + mp3Act Port

Reported by: Colin Guthrie <mythtv@…> Owned by: xris
Priority: minor Milestone: 0.21
Component: mythweb Version: head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

Hi,

As mentioned on the list, here is the (hopefully) final version of the patch started on Ticket #1699. I decided to open a new ticket for the sake of neatness as #1699 is a bit crazy now!).

This patch changes the schema of the database as used by MythMusic. It splits out the tables into separate songs, artists and albums tables and names the tables consistently.

This split reportedly speeds up loading of music due to the reduced communications between the frontend and the db for large collections.

This also adds a port of mp3Act (a web based interface for music streaming) to MythWeb (just visit the music module when not using the compact theme). This interface allows full playlist management with the ability to create, edit, delete and reorder your playlists.

This patch is large but has so far been tested by 4 people with success (some issues have cropped up concerning MySQL3 support but these have now been resolved). It has undergone several revisions in my own SVN mirror of the live trunk version to allow for this testing and polishing.

Some refactoring in MythMusic was also performed to reduce the code footprint and duplication (e.g. concerning playlist saving and filling lists of artists/albums/genres on 3 separate UI screens).

For the sake of convenience to the patcher here is the list of commands needed to apply this patch:

svn co http://svn.mythtv.org/svn/trunk/mythplugins mythplugins.schema+act
cd mythplugins.schema+act
tar xjf ../binary_files.tar.bz2
svn add mythweb/skins/{default,grey}/img/music
svn propset svn:mimetype image/gif mythweb/skins/{default,grey}/img/music/*.gif
svn propset svn:mimetype image/x-png mythweb/skins/{default,grey}/img/music/*.png
bzcat ../music_schema_change+mythweb_mp3act.diff.bz2 | patch -p0
svn add mythweb/skins/{default,grey}/music.css 
svn add mythweb/modules/music/mp3act_* 
svn add mythweb/modules/music/music_handler.php

Attachments (8)

binary_files.tar.bz2 (4.2 KB) - added by Colin Guthrie <mythtv@…> 14 years ago.
New images for MythWeb/mp3Act
music_schema_change+mythweb_mp3act.diff.bz2 (64.6 KB) - added by Colin Guthrie <mythtv@…> 14 years ago.
v1 - changes+new text files
mythmusic_schema_mythmusic.diff (74.3 KB) - added by Colin Guthrie <mythtv@…> 13 years ago.
v2 Myth Music Only Changes
mythmusic_schema_mythweb_full.diff (124.5 KB) - added by Colin Guthrie <mythtv@…> 13 years ago.
v2 Myth Web Only Changes
mythmusic_schema_mythweb_schema_only.diff (7.7 KB) - added by Colin Guthrie <mythtv@…> 13 years ago.
v2 Myth Web Only Schema Fixup (No change)
mfd.diff (20.8 KB) - added by Colin Guthrie <mythtv@…> 13 years ago.
Jochen's MFD Patch.
dbcheck.diff (474 bytes) - added by Colin Guthrie <mythtv@…> 13 years ago.
Whoops dbcheck.cpp diff to be applied to both MythMusic and MFD after the other patches
dbcheck.cpp.diff (748 bytes) - added by Colin Guthrie <mythtv@…> 13 years ago.
Fix a logic error in dbcheck.cpp

Download all attachments as: .zip

Change History (39)

Changed 14 years ago by Colin Guthrie <mythtv@…>

Attachment: binary_files.tar.bz2 added

New images for MythWeb/mp3Act

Changed 14 years ago by Colin Guthrie <mythtv@…>

v1 - changes+new text files

comment:1 Changed 14 years ago by Colin Guthrie <mythtv@…>

I should also comment that I will update MFD code also to be compatible with this if/when it is commited to SVN. As MFD generally lags behind, I decided not to submit the patches for this until the main changes are accepted (and in order to keep this ticket cleaner than #1699).

comment:2 Changed 14 years ago by jochen

I was the initial creator of this patch, and now i've tested cloins patches.

I have to say that erverythink is working verry well, and colin fixed all bug's that were left in mine. The loading of mythmusic still faster than with my patches. I have 40000 songs and it only needs 5 seconds to load. The web part is also works verry well and i couldn't fin out any left bugs...

hope this will be included in main mythmusic and web.

Maybe the ajax scripts used in this patch can be moved to the general include directory of mythweb, because i think they are a lot eaisier to use than the ajax.js wich is included in mythweb.

But try it out...

comment:3 Changed 14 years ago by jochen

Remove Instructions:

As this patch changes the database sturucture of your mythmusic i add here a instruction to remove it and get the old mythmusic.

  • Delete every table begining with music_.
  • Remove the tables musicmetadata and musicplaylist.
  • Remove the setting "MusicDBSchemaVer" from the settings table

Now you can reinstall the old mythmusic and the table structure for it will be rebuild.

But i think, noone who tried these patches will return to the old style!

comment:4 Changed 14 years ago by Colin Guthrie <mythtv@…>

Actually, there is a better way to remove it. See the commented out bit of SQL at the end of dbcheck.cpp which will "rollback" your database to the previous state without the need to rescan or lose your ratings etc.

comment:5 Changed 13 years ago by mikeNOSPAMmacleodPLEASEatIgmailHATEdotITcom

I just applied this patch to my mythtv install using svn revision 10246. It works well. I just wish that I could enable the 'jukebox' mode of mp3act. Then I wouldn't have to deal with MythMusic at all. Although it's almost tolerable now that I can make the playlists using this mp3act patch.b

comment:6 Changed 13 years ago by Isaac Richards

Resolution: invalid
Status: newclosed

I was going to apply this today, but:

  • I need this split up into separate patches for mythmusic and mythweb.
  • There is no patch for mfd?
  • There are a _lot_ of unnecessary whitespace changes in the mythmusic part of the patch.

Please reopen when these are addressed.

comment:7 Changed 13 years ago by Colin Guthrie <mythtv@…>

Cool. Thanks for the feedback Isaac.

I will split up the patchs and try and reduce the whitespace changes (this will be a pain to put right but serves me right for using an editor that "cleans" whitespace). I've asked Jochen to do the MFD side of things. As I said originally, I didn't want to go the full hog and do the MFD side of things too if this wasn't going to be comitted, but you've now given me reason to get it all finished :)

Changed 13 years ago by Colin Guthrie <mythtv@…>

v2 Myth Music Only Changes

Changed 13 years ago by Colin Guthrie <mythtv@…>

v2 Myth Web Only Changes

Changed 13 years ago by Colin Guthrie <mythtv@…>

v2 Myth Web Only Schema Fixup (No change)

Changed 13 years ago by Colin Guthrie <mythtv@…>

Attachment: mfd.diff added

Jochen's MFD Patch.

Changed 13 years ago by Colin Guthrie <mythtv@…>

Attachment: dbcheck.diff added

Whoops dbcheck.cpp diff to be applied to both MythMusic and MFD after the other patches

comment:8 Changed 13 years ago by Colin Guthrie <mythtv@…>

Resolution: invalid
Status: closedreopened

OK Isaac. I think I've covered all the bases here.

  • I've separated out the patches for MythMusic and MythWeb.
  • Whitespace has been re-inserted ;) I think I've got most of it OK, and the only ones left are either coding standards or very close to another change and were more awkward to reverse.
  • I've provided a severly cut down MythWeb patch that you (Isaac) should apply if you don't want to tred on Chris' toes and only want to update mythweb to work with the new schema. This would leave Chris to reverse the patch and apply the full patch himself. :)
  • I've given the MFD patch that Jochen emailed me (I've not tested this myself but it looks OK when glancing over it.) Whilst having a quick look over it, I realised that I, somewhat overzealously, stripped out some fields used by MFD from the schema (I thought they were not used). I've therefore provided an additional dbcheck.diff file that adds these fields back in (this patch should be applied to both MFD and MythMusic's dbcheck.cpp - hope I got all the fields....
  • Don't forget to look at the comments at the end of dbcheck.cpp and remove the redundant/testing code and uncomment the DROP TABLEs etc. (again in both the dbcheck.cpp's)

Let me know if you need anything else.

comment:9 Changed 13 years ago by Isaac Richards

One last question - why were existing column names (eg., artist, album, title) renamed? Lot less code changes if they had been kept the same (in their various tables)..

comment:10 Changed 13 years ago by Colin Guthrie <mythtv@…>

I don't actually know why they were changed. Jochen initially started work on mergeing mp3Act to mythweb and I think the schema changes were because that's how mp3Act did it and it was easier to do it that way round. In the end I mostly ripped out and rewrote all the SQL in mp3Act side of things as I wasn't happy with it (SQL injection vulns a plenty, so could have renamed the fields to the myth side of things, but the work was already done by then). Hope it's not a major problem, the tables that contain these fields has obviously changed, so any one with contrib/custom scripts etc, will have to update them regardless of the name change.... and I'm pretty sure all the code bound changes are caught (at least in MM/MW, like I say I've not been over Jochen's MFD patch with a fine tooth comb). Hope you wont ask me to change them back..... :)

comment:11 Changed 13 years ago by Isaac Richards

(In [10893]) Code/schema cleanup for mythmusic from Colin + Jochen.

mfd + mythweb bits to come.

Refs #1919.

comment:12 Changed 13 years ago by Isaac Richards

(In [10895]) mfd part of the mythmusic update.

Refs #1919.

comment:13 Changed 13 years ago by Isaac Richards

(In [10896]) Final part of mythmusic schema change: mythweb part (only schema change patch).

Refs #1919.

comment:14 Changed 13 years ago by Isaac Richards

Owner: changed from Isaac Richards to xris
Status: reopenednew

xris, it's up to you to decide on the more complex mythweb stuff now.

comment:15 Changed 13 years ago by xris

Resolution: fixed
Status: newclosed

I applied the patch and updated the parts that seemingly needed to be updated, but the music page now looks to me like a mostly blank page and clicking on any of the options just gives me a big "loading" bit with nothing else (just hangs).

Since the schema has been applied and this ticket is getting a bit old/outdated, please open a new one if you get the mp3act stuff working properly with svn.

comment:16 in reply to:  15 Changed 13 years ago by jochen

Replying to xris:

I applied the patch and updated the parts that seemingly needed to be updated, but the music page now looks to me like a mostly blank page and clicking on any of the options just gives me a big "loading" bit with nothing else (just hangs).

Since the schema has been applied and this ticket is getting a bit old/outdated, please open a new one if you get the mp3act stuff working properly with svn.

i'm look at mp3act part after work...

comment:17 Changed 13 years ago by Colin Guthrie <mythtv@…>

Xris,

It should just be a matter of reverting [10896] (or mythmusic_schema_mythweb_schema_only.diff - same thing!), then applying the patch mythmusic_schema_mythweb_full.diff (this contains the changes in [10896] but in a differently named file - music_handler.php).

Oh and don't forget, you will have to run the mythmusic plugin in mythfrontend in order to upgrade the db schema itself before the webstuff will work ;)

comment:18 Changed 13 years ago by Colin Guthrie <mythtv@…>

Ahh wait. I notice the bug I think you are having Xris. You are on https right?

Line 19 of mp3act_main.php - change to read (sans the PHP tags for syntax niceness on Trac):

<?php
$sajax_remote_uri = root.'music/';
?>

Changed 13 years ago by Colin Guthrie <mythtv@…>

Attachment: dbcheck.cpp.diff added

Fix a logic error in dbcheck.cpp

comment:19 Changed 13 years ago by Colin Guthrie <mythtv@…>

Found a bug in the moudles dbcheck.cpp - just a slight logical error when updating saved playlists to be global (rather than frontend specific). It currently only globalises the internal playlists (which is should not do), rather than the user defined ones (which it should do). This small patch fixes this. It wont help those who have already upgraded but it's not really devistating so no major harm done IMO. Sorry tho' :)

Should be applied in trunk/mythplugins/mythmusic/mythmusic/

comment:20 Changed 13 years ago by jochen

Patch for https...

i think also in the mp3act_functions.php the lines 29 and 34 need to be changed, so that https works right...

mfg.

comment:21 Changed 13 years ago by paulh

(In [10991]) Apply patch from Colin Guthrie to fix a logic error in dbcheck.cpp Refs #1919.

comment:22 Changed 13 years ago by jochen

@xris:

have you read my and colins comment about https... maybe this was your error! Or is still anything more needed for this patch???

comment:23 Changed 13 years ago by anonymous

Resolution: fixed
Status: closedreopened

comment:24 Changed 13 years ago by Isaac Richards

Resolution: fixed
Status: reopenedclosed

comment:25 Changed 13 years ago by xris

Component: mythmusicmythweb
Milestone: 0.21

I will reexamine this. didn't see the emails from when it got reopened.

comment:26 Changed 13 years ago by xris

Resolution: fixed
Status: closedreopened

comment:27 Changed 13 years ago by anonymous

When will this be integratet into mythweb???

comment:28 Changed 13 years ago by xris

Resolution: fixed
Status: reopenedclosed

(In [11344]) Apply the various patches to add the mp3act code to mythweb. should finally close #1919

comment:29 Changed 13 years ago by jochen

Resolution: fixed
Status: closedreopened

Patch in mp3act_functions.php

the function picdir need to be changes (line 29)

root.'music'

need to be changed to

skin_url.'img/music/'

comment:30 Changed 13 years ago by jochen

can be closed.

teh patches for the last probs are in ticket [2571]

comment:31 Changed 13 years ago by xris

Resolution: fixed
Status: reopenedclosed

closing as requested.

Note: See TracTickets for help on using tickets.