Duplicate detection problem with descriptions method

[otto at kolsi dot fi]

Duplicate detection with descriptions method work mostly very well. However, a series has been aired and I've recorded all the shows. Now it is shown again and for this particular series, the duplicate detection does not work for any of the episodes.

Following shows one current example episode in current program info and in old recorded:

mysql> select * from program where description like 'Erikoinen suhde 2/2%'; select * from oldrecorded where description like 'Erikoinen suhde 2/2%';
+--------+---------------------+---------------------+----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------------+---------+-------+-----------------+-----------------+--------+-----------+------+----------------+------------+-----------+----------+-----------------+----------+-----------+-------------------------+-----------+----------+---------+---------------+-------+------+
| chanid | starttime           | endtime             | title          | subtitle | description                                                                                                                                                                                  | category | category_type | airdate | stars | previouslyshown | title_pronounce | stereo | subtitled | hdtv | closecaptioned | partnumber | parttotal | seriesid | originalairdate | showtype | colorcode | syndicatedepisodenumber | programid | manualid | generic | listingsource | first | last |
+--------+---------------------+---------------------+----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------------+---------+-------+-----------------+-----------------+--------+-----------+------+----------------+------------+-----------+----------+-----------------+----------+-----------+-------------------------+-----------+----------+---------+---------------+-------+------+
|   1001 | 2008-01-12 23:30:00 | 2008-01-13 00:21:27 | Murhan jäljet |          | Erikoinen suhde 2/2. Boydin tutkimukset ärsyttävät jotain vallan huipulla, ja miehen jokaista askelta seurataan tarkkaan. Myös murhan entinen tutkinnanjohtaja on Boydille varsin tuttu. | Movie    | movie         |    0000 |     0 |               1 |                 |      0 |         0 |    0 |              0 |          2 |         2 |          | NULL            |          |           |                         |           |        0 |       0 |             1 |     0 |    0 | 
+--------+---------------------+---------------------+----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------------+---------+-------+-----------------+-----------------+--------+-----------+------+----------------+------------+-----------+----------+-----------------+----------+-----------+-------------------------+-----------+----------+---------+---------------+-------+------+
 
+--------+---------------------+---------------------+----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+----------+-----------+--------+----------+---------+---------+-----------+-----------+------------+---------+
| chanid | starttime           | endtime             | title          | subtitle | description                                                                                                                                                                                  | category | seriesid | programid | findid | recordid | station | rectype | duplicate | recstatus | reactivate | generic |
+--------+---------------------+---------------------+----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+----------+-----------+--------+----------+---------+---------+-----------+-----------+------------+---------+
|   1001 | 2007-09-30 22:30:26 | 2007-09-30 23:21:53 | Murhan jäljet |          | Erikoinen suhde 2/2. Boydin tutkimukset ärsyttävät jotain vallan huipulla, ja miehen jokaista askelta seurataan tarkkaan. Myös murhan entinen tutkinnanjohtaja on Boydille varsin tuttu. | Movie    |          |           |      0 |      709 | YLE TV1 |       3 |         0 |        -3 |          0 |       0 | 
+--------+---------------------+---------------------+----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+----------+-----------+--------+----------+---------+---------+-----------+-----------+------------+---------+

I've verified from the DB that descriptions are identical, with following query:

mysql>  select * from oldrecorded where description = (select description from program where description like 'Erikoinen suhde 2/2%');                                                                                              
+--------+---------------------+---------------------+----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+----------+-----------+--------+----------+---------+---------+-----------+-----------+------------+---------+
| chanid | starttime           | endtime             | title          | subtitle | description                                                                                                                                                                                  | category | seriesid | programid | findid | recordid | station | rectype | duplicate | recstatus | reactivate | generic |
+--------+---------------------+---------------------+----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+----------+-----------+--------+----------+---------+---------+-----------+-----------+------------+---------+
|   1001 | 2007-09-30 22:30:26 | 2007-09-30 23:21:53 | Murhan jäljet |          | Erikoinen suhde 2/2. Boydin tutkimukset ärsyttävät jotain vallan huipulla, ja miehen jokaista askelta seurataan tarkkaan. Myös murhan entinen tutkinnanjohtaja on Boydille varsin tuttu. | Movie    |          |           |      0 |      709 | YLE TV1 |       3 |         0 |        -3 |          0 |       0 | 
+--------+---------------------+---------------------+----------------+----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+----------+-----------+--------+----------+---------+---------+-----------+-----------+------------+---------+

Following shows the backend scheduling print containing this episode:

--- print list start ---
Title - Subtitle                Ch Station Day Start  End   S C I  T N   Pri
...
Murhan jäljet                     1 YLE TV1 12 23:30-00:21  1 3 3  A 3  18/0
...
---  print list end  ---

I've noticed that if I manually change the duplicate field in in oldrecorded from 0 to 1, then (naturally) dupdetection works.

I can do some debugging on my own but would need some guidance where to look etc. This issue would be easier to fix sooner than later since right now this particular series causes this problem to show up.

[Stuart Auchterlonie]

As you have noticed setting the duplicate field in oldrecorded to 1 lets dup detection work as intended.

What this means is that at some time past you've hit "delete & re-record" rather than "delete"

Marking this as invalid since this is the intended behaviour

Stuart

[otto at kolsi dot fi]

I did dump couple of weeks ago (6.1.) relevant parts of oldrecorded table to file and there one particular episode of this series had duplicate value 1. After that, this same episode has now become again available through EIT and now that I look oldrecorded this episode has duplicate value 0!

I definitely have not used "delete & re-record", especially for this episode during last couple of weeks. I have special Deleted recgroup in use instead of immediate deleting and it might be that this episode has been auto-expired from the Deleted group during this period. Don't know if this has anything to do with this..

Anyway, something strange is happening. I've enabled database logging in backend log and I'll probably add nightly cron dump of relevant parts of the program, recorded and oldrecorded tables to see if I can spot something. I'll reopen the ticket if/when I have more info or logs available.

[sphery <mtdean@…>]

Could this be a duplicate of #5048 (now fixed). In short, for a while MythWeb's Delete functionality always imposed the "and allow re-record" (even if you didn't say to allow re-record). So, could it be that you simply deleted the shows through MythWeb, rather than through mythfrontend, and it messed up the duplicate field for you? If so, the behavior would have been exactly as you described.

[otto at kolsi dot fi]

Replying to sphery <mtdean@thirdcontact.com>:

Could this be a duplicate of #5048 (now fixed). In short, for a while MythWeb's Delete functionality always imposed the "and allow re-record" (even if you didn't say to allow re-record). So, could it be that you simply deleted the shows through MythWeb, rather than through mythfrontend, and it messed up the duplicate field for you? If so, the behavior would have been exactly as you described.

Yes, I think it is very likely that programs that caused this issue were deleted in MythWeb. Haven't seen the same problem anymore.