Opened 13 years ago

Closed 13 years ago

Last modified 13 years ago

#9109 closed defect (Fixed)

Wrong end time displayed for MPEG2-TS

Reported by: outleradam@… Owned by: tralph
Priority: minor Milestone: unknown
Component: MythTV - Video Playback Version: Master Head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

When viewing a recording, MythTV reports the file length is much shorter then the actual duration.

In this 1 hour recording of caprica (exported by mythicalLibrarian): http://i236.photobucket.com/albums/ff111/DrivingTibNaked/Screenshot-1-6.jpg MythTV reports: 49:33 XBMC reports 59:57 Windows Media Player reports 59:56 FFMPEG reports 59:57

The file plays at normal speed. However, the MythTV time moves slower. MythTV Frontend displays 49:33 duration for the 59:57 recording but plays at a normal speed.

This problem affects exporting of commercial skip data frames through the python bindings, since frames are linked and calculated from time which displays a substantial deviation from physical time in the MythTV frontend.

Change History (34)

comment:1 Changed 13 years ago by robertm

Status: newinfoneeded_new

Please see ticket howto and provide required version and revision information. Unable to reproduce in trunk.

comment:2 Changed 13 years ago by anonymous

root@XBMC-live:~# mythbackend --version
Please attach all output as a file in bug reports.
MythTV Version   : 26835
MythTV Branch    : trunk
Network Protocol : 63
Library API      : 0.23.20100917-1
QT Version       : 4.6.2
Options compiled in:
 linux debug using_alsa using_jack using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_dvb using_firewire using_frontend using_hdhomerun using_hdpvr using_iptv using_ivtv using_joystick_menu using_lirc using_mheg using_opengl_video using_opengl_vsync using_qtdbus using_qtwebkit using_v4l using_x11 using_xrandr using_xv using_xvmc using_xvmc_vld using_xvmcw using_bindings_perl using_bindings_python using_mythtranscode using_opengl using_vdpau using_ffmpeg_threads using_live using_mheg

Which output files should I add?

comment:3 Changed 13 years ago by robertm

What is the source of the content?

comment:4 Changed 13 years ago by outleradam@…

Source of content: The device is a silicondust HdHomeRun? dual http://www.silicondust.com/products/hdhomerun/atsc/ The recording was recorded with MythTV through add-apt-repository ppa:mythtv/0.24; apt-get install mythtv The recording was processed, renamed and symlinked through mythicalLibrarian http://wiki.xbmc.org/?title=MythicalLibrarian

In the picture above, you can see MythTV is reading the file as 49:33, when the file is actually a 1 hour recording of Caprica from SyFy?.

I performed a database search if it is helpful.

root@XBMC-live:~# mysql -umythtv -pmythtv
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 475
Server version: 5.1.41-3ubuntu12.6 (Ubuntu)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use mythconverg
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from recorded where subtitle like "Retribution";
+--------+---------------------+---------------------+---------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-----------+----------+---------+---------+------------+-------------+----------+----------+------------+----------------+---------------------+------------+-------+-----------------+-----------------+----------+--------+---------------+------------+-------------+-------------+-------------------------+---------------------+---------------------+-----------+---------+-----------+------------+---------+--------------+---------------------+
| chanid | starttime           | endtime             | title   | subtitle    | description                                                                                                                                                   | category        | hostname  | bookmark | editing | cutlist | autoexpire | commflagged | recgroup | recordid | seriesid   | programid      | lastmodified        | filesize   | stars | previouslyshown | originalairdate | preserve | findid | deletepending | transcoder | timestretch | recpriority | basename                | progstart           | progend             | playgroup | profile | duplicate | transcoded | watched | storagegroup | bookmarkupdate      |
+--------+---------------------+---------------------+---------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-----------+----------+---------+---------+------------+-------------+----------+----------+------------+----------------+---------------------+------------+-------+-----------------+-----------------+----------+--------+---------------+------------+-------------+-------------+-------------------------+---------------------+---------------------+-----------+---------+-----------+------------+---------+--------------+---------------------+
|   1045 | 2010-10-15 01:00:00 | 2010-10-15 02:00:00 | Caprica | Retribution | Lacy botches an operation, and the Adama brothers begin a campaign of coercion against Daniel's former colleagues while Amanda tries to cope with her losses. | Science fiction | XBMC-live |        0 |       0 |       0 |          0 |           1 | Default  |       21 | EP01213751 | EP012137510011 | 2010-10-15 01:00:02 | 1751512340 |     0 |               0 | 2010-10-12      |        0 |      0 |             0 |          0 |           1 |          98 | 1045_20101015010000.mpg | 2010-10-15 01:00:00 | 2010-10-15 02:00:00 | Default   | Default |         1 |          0 |       0 | Default      | 2010-10-16 12:53:50 |
+--------+---------------------+---------------------+---------+-------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-----------+----------+---------+---------+------------+-------------+----------+----------+------------+----------------+---------------------+------------+-------+-----------------+-----------------+----------+--------+---------------+------------+-------------+-------------+-------------------------+---------------------+---------------------+-----------+---------+-----------+------------+---------+--------------+---------------------+
1 row in set (0.00 sec)

mysql> select * from channel where chanid like "1045";
+--------+---------+--------+----------+----------+------+------+----------+--------------+---------+-------------+----------+------------+--------+-------+----------+---------+---------------+---------------+---------+-----------+----------+-----------------+-----------------+---------------------+-------------------+------------+
| chanid | channum | freqid | sourceid | callsign | name | icon | finetune | videofilters | xmltvid | recpriority | contrast | brightness | colour | hue   | tvformat | visible | outputfilters | useonairguide | mplexid | serviceid | tmoffset | atsc_major_chan | atsc_minor_chan | last_record         | default_authority | commmethod |
+--------+---------+--------+----------+----------+------+------+----------+--------------+---------+-------------+----------+------------+--------+-------+----------+---------+---------------+---------------+---------+-----------+----------+-----------------+-----------------+---------------------+-------------------+------------+
|   1045 | 45      | 64     |        1 | SYFY     | SYFY |      |     NULL |              | 11097   |           0 |    32768 |      32768 |  32768 | 32768 |          |       1 |               |             1 |      11 |        45 |        0 |               0 |               0 | 2010-10-19 21:00:02 |                   |         -1 |
+--------+---------+--------+----------+----------+------+------+----------+--------------+---------+-------------+----------+------------+--------+-------+----------+---------+---------------+---------------+---------+-----------+----------+-----------------+-----------------+---------------------+-------------------+------------+
1 row in set (0.00 sec)

mysql> select * from capturecard;
+--------+-------------+-------------+-----------+-----------+--------------+----------------+-----------+--------------+--------------+-----------------------+-------------+---------------+-----------------+----------------+----------------+---------------------+----------------+-----------------+------------------+----------+------------+--------+-----+----------+-------------+
| cardid | videodevice | audiodevice | vbidevice | cardtype  | defaultinput | audioratelimit | hostname  | dvb_swfilter | dvb_sat_type | dvb_wait_for_seqstart | skipbtaudio | dvb_on_demand | dvb_diseqc_type | firewire_speed | firewire_model | firewire_connection | signal_timeout | channel_timeout | dvb_tuning_delay | contrast | brightness | colour | hue | diseqcid | dvb_eitscan |
+--------+-------------+-------------+-----------+-----------+--------------+----------------+-----------+--------------+--------------+-----------------------+-------------+---------------+-----------------+----------------+----------------+---------------------+----------------+-----------------+------------------+----------+------------+--------+-----+----------+-------------+
|      1 | 10183B8C-1  | NULL        | NULL      | HDHOMERUN | MPEG2TS      |           NULL | XBMC-live |            0 |            0 |                     1 |           0 |             0 |            NULL |              0 | NULL           |                   0 |           1000 |            3000 |                0 |        0 |          0 |      0 |   0 |     NULL |           1 |
+--------+-------------+-------------+-----------+-----------+--------------+----------------+-----------+--------------+--------------+-----------------------+-------------+---------------+-----------------+----------------+----------------+---------------------+----------------+-----------------+------------------+----------+------------+--------+-----+----------+-------------+
1 row in set (0.00 sec)

mysql> select * from recordedmarkup where chanid like "1045" and starttime like "2010-10-15 01:00:00" ;
+--------+---------------------+-------+------+-------+
| chanid | starttime           | mark  | type | data  |
+--------+---------------------+-------+------+-------+
|   1045 | 2010-10-15 01:00:00 |     0 |   -3 |  NULL |
|   1045 | 2010-10-15 01:00:00 |  8763 |    4 |  NULL |
|   1045 | 2010-10-15 01:00:00 |   501 |    5 |  NULL |
|   1045 | 2010-10-15 01:00:00 |     0 |    4 |  NULL |
|   1045 | 2010-10-15 01:00:00 | 30197 |    4 |  NULL |
|   1045 | 2010-10-15 01:00:00 | 13008 |    5 |  NULL |
|   1045 | 2010-10-15 01:00:00 | 34807 |    5 |  NULL |
|   1045 | 2010-10-15 01:00:00 |     1 |   32 | 29970 |
|   1045 | 2010-10-15 01:00:00 |     1 |   31 |   480 |
|   1045 | 2010-10-15 01:00:00 |     1 |   30 |   704 |
|   1045 | 2010-10-15 01:00:00 |     1 |   11 |  NULL |
|   1045 | 2010-10-15 01:00:00 | 58300 |    4 |  NULL |
|   1045 | 2010-10-15 01:00:00 | 48602 |    5 |  NULL |
|   1045 | 2010-10-15 01:00:00 | 43895 |    4 |  NULL |
|   1045 | 2010-10-15 01:00:00 | 74064 |    4 |  NULL |
|   1045 | 2010-10-15 01:00:00 | 77335 |    5 |  NULL |
|   1045 | 2010-10-15 01:00:00 | 63913 |    5 |  NULL |
+--------+---------------------+-------+------+-------+
17 rows in set (0.00 sec)

Let me know what you need. I will try my best to respond promptlty.

Last edited 13 years ago by robertm (previous) (diff)

comment:5 Changed 13 years ago by beirdo

Owner: set to beirdo

Please try:

mythcommflag --rebuild --file filename

and take a look again. This is looking suspiciously like my OTA 1080i issues I'm currently tracking down. Looks like your recording is 480i.

comment:6 Changed 13 years ago by anonymous

That did not work.

I am including a pic which shows the comskip information did not change, but now MythTV reads that the file is even SHORTER in duration. It should be 59 minutes 57 seconds long.

In this picture you can see

  1. VNC into mythtvfrontend playing the file which shows time duration of player which changed from 49:33 to 49:19 (it got shorter after the rebuild)
  1. ffmpeg and windows media player displaying duration 59:57 and 59:56 respectively
  1. terminal whicich shows the --rebuild operation and mythicalLibrarian pulling commercial skip information out of the db.
  1. the originally generated comskip and the new comskip which are the same

http://i236.photobucket.com/albums/ff111/DrivingTibNaked/Screenshot-11.png

comment:7 Changed 13 years ago by beirdo

Version: UnspecifiedTrunk Head

Commercial skip marks are not affected by rebuilding the seektable.

Since you were watching it run, did it make it to 100% on the rebuild, or die early?

Please try SVN version 26889 or later.

comment:8 Changed 13 years ago by sphery

and note that if the video is in MythVideo?, and not in Watch Recordings, you need to use the --video argument to mythcommflag, instead

comment:9 Changed 13 years ago by anonymous

Thanks. It was in Watch Recordings. The actual file is symlinked back to the original to maintain consistancy in mythtv and allow mythtv to manage free space. This allows export to XBMC as a front-end.

beirdo: It did finish at 100%, but please examine the last line and tell me if it is important here:

mythtv@XBMC-live:~/Videos$ mythcommflag --rebuild --file 1045_20101015010000.mpg 
2010-10-20 17:37:43.161 Using runtime prefix = /usr
2010-10-20 17:37:43.161 Using configuration directory = /home/mythtv/.mythtv
2010-10-20 17:37:43.162 Empty LocalHostName.
2010-10-20 17:37:43.291 New DB connection, total: 1
2010-10-20 17:37:43.330 Closing DB connection named 'DBManager0'
2010-10-20 17:37:43.394 mythcommflag version: 0.23.20100917-1 www.mythtv.org
2010-10-20 17:37:43.394 Enabled verbose msgs: important

MythTV Commercial Flagger, started at Wed Oct 20 17:37:43 2010
Rebuilding SeekTable(s) for:
ChanID  Start Time      Title                                      Status
------  --------------  -----------------------------------------  ------
1045   20101015010000 Caprica                                   2010-10-20 17:37:43.555 New DB connection, total: 2
2010-10-20 17:37:43.585 Using protocol version 63
Rebuilt    ps

Finished commercial break flagging at Wed Oct 20 17:38:24 2010

Error in my_thread_global_end(): 1 threads didn't exit
mythtv@XBMC-live:~/Videos$ 

comment:10 Changed 13 years ago by beirdo

It's irrelevant. It's a known quantity that the mysql thread sometimes won't happily shut down, but certainly not relevant to the issue at hand.

comment:11 Changed 13 years ago by Adam Outler <outleradam@…>

I just svn up'd from the ppa:mythtv/0.24

root@XBMC-live:~# mythbackend --version
Please attach all output as a file in bug reports.
MythTV Version   : 26914
MythTV Branch    : trunk
Network Protocol : 63
Library API      : 0.25.20101017-1
QT Version       : 4.6.2
Options compiled in:
 linux release using_alsa using_jack using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_dvb using_firewire using_frontend using_hdhomerun using_hdpvr using_iptv using_ivtv using_joystick_menu using_lirc using_mheg using_qtdbus using_qtwebkit using_v4l using_x11 using_xrandr using_xv using_xvmc using_xvmc_vld using_xvmcw using_bindings_perl using_bindings_python using_mythtranscode using_ffmpeg_threads using_live using_mheg
root@XBMC-live:~# 

issues i'm working on now:

mythbackend did not start automatically.

mythfrontend won't start at all.

Once I get past this, I will be able to get more information. mythweb works when

comment:12 Changed 13 years ago by robertm

Adam, plaese don't pollute the ticket with issues having nothing to do with the issue, it makes it very hard to work with the ticket effectively. Please try to respond concisely to the information requested, and resist the urge to add additional information, or to re-explain the issue.

comment:13 Changed 13 years ago by beirdo

Are there bad frames (or effects caused by bad signal) in this recording? This is another possible issue that can affect frame numbering.

comment:14 Changed 13 years ago by Adam Outler <outleradam@…>

How can I tell if I have bad frames?

hdhomerun_config_gui (the manufacturer's app for my recording device): Signal Strength 96% Signal Quality 100% Symbol Quality 100%

There was no change in the problem when switching to svn revision 26914.

comment:15 Changed 13 years ago by Adam Outler <outleradam@…>

any ideas?

comment:16 Changed 13 years ago by beirdo

You can tell if there are bad frames by watching the video and noticing pixelation, etc.

Is this an issue with just one particular file, or with several? So far I have not been able to reproduce your issue with my setup, although I have seen what may be actually symptoms of the same underlying issue.

comment:17 Changed 13 years ago by anonymous

Oh, I've got nothing like that in my recordings. My frames are good.

This problem affects all mythtv recordings. Mythtv reports the showing is about 10 minutes shorter then actual play time per hour. So in mythtv the comskips line up with mythtv time, but they do not line up with ffmpeg.or any other player because they are reporting that the show is 1 hour duration instead of 50 minutes

comment:18 Changed 13 years ago by beirdo

I have not seen this being reproducible. If you can find a short show to record (say less than 15min) and that exhibits this issue, please do so. We'll need a short sample, and I'd like it to be a complete recording if possible. FX plays some 3-5 min summaries of some shows (like Terriers for example) that would be a good sample recording if you get that channel. Looking for a complete recording, but in the range of 100MB total or so if you can.

Once you have that recording, I'd like to see the entire contents of the recordedseek table *for that recording ONLY*, as I'd like to see that the seektable is useful, etc, and because I'd be rebuilding a seektable and want to compare them. If you need a hand to get that information, let me know.

comment:19 Changed 13 years ago by Adam Outler <outleradam@…>

Alright. I'm scheduling a recording for 1 November 2010, it is the first available recording.

Category: 	 Drama
Type: 	         series (EP01304087)
Episode Number:  108/3min
Original Airdate:2010-10-31
Program ID: 	 EP013040870016
Cast: 	         Jamie Denbo, Donal Logue, Rockmond Dunbar, Laura Allen, Michael Raymond James
Length:  	 3 mins
Playback Group:  Default

It will be comskipped, then userjob mythicalLibrarian which changes the name of the file to match tvdb information, and exports the recordedmarkup database into a comskip v1 format, then symlinks back to the original. Mainly because it makes the recording title more legible.

This recording should be aproximately 25 megabytes.

comment:20 Changed 13 years ago by otto at kolsi dot fi

I see this problem and have seen ever since starting with Myth (0.19?). It hasn't been fixed by any of the ffmpeg syncs for example.

I just did quick test and recorded 15min recording (+ 6min overtime) with channels 1 and 3 with two separate but identical DVB-T cards. When watching these, both have correct duration, around 0:21:32 or so.

However, at the same time I have a 1h 49min recording from the same channel (nr 1) with same card from yesterday evening which shows duration of 1h 30min when watching. This particular recording has gone from one day to another but I don't think that should be relevant.

It seems that this issues occurs with longer recordings, say 1.5h - 3h, so it's quite hard to produce short sample that shows the problem..

Earlier (up until couple of months ago?) when mythcommflag rebuild was run for these recordings, timing was corrected. At the moment there seems to be some other issue with commflag rebuild since just recently it hasn't been able to fix the timing (see comment 6 in this ticket where original reporter sees the same behavior).

Recordings are fine when watching, so there shouldn't be any dropped frames or other that kind of problems.

comment:21 Changed 13 years ago by beirdo

Otto: Please don't add "me too" comments to tickets. Your comment has added no additional help to the process here, you have simply repeated what the original poster has said. If you'd like to help, please make a recording that it faulty in this way that is small enough for me to download in its entirety so I can look into this.

comment:22 Changed 13 years ago by beirdo

outleradam: Please produce the sample recording and its recordedseek data, or indicate if this is no longer an issue.

comment:23 Changed 13 years ago by outleradam@…

I'm sorry. The file came out as 124megs. I tried to upload to mediafire a couple of times, but it timed out. The difference in length is only 2 seconds.

May I mail a DVD to you? Send an email to me if so.

comment:24 Changed 13 years ago by beirdo

Information sent if you want to mail a CD-R. Additionally, could you get a log from the frontend playing one of these recordings with -v playback,timestamp?

Thanks.

comment:25 Changed 13 years ago by Adam Outler <outleradam@…>

I fired off a snail mail to you. You should get it at snail mail speeds.

I will get that -v information soon.

comment:26 Changed 13 years ago by Adam Outler <outleradam@…>

The information you requested is a text file called mythtimestamp compressed in .tar.bz2 format. This information was drawn from the episode of caprica you have on the DVD I sent. You can find the .tar.bz2 file here: http://www.mediafire.com/file/7wab285sho9crtc/mythtimestamp.tar.bz2

Did you get the DVD+R yet?

comment:27 Changed 13 years ago by Adam Outler <outleradam@…>

What more info is needed to get this ticket out of infoneeded_new status?

comment:28 Changed 13 years ago by beirdo

Status: infoneeded_newnew

Nothing at the moment. I'll be working on this in a few minutes.

comment:29 Changed 13 years ago by beirdo

Status: newassigned

comment:30 Changed 13 years ago by beirdo

Component: MythTV - GeneralMythTV - Video Playback
Owner: changed from beirdo to tralph

Running this file through mplayer shows what seems to be the core of the issue: This file has variable frame rate. The show parts seem to be at 24.976fps, and the commercials at 29.97fps. If you take the total time that it should be (3596s) in ratio to what we think it is (2973s), you will see:

2973/3596 = 23.976/29.97 (or fairly close to it). As our code counts frames (accurately) as they decode...

This is an issue of not dealing with framerate changes properly. Our code is assuming that the framerate will stay constant throughout the recording, so it is assuming 29.97fps throughout this file, but the recording is at film rate for much of the show.

I am reassigning this... tralph, if you'd like to send it back to have me rework the framerate stuff later, be my guest. I'm not sure where fixing this falls in the priorities, and if you need more details, ping me.

comment:31 Changed 13 years ago by danielk

We had a ticket for this in the 800-900 ticket # range, but I believe we eventually closed it as a wontfix. It would of course be nice to fix it, but the fix is non-trivial and the bugs it causes are of the trivial sort.

comment:32 Changed 13 years ago by Adam Outler <outleradam@…>

Well, it would seem that the fix for mythtv would be to move the current timeline to what would be called a "variable timeline". Then display to the user a static timeline which the current position displayed on the ticker would be

Starttime=0,casted as time, Endtime=100(Displayed ffmpeg endtime), currentPosition=((framenumber/totalframes*100)*endtime)casted as time. This would create a 100 point resolution for the ticker and display the proper end time.

comment:33 Changed 13 years ago by tralph

Resolution: Fixed
Status: assignedclosed

(In [27326]) Several changes to fix the current position and duration of the playback OSD

  • Start using the currently displayed video timecode for the OSD position instead of frame number and total frames which is inaccurate.
  • Use FFmpeg duration for pre-recorded and video playback length. LiveTV is already correct since it uses the actual recording times but in-progress recordings still uses frames written and fps which is obviously wrong for variable framerate and repeat frame material. The problem with in-progress duration should be able to be fixed by using recording times like livetv does. An update to mythplayer and the recorder class will be necessary.
  • Refactor and clean-up some timestamp/timecode handling.

Refs #8631.

Fixes #9109.

comment:34 Changed 13 years ago by Taylor Ralph

mythplayer: Revert back to the old method used for duration/position in the OSD slider.

This has proven to be problematic given that many encoders produce bad timestamps or reset the timestamp (timestamp discontinuities) yielding this method insufficient. In trunk Gavin has added the ability for commflag to generate and store the duration. Using the commflag duration will allow accurate duration/position for broken videos.

Refs #9109.

Changeset: 80192ec90d0c8937c642f41a11c5316830472545

Note: See TracTickets for help on using tickets.