61f9ac9a8d7f6eb5cf1fac74376245e8b82e42f1 breaks playback on VLC

[doug@…]

As per the mail thread here: ​http://lists.mythtv.org/pipermail/mythtv-users/2015-March/377445.html The below commit breaks playback on VLC. Rerverting the commit resolves the issue.

Here is the offending commit: ​https://github.com/MythTV/mythtv/commit/61f9ac9a8d7f6eb5cf1fac74376245e8b82e42f1

Thanks!

Doug

[stuartm]

We'll probably need an example recording to understand what the problem is and why it only seems to affect your HD-PVR recordings.

[stuartm]

Doug, it would also be helpful to know what options you have setup in your default recording profile for the HD-PVR (mythtv-setup)

It would also help if you can attach a backend log with -v record including the start of a recording.

Lastly the output of 'mediainfo <path to recording>' will show us which streams are present in the recording.

[doug@…]

Hi,

How can I attach a large file? trac says the max size is 2MB. Here are my options from the capturecard table:

mysql> select * from capturecard;
+--------+----------+-------------+-------------+-----------+-----------+--------------+----------------+-----------+--------------+--------------+-----------------------+-------------+---------------+-----------------+----------------+----------------+---------------------+----------------+-----------------+------------------+----------+------------+--------+-----+----------+-------------+-----------+----------+------------------------+----------------+---------------+----------+-----------+-------------+-------------+-------------+-----------+------------+-------------+
| cardid | parentid | videodevice | audiodevice | vbidevice | cardtype  | defaultinput | audioratelimit | hostname  | dvb_swfilter | dvb_sat_type | dvb_wait_for_seqstart | skipbtaudio | dvb_on_demand | dvb_diseqc_type | firewire_speed | firewire_model | firewire_connection | signal_timeout | channel_timeout | dvb_tuning_delay | contrast | brightness | colour | hue | diseqcid | dvb_eitscan | inputname | sourceid | externalcommand        | changer_device | changer_model | tunechan | startchan | displayname | dishnet_eit | recpriority | quicktune | schedorder | livetvorder |
+--------+----------+-------------+-------------+-----------+-----------+--------------+----------------+-----------+--------------+--------------+-----------------------+-------------+---------------+-----------------+----------------+----------------+---------------------+----------------+-----------------+------------------+----------+------------+--------+-----+----------+-------------+-----------+----------+------------------------+----------------+---------------+----------+-----------+-------------+-------------+-------------+-----------+------------+-------------+
|      1 |        0 | /dev/hdpvr1 | 2           | NULL      | HDPVR     | Component    |           NULL | freshertv |            0 |            0 |                     1 |           0 |             0 |            NULL |              0 | NULL           |                   0 |           1000 |            7000 |                0 |        0 |          0 |      0 |   0 |     NULL |           1 | Component |       11 | /usr/local/bin/pvr1.sh | NULL           | NULL          | NULL     | 155       | PVR-1       |           0 |           0 |         0 |          1 |           1 |
|      2 |        0 | /dev/hdpvr0 | 2           | NULL      | HDPVR     | Component    |           NULL | freshertv |            0 |            0 |                     1 |           0 |             0 |            NULL |              0 | NULL           |                   0 |           1000 |            7000 |                0 |        0 |          0 |      0 |   0 |     NULL |           1 | Component |       11 | /usr/local/bin/pvr0.sh | NULL           | NULL          | NULL     | 155       | PVR-0       |           0 |           0 |         0 |          2 |           2 |

Here is the output from mediainfo:

General
ID                                       : 0 (0x0)
Complete name                            : Loss_20150322.mpg
Format                                   : MPEG-TS
File size                                : 4.45 GiB
Duration                                 : 59mn 56s
Overall bit rate mode                    : Variable
Overall bit rate                         : 10.6 Mbps

Video
ID                                       : 4113 (0x1011)
Menu ID                                  : 1 (0x1)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L4.0
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : 27
Duration                                 : 59mn 55s
Bit rate mode                            : Variable
Bit rate                                 : 9 705 Kbps
Maximum bit rate                         : 20.0 Mbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 29.970 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.156
Stream size                              : 4.06 GiB (91%)
Color primaries                          : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics                 : BT.709-5, BT.1361
Matrix coefficients                      : BT.709-5, BT.1361, IEC 61966-2-4 709, SMPTE RP177

Audio
ID                                       : 4352 (0x1100)
Menu ID                                  : 1 (0x1)
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Mode extension                           : CM (complete main)
Codec ID                                 : 129
Duration                                 : 59mn 56s
Bit rate mode                            : Constant
Bit rate                                 : 384 Kbps
Channel(s)                               : 6 channels
Channel positions                        : Front: L C R, Side: L R, LFE
Sampling rate                            : 48.0 KHz
Bit depth                                : 16 bits
Compression mode                         : Lossy
Delay relative to video                  : -39ms
Stream size                              : 165 MiB (4%)

Thanks!

Doug

[peper03]

Dropbox or the like is pretty useful for this.

A full-length recording isn't necessary. I would expect twenty seconds or so should be more than enough (so about 15-25MB based on your figures above).

A good case and a bad case would be very useful.

[Frank Phillips <fphillips81@…>]

You can cut out the first 20s to new file like this: ffmpeg -i input.mpg -t 20 -codec copy output.mpg

Google Drive is another option for storage.

[david.madsen@…]

I've run into this issue as well.

I don't normally playback the video files directly, but I was trying to do so recently and had the same problem. Only the recordings from the HD-PVR have issues. I analyzed a recording that works (from a 0.27.3 compiled version of mythtv) and one that doesn't work (from a recent compile of 0.27.4).

The working recording has the following PIDS:

0, 256, 4097, 4113, 4352

The recording that will not play back has these:

0, 256, 4113, 4352

4097 has been stripped out, and after some further analysis it looks like that PID contains the PCR in some fashion or another. I'm no expert on TS streams, just observations after analyzing the stream with a few TS tools.

[paulh]

David,

Thanks for looking in to this. It's clearly a consequence of [61f9ac9a8d] which is being a little too aggressive removing streams.

If you run the good file through mediainfo what is the 4097 identified has?

[david.madsen@…]

Mediainfo doesn't appear to identify the PID at all, though it does pick up the video (4113) and audio (4352). 0 looks to be the PAT, and 256 the PMT.

I've provided the data from the first packets for each PID in the working recording.

PID 0:

47 40 00 13 00 00 B0 0D 00 00 C1 00 00 00 01 E1
00 B3 58 82 B7 FF FF FF FF FF FF FF FF FF FF FF
...

Table
   table_id: 0x0 (program_association_section)
   section_length: 13
   transport_stream_id: 0
   version_number: 0
   current_next: True
   section_number: 0
   last_section_number: 0

   program_number: 1
   program_map_PID: 256

Section CRC: B3 58 82 B7 

PID 256:

47 41 00 13 00 02 B0 3B 00 01 C1 00 00 F0 01 F0
0C 05 04 48 44 4D 56 88 04 0F FF 06 04 1B F0 11
F0 0A 05 08 48 44 4D 56 FF 1B 44 3F 81 F1 00 F0
0E 05 04 41 43 2D 33 81 06 06 38 0F 01 0F 00 9C
78 FF 76 FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
...

PID 4097:

47 10 01 22 B7 10 00 00 26 10 7E E5 FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
...

Adaptation fields
   Adaptation_field_length: 183
   discontinuity_indicator: False
   random_access_indicator: False
   ES_priority_indicator: False
   PCR_flag: True
   OPCR_flag: False
   splicing_point_flag: False
   transport_private_data_flag: False
   adaptation_field_extension_flag: False
   PCR: 5846629

[Karl Egly]

Can we see the decoded PMT? A supposed fix went in with [d947e91eedf73a43dde79446b87e0095e54bf628] in early January.

[david.madsen@…]

The tool I'm using isn't decoding it automatically for some reason. I'm working on getting some sample clips hosted that should be more useful.

Here's the version info for my current install:

MythTV Version : f1115fc MythTV Branch : fixes/0.27

It was compiled sometime around the end of March so it should include the fix as long as it was backported into fixes. Maybe that's the issue?

[david.madsen@…]

I just checked my source tree, and that commit isn't in the fixes/0.27 branch as far as I can tell. I'll pull it in, and give it a try.

[Stuart Morgan <smorgan@…>]

In a14b34bf706f11ba5d2af11bc92d1f9a5e82c550/mythtv:

Fix PCR packets being stripped from transport stream following [0f0e6787]

(cherry picked from commit d947e91eedf73a43dde79446b87e0095e54bf628)

Refs #12422

[david.madsen@…]

I tried running with the previous bugfix, but seems there's still an issue somewhere. I added some instrumentation code in MPEGStreamData::CreatePMTSingleProgram to print out the PIDs, types, and PCRPID.

It looks like the incoming PMT doesn't have the PCR PID listed as a valid stream, so it didn't get propagated to the output PMT.

2015-05-18 04:46:29.661094 W [31238/31361] RecThread mpeg/mpegstreamdata.cpp:574 (CreatePMTSingleProgram) - MPEGStream[40](0xffffffffaac38640): PID 0x1011 - Type 0x1b - PCRPID 0x1001
2015-05-18 04:46:29.661108 W [31238/31361] RecThread mpeg/mpegstreamdata.cpp:574 (CreatePMTSingleProgram) - MPEGStream[40](0xffffffffaac38640): PID 0x1100 - Type 0x81 - PCRPID 0x1001

2015-05-18 04:46:29.660892 I [31238/31361] RecThread mpeg/mpegstreamdata.cpp:511 (CreatePMTSingleProgram) - MPEGStream[40](0xffffffffaac38640): CreatePMTSingleProgram()
2015-05-18 04:46:29.660898 I [31238/31361] RecThread mpeg/mpegstreamdata.cpp:512 (CreatePMTSingleProgram) - MPEGStream[40](0xffffffffaac38640): PMT in input stream
2015-05-18 04:46:29.661078 I [31238/31361] RecThread mpeg/mpegstreamdata.cpp:513 (CreatePMTSingleProgram) - MPEGStream[40](0xffffffffaac38640): Program Map Section
 PSIP tableID(0x2) length(59) extension(0x1)
      version(0) current(1) section(0) last_section(0)
      pnum(1) pid(0x100)
  Registration Descriptor: 'HDMV' Blu-Ray A/V for read-only media (H.264 TS)
  ATSC CA Descriptor Descriptor (0x88) length(4)
  Stream #0 pid(0x1011) type(0x1b video-h264)
    Registration Descriptor: 'HDMV' Blu-Ray A/V for read-only media (H.264 TS)
  Stream #1 pid(0x1100) type(0x81 audio-ac3)
    Registration Descriptor: 'AC-3' ATSC audio stream A/52
    Audio Stream Descriptor  full_srv(1) sample_rate(48kbps) bit_rate(=384kbps, 14)
      bsid(6) bs_mode(0) channels(2/0) Dolby(Not indicated)
      mainID(0)

2015-05-18 04:46:29.661094 W [31238/31361] RecThread mpeg/mpegstreamdata.cpp:574 (CreatePMTSingleProgram) - MPEGStream[40](0xffffffffaac38640): PID 0x1011 - Type 0x1b - PCRPID 0x1001
2015-05-18 04:46:29.661108 W [31238/31361] RecThread mpeg/mpegstreamdata.cpp:574 (CreatePMTSingleProgram) - MPEGStream[40](0xffffffffaac38640): PID 0x1100 - Type 0x81 - PCRPID 0x1001
2015-05-18 04:46:29.661128 I [31238/31361] RecThread mpeg/mpegstreamdata.cpp:686 (CreatePMTSingleProgram) - MPEGStream[40](0xffffffffaac38640): PMT for output stream
2015-05-18 04:46:29.661173 I [31238/31361] RecThread mpeg/mpegstreamdata.cpp:687 (CreatePMTSingleProgram) - MPEGStream[40](0xffffffffaac38640): Program Map Section
 PSIP tableID(0x2) length(59) extension(0x1)
      version(0) current(1) section(0) last_section(0)
      pnum(1) pid(0x100)
  Registration Descriptor: 'HDMV' Blu-Ray A/V for read-only media (H.264 TS)
  ATSC CA Descriptor Descriptor (0x88) length(4)
  Stream #0 pid(0x1011) type(0x1b video-h264)
    Registration Descriptor: 'HDMV' Blu-Ray A/V for read-only media (H.264 TS)
  Stream #1 pid(0x1100) type(0x81 audio-ac3)
    Registration Descriptor: 'AC-3' ATSC audio stream A/52
    Audio Stream Descriptor  full_srv(1) sample_rate(48kbps) bit_rate(=384kbps, 14)
      bsid(6) bs_mode(0) channels(2/0) Dolby(Not indicated)
      mainID(0)

[peper03]

The PCR PID missing from the PMT would certainly explain why it is not included in the final recording.

I'm curious why more people aren't reporting problems given that the commit was pushed to the 0.27 branch last November. Could it related to the firmware/model?

[david.madsen@…]

Could be it's not all that common to play the recordings directly with a 3rd party player, or enough of the 3rd party players don't care about the PCR and just use the PTS/DTS for synchronization.

Here's the version of the firmware I'm running. I'm not sure if there's been any hardware revisions, but the one I have is quite old.

[   10.824128] hdpvr 2-1.4:1.0: firmware version 0x1e dated Mar  7 2012 08:25:15
[   11.325512] hdpvr 2-1.4:1.0: device now attached to video0
[   11.325529] usbcore: registered new interface driver hdpvr

I looked at the raw TS output directly from the HDPVR, here's the original PMT:

Table
   table_id: 0x2 (program_map_section)
   section_length: 59
   program_number: 1
   version_number: 0
   current_next: True
   section_number: 0
   last_section_number: 0

   PCR_PID: 4097
   Program_descriptor_length: 12

   stream_type: 27
   elementary_PID: 17
   ES_descriptor_length: 10
No known descriptors

   stream_type: 129
   elementary_PID: 256
   ES_descriptor_length: 14
No known descriptors

Section CRC: 9C 78 FF 76 

It appears it doesn't define a stream at all for the PCR, just references it in the header bits, or PCR specific field.

I threw together a quick patch to test, and I did get it working. I'm not sure this is the best method at solving the problem, but seems to work fine for me.

[david.madsen@…]

Fixes filtering of PCR from HDPVR streams

[peper03]

As I understand it, a stream containing only the PCR will not appear in the PMT, but, as you mentioned, the PCR PID is stored in a separate field in the PMT.

DVB recordings have the PCR in the video stream, for example.

One problem with your patch is that the PCR stream will *always* be included in the recording. A DVB tuner with a recording profile set to 'audio only' would therefore no longer work because the video stream would always be written.

I think I've solved the problem in a slightly different way by marking the PCR PID as 'in use' if it isn't referenced in the PMT.

Please try the patch and let me know.

[david.madsen@…]

I suspected there may be some issues with that method outside of my specific setup. I initially attempted to something similar to what your patch does, but wasn't sure about side effects or what types to use, so you've solved that issue. :)

I backed out my changes, and pulled in your patch, though I did need to move your addition from DTVRecorder::HandleSingleProgramPAT to DTVRecorder::HandleSingleProgramPMT to compile.

It looks like with the updated patch it's retaining the PCR stream and plays back fine in VLC again.

Thanks for the help!

[Richard Hulme <peper03@…>]

In 98c47b5c70014e741d4a9fd964ef1d0f5fe2a4f3/mythtv:

Make sure the PCR PID is not discarded if it is not listed in the PMT (i.e. the PCR PID contains only PCR packets).

Fixes #12422

[Richard Hulme <peper03@…>]

In 8332919c43b51ab06db9516150a2762498713c86/mythtv:

Make sure the PCR PID is not discarded if it is not listed in the PMT (i.e. the PCR PID contains only PCR packets).

Fixes #12422
(cherry picked from commit 98c47b5c70014e741d4a9fd964ef1d0f5fe2a4f3)