Opened 6 years ago

Closed 4 years ago

#11852 closed Bug Report - General (Fixed)

IPTV recording with RTP protocol creates unusable videos

Reported by: dvd-ram@… Owned by: Karl Egly
Priority: minor Milestone: 0.27.6
Component: MythTV - Recording Version: 0.27-fixes
Severity: medium Keywords: IPTV RTP
Cc: Ticket locked: no

Description

Hallo, i have installed mythtv 0.27 RC1 and would like use IPTV recording with German Telekom Entertainment. The German Telekom use the RTP protocol. IPTV streaming works perfect with vlc, but the recorded videos with mythtv 0.27rc1 are unusable. Have a look a the attachment.

Attachments (5)

playlist.m3u (1.1 KB) - added by dvd-ram@… 6 years ago.
IPTV Channel list German Telekom Entertain
IPTV_Snapshot.jpeg (216.1 KB) - added by dvd-ram@… 6 years ago.
IPTV Snapshot
vlc-record-2014-02-03-20h55m13s-rtp___239.35.10.1_10000-.ts (825.6 KB) - added by publicaddress004@… 6 years ago.
Recorded stream with vlc from german Telekom "T-Home" IPTV, channel "Das Erste HD"
captured_orf1.pcap.zip (1.7 MB) - added by mbt2015@… 4 years ago.
Captured RTP/UDP h264 stream.
0001-handle-RTP-packets-with-header-extensions-and-or-les.patch (2.2 KB) - added by Karl Egly 4 years ago.
proof of concept (compile tested)

Change History (36)

Changed 6 years ago by dvd-ram@…

Attachment: playlist.m3u added

IPTV Channel list German Telekom Entertain

Changed 6 years ago by dvd-ram@…

Attachment: IPTV_Snapshot.jpeg added

IPTV Snapshot

comment:1 Changed 6 years ago by publicaddress004@…

Did you had any success with fixing this? I just downloaded your m3u and implemented it in MythTV ... with the same result.

comment:2 Changed 6 years ago by dvd-ram@…

Workaround: After channel scanning with the playlist.m3u, open the mysql daterbase vom mythtv and rename the address from rtp to udp. Than it works!

comment:3 in reply to:  1 ; Changed 6 years ago by dvd-ram@…

Replying to publicaddress004@…:

Did you had any success with fixing this? I just downloaded your m3u and implemented it in MythTV ... with the same result.

Workaround:

After channel scanning with the playlist.m3u, open the mysql daterbase vom mythtv and rename the address from rtp to udp. Then it works!

comment:4 Changed 6 years ago by Karl Egly

Status: newinfoneeded_new

Please try fixing the playlist to use the udp scheme instead of the rtp scheme. Does it work then?

comment:5 in reply to:  4 ; Changed 6 years ago by dvd-ram@…

Replying to dekarl:

Please try fixing the playlist to use the udp scheme instead of the rtp scheme. Does it work then?

The German Telekom use the RTP protocol. UDP doesn't work. It's a Mythtv bug, not the wrong protocol.

Have a look at the multicast address list: http://grinch.itg-em.de/entertain/faq/allgemein/multicastadressliste/ http://www.ard-digital.de/Empfang--Technik/IPTV/Software-Download/Software-Download

comment:6 in reply to:  3 Changed 6 years ago by publicaddress004@…

Replying to dvd-ram@…:

Replying to publicaddress004@…:

Did you had any success with fixing this? I just downloaded your m3u and implemented it in MythTV ... with the same result.

Workaround:

After channel scanning with the playlist.m3u, open the mysql daterbase vom mythtv and rename the address from rtp to udp. Then it works!

This worked, thanks! IPTV is faster than DVB-C, great for LiveTV - yeah!!

comment:7 Changed 6 years ago by publicaddress004@…

OFFTOPIC WARNING: dvd-ram, your m3u lacks some channels, e. g. ZDFinfo, RTL Nitro, hr. I assume you know about that and you have posted the links already that contain complete lists of addresses, so others may have a look there.

comment:8 in reply to:  5 Changed 6 years ago by Karl Egly

Replying to dvd-ram@…:

Replying to dekarl:

Please try fixing the playlist to use the udp scheme instead of the rtp scheme. Does it work then?

The German Telekom use the RTP protocol. UDP doesn't work. It's a Mythtv bug, not the wrong protocol.

Can you post a short streamdump to take a look at in wireshark? It appears like something is wrong because RTP should "just work". The suggestion "change it to UDP in the playlist" was meant as an alternative to "change it to UDP in the database".

Changed 6 years ago by publicaddress004@…

Recorded stream with vlc from german Telekom "T-Home" IPTV, channel "Das Erste HD"

comment:9 Changed 6 years ago by publicaddress004@…

Unsure if the above is a "streamdump" ... :D Feel free to hint me how to properly create it if it is wrong!

comment:10 Changed 6 years ago by Karl Egly

Summary: IPTV recording with RTP protocol creates unusable videosIPTV recording with RTP protocol creates unusable videos

I'm looking for a network packet capture to look at with wireshark. Maybe 5 seconds from a public and 5 seconds from a private stream.

comment:11 Changed 6 years ago by JYA

Resolution: Invalid
Status: infoneeded_newclosed

change your playlist

comment:12 Changed 4 years ago by mbt2015@…

Resolution: Invalid
Status: closednew

Re-opening as changing the playlist is a workaround but not a real solution to the problem. Why is the video unusable when using the RTP protocol? I have the same problem with Austrian IPTV A1 TV.

comment:13 Changed 4 years ago by mbt2015@…

I believe that the related issue #11962 only exists because of the broken RTP support and the suggested UDP workaround. The official channellists of German Telekom Entertainment and Austrian A1 TV (see http://epggw.a1.net/a/service.plus.m3u?plain) contain RTP urls and not UDP urls. So I believe they only properly support RTP and not UDP.

The same holds true for VLC: It can play the RTP urls without problems, but has glitches too when playing UDP urls (similar to the glitches mentioned in #11962).

E.g. for A1 TV:

vlc "rtp://@239.2.16.1:8208"

... works fine

vlc "udp://@239.2.16.1:8208"

... has horrible glitches

comment:14 Changed 4 years ago by mbt2015@…

By the way: The problem still exists in v0.28 (tested with v0.28-pre-2945-g19b83b8)

comment:15 Changed 4 years ago by Karl Egly

Owner: changed from danielk to Karl Egly
Status: newaccepted

mbt2015, please provide the information that I asked publicaddress004 for. As this is a multicast transmission it is either transmitted wrapped into RTP or it is plain UDP, the client has nothing to choose. So if it works after fixing the playlist, then the playlist is simply wrong! But to prove either way we need to look at raw network packages and see if they have a RTP header or not.

comment:16 Changed 4 years ago by Karl Egly

Status: acceptedinfoneeded

Changed 4 years ago by mbt2015@…

Attachment: captured_orf1.pcap.zip added

Captured RTP/UDP h264 stream.

comment:17 Changed 4 years ago by mbt2015@…

Status: infoneededassigned

Hi dekarl, thanks for looking into this! I started Wireshark and ran

vlc rtp://@239.2.16.1:8208

for a few seconds. The resulting capture file (zipped Wireshark/tcpdump format) is attached as captured_orf1.pcap.zip​.

comment:18 Changed 4 years ago by JYA

Don't get too attached to what the url type is in the URL.

RTP or UDP are fundamentally the same, both use UDP transport.

VLC incorrectly differentiate UDP and RTP protocol due to their internal implementation. Many internet providers provide their playlist in a format VLC can understand as this is most likely most popular player.

I would close this bug as there's nothing to fix. If it works with UDP vs RTP well, that's it.

Last edited 4 years ago by JYA (previous) (diff)

comment:19 Changed 4 years ago by mbt2015@…

jyavenard, please do not close this.

Obviously MythTV also differentiates between UDP and RTP urls and uses different implementations for the two. It seems that for our German and Austrian IPTV streams (which I believe are just standard h264 RTP streams) currently the MythTV RTP implementation is completely broken (resulting in the "unusable" video that this bug is about), and the MythTV UDP implementation works but still has some glitches that occur every few seconds (and make the video very unpleasant to look at) (see #11962).

If you are saying that UDP and RTP is basically the same, then MythTV should use the same implementation for both URL formats, and ensure that this implementation works correctly (without glitches) for h264 streams.

comment:20 Changed 4 years ago by JYA

Surely, you can edit the playlist. it's something that needs to be done just once.

While they are basically the same, I do believe MythTV implementation *is* correct, it's VLC that isn't and it treats a UDP streams as RTP and vice-versa.

I don't see why we should make our handling incorrect just to be in line with VLC and provider giving invalid format playlist.

Also, VLC requires to use the @ symbol to indicate it's a multicast stream ; those are invalid yet they use it, and some providers also use those in their playlist.

There's nothing wrong with mythtv handling of UDP or RTP stream.

comment:21 Changed 4 years ago by mbt2015@…

Hi jyavenard! Sure I agree, editing the playlist once is no problem.

But the problem is that even when using udp:// MythTV still has glitches when playing those streams (see #11962). I believe that those glitches are due to the RTP headers that the MythTV UDP implementation does not consider correctly.

Did you have a look at the captured stream I attached (captured_orf1.pcap.zip)?

I did now using Wireshark. Wireshark has a nice option that can find RTP packets within the UDP stream (open "Preferences" from the menu, then "Protocols", "RTP" and activate "Try to decode RTP outside of conversations"). And indeed it does find RTP headers! So actually VNC is correct here. The rtp:// url is correct and udp:// is incorrect.

comment:22 Changed 4 years ago by mbt2015@…

(sorry, "VNC" in my comment above, should have been "VLC")

comment:23 Changed 4 years ago by mbt2015@…

The problem can also easily be reproduced by producing an RTP h264 stream using VLC and trying to play that in MythTV.

Run

vlc -vvv video.mpg ':sout=#transcode{vcodec=h264,acodec=mpga,ab=128,channels=2,samplerate=44100}:rtp{dst=localhost,port=5000,mux=ts,sap}' :sout-keep

where video.mpg is some (not too short) video file that VLC can read. This produces a stream available at rtp://localhost:5000

It can be successfully played with VLC using

  vlc rtp://localhost:5000

Running

  mythavtest rtp://localhost:5000

(or importing the stream as an IPTV channel into MythTV) will show the problem. The video is completely unwatchable. (Tested with VDPAU and Default playback.)

comment:24 in reply to:  17 Changed 4 years ago by Karl Egly

Replying to mbt2015@…:

I started Wireshark and ran

vlc rtp://@239.2.16.1:8208

for a few seconds.

Thanks, that is exactly what I'm looking for.

A1 TV appears to use encoders that implement RFC 5285 wrong. They use random bytes for padding when they must use 0 bytes for padding. Just open your trace in Wireshark and decode as RTP. Notice how most packets are malformed. This is due to the dissector trying to decode past the second header extension, when it actually is always just three random bytes of padding. (must be 0x00 0x00 0x00 instead)

Find attached a candidate patch (compile tested) for review.

Changed 4 years ago by Karl Egly

proof of concept (compile tested)

comment:25 Changed 4 years ago by Karl Dietz <dekarl@…>

In fee554cc8c2b339175c565364db6859ae85e9f14/mythtv:

unit test for RTP packet handling

Refs #11852

comment:26 Changed 4 years ago by Karl Dietz <dekarl@…>

In c8d65f00eb073946cb624318f28d0d38cec20977/mythtv:

extend unit test to include a regression test

Refs #11852

comment:27 Changed 4 years ago by Karl Dietz <dekarl@…>

In 90b98524e04e5a37112e038bd9c5da6958fbf4f1/mythtv:

handle RTP packets with header extensions and/or less then 7 TS packets

1328 is a magic number of 12 (minimum RTP header size) plus
7 times 188 (length of a plain TS packet)

Fix handling of padding length calculation while here.
from RFC 1889, page 11:

The last octet of the padding contains a count of how many padding
octets should be ignored.

Refs #11852

comment:28 in reply to:  14 Changed 4 years ago by Karl Egly

Status: assignedinfoneeded

Replying to mbt2015@…:

The problem still exists in v0.28 (tested with v0.28-pre-2945-g19b83b8)

I have pushed the possible fix to master two days ago and Mythbuntu nightly builds have picked it up yesterday. (I'm guessing you are still running Mythbuntu.) Please update and try a recording.

As the code is basically unchanged the fix would apply to fixes/0.27, too.

comment:29 Changed 4 years ago by mbt2015@…

Hi Karl!

I just tested it with the newest version from the Mythbuntu 0.28 repo. Both LiveTV and Recording work like a charm now!

Finally I can use rtp:// instead of udp:// again and there are no strange glitches any more!

Thank you very, very much for fixing this! Hermann

comment:30 Changed 4 years ago by Karl Dietz <dekarl@…>

In fbd5ef3122abff438d4d706b4ccc046213f47f22/mythtv:

handle RTP packets with header extensions and/or less then 7 TS packets

1328 is a magic number of 12 (minimum RTP header size) plus
7 times 188 (length of a plain TS packet)

Fix handling of padding length calculation while here.
from RFC 1889, page 11:

The last octet of the padding contains a count of how many padding
octets should be ignored.

Refs #11852

(cherry picked from commit 90b98524e04e5a37112e038bd9c5da6958fbf4f1)

comment:31 Changed 4 years ago by Karl Egly

Milestone: 0.27.6
Resolution: Fixed
Status: infoneededclosed

Merging to fixes/0.27 after success report. After looking at a packet capture from T-Entertain that also needed the same fix.

Note: See TracTickets for help on using tickets.