Opened 9 years ago

Closed 6 years ago

#12444 closed Bug Report - General (Won't Fix)

/Content/GetRecording sets partial Content-Length when serving unfinishing recordings

Reported by: ajb@… Owned by: stuartm
Priority: major Milestone: 0.28.2
Component: MythTV - Services API - Backend Version: 0.27-fixes
Severity: medium Keywords: Content-Length Chase-Play
Cc: Ticket locked: no

Description

Hi,

I'm trying to do chase-play on an unfinished recording using the SmartMythTV app on a Samsung smart tv. When the client requests an as yet incomplete recording, the response includes Content-Length and Content-Range headers with the current size of the file. The player always stops before the end of the program although the recording has completed in the meantime. I suspect the client is obeying the indicated length of the data rather than reading until hitting EOF.

Normal recording

GET /Content/GetRecording?ChanId=3004&StartTime=2015-05-03T15:00:00Z HTTP/1.1
User-Agent: Lavf52.104.0
Accept: */*
Range: bytes=0-
Connection: close
Host: 192.168.3.45:6544
                                                                                
HTTP/1.1 206 Partial Content
Date: 3 May 2015 16:48:26
Server: Linux 3.13.0-49-generic, UPnP/1.0, MythTV 0.27.20141016-1
Accept-Ranges: bytes
Cache-Control: no-cache="Ext", max-age = 5000
Content-Range: bytes 0-1108512483/1108512484
Connection: Close
Content-Type: video/mpeg
Content-Length: 1108512484

Unfinished recording I want to chase play

GET /Content/GetRecording?ChanId=3001&StartTime=2015-05-03T16:52:00Z HTTP/1.1
User-Agent: Lavf52.104.0
Accept: */*
Range: bytes=0-
Connection: close
Host: 192.168.3.45:6544
                                                                                
HTTP/1.1 206 Partial Content
Date: 3 May 2015 16:52:27
Server: Linux 3.13.0-49-generic, UPnP/1.0, MythTV 0.27.20141016-1
Accept-Ranges: bytes
Cache-Control: no-cache="Ext", max-age = 5000
Content-Range: bytes 0-14843351/14843352
Connection: Close
Content-Type: video/mpeg
Content-Length: 14843352

As you can see there's no difference in the type of headers sent between the complete and the incomplete recording. I'm thinking that if the Content-Range and the Content-Length were not present the Smart TV player would continue until the EOF. Is that a reasonable assumption, or does HTTP 1.1 mandate them?

Thanks,

Andrew

Change History (13)

comment:1 Changed 9 years ago by stuartm

Owner: set to stuartm
Status: newaccepted

comment:2 Changed 9 years ago by stuartm

Milestone: unknown0.28
Priority: minormajor

comment:3 Changed 9 years ago by ajb@…

Hi Stuart,

I'm thinking that it should be possible to fix in 0.27 by switching to chunked transfer encoding for all files, as then we only need to know if the file is complete when we hit the end of file

Basically it means omitting the Content-Length header (and presumably Content-Range - I'm not sure what that's used for myself), and replacing it with Transfer-Encoding: Chunked

Then in SendData?() after each pDevice->read() of 64k the server writes to the client

FFFF\r\n
DATA\r\n

where FFFF is the length actually read from file as hex number in ascii followed by CR LF and the corresponding raw DATA that was read from the file is followed again by CR LF

when EOF is hit the response to the client is

0\r\n

The zero indicating the end of file.

Actually as an enhancement before writing zero to the client you might try a slight delay then read it again just in case data became available in the meantime.

Here are a few sites with more info about the transfer method

http://en.wikipedia.org/wiki/Chunked_transfer_encoding

http://129.33.205.81/support/knowledgecenter/SSGMCP_4.2.0/com.ibm.cics.ts.internet.doc/topics/dfhtl_httpchunking.html

http://www.tcpipguide.com/free/t_HTTPDataLengthIssuesChunkedTransfersandMessageTrai-2.htm

I didn't try this yet, as I only have a live mythtv instance and the wife will kill me if I break it!!!

comment:4 Changed 8 years ago by Stuart Auchterlonie

Milestone: 0.280.28.1

Moving unresolved tickets to next point release

comment:5 Changed 8 years ago by Mitch Capper <mitch.capper@…>

I am curious how this would affect seeking, extensions may need to be used but not sure how many clients then support it. Will chunked just be an option? HLS obviously works for streaming in-progress recordings too.

comment:6 Changed 8 years ago by ajb@…

Hi Mitch,

As understand it the chunked transfer is a requisite part of HTTP 1.1, which is quite old now, see RFC 2616 http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1 section 3.6.1, so I expect that all modern clients would support it. That said, since the fix didn't make it into Myth 0.27, switching wholesale to chunked transfer encoding would not be necessary as I believe it's possible for the webserver in 0.28 to determine whether the recording has finished. This means the chunked transfer would be used only when the file is incomplete. I don't know about seeking methods, but I currently use a Samsung smarttv with the SmartMythTV application https://sourceforge.net/projects/smartmythtv/files/. I very much doubt the TV has enough memory to store a complete recording locally (I've played some 3GB recordings ok) to allow on disk seeking, so I believe that it must interrupt transfer, and request new transfer at an offset for both forward and backward jumps. The Samsung API has callback hooks on the player component for buffering start/complete/progress conditions, so by inference playing must start before the whole file has been downloaded.

I presume 'HLS' is 'HTTP Live Streaming' this https://www.samsungdforum.com/Guide/?FolderName=tut00055&FileName=index.html suggests it is supported by the TV API. Is that already implemented for recordings? Please bear in mind that I want to start watching at the beginning of the programme, not just pick up at the current position.

Thanks,

Andrew

comment:7 Changed 8 years ago by Mitch Capper <mitch.capper@…>

Hi Andrew, chunked transfer itself should certainly be supported by most clients, however in the spec for chunked transfer I don't see how a total length would be indicated. For chunked video transfer if the video has an index it could allow the device to have an idea of where to seek to (but that wouldn't be for in progress I believe). So the problem one runs into is I don't see an easy way of communicating the 'length so far' in terms of chunked transfer except with extensions which I don't think many clients would support. If this is the case the device would only know how far it could seek but downloading as much as possible in a chunked encoding environment, verse right now where any time it gets the content length it knows how far it could seek at that point. So this could possibly (if not left as a request option) degrade the experience by not allowing much seeking at all for chunked transfers. I think an easy work around would be to allow the requester to specify if they want chunked or not.

HLS is the live streaming and is currently implemented by mythtv. It does start at the beginning of the recording and is meant to work for in progress recordings ( so the manifest is consistently updated). Further it has full seek support (as every 4 seconds a new chunk is created and you can seek for that chunk). See the service API for examples and the API calls for HLS but its pretty easy to use.

comment:8 Changed 8 years ago by Mitch Capper <mitch.capper@…>

I should also say I am not an expert on chunked encoding and only briefly read the RFC so I could be wrong if anyone has more information there:)

comment:9 Changed 8 years ago by ajb@…

Hi Mitch,

Yes I understand your point now. I wouldn't wish to limit the ability to seek on an unfinished recording. So I'd like to try switching the client to use HLS, in the case that the recording is unfinished. I've looked at the 0.28 services API, but the nearest thing I can see seems to be GetLiveStream?, but I don't understand how that could be used to watch a recording?

How would a call from the client like below be converted to HLS?

http://BackendServerIP:6544/Content/GetRecording?ChanId=34736&StartTime=2011-08-29T18:59:00

I did read this, https://www.mythtv.org/wiki/HTTP_Live_Streaming_Server but it was non obvious to me how to modify the SMartTV app to do it.

Thanks,

Andrew

comment:10 Changed 8 years ago by Bill Meek <keemllib@…>

Hi, take a look at Content/AddLiveStream?

comment:11 Changed 8 years ago by Mitch Capper <mitch.capper@…>

Yes you want to call AddLiveStream? with whatever args you want (recid is the only required one), it will return to you livestreaminfo, which will almost certainly not be ready to play yet. So wait for it to actually start processing (sleep for a few seconds and then call GetLivestream? with the livestream id to check on the status) and then just pass the HLS url to the TV and in theory you should be good to go. If your backend is underpowered you may want to wait to make sure you don't play faster than it can transcode.

Keep in mind mythtv does not automatically cleanup transcoded streams (so once you start streaming it will transcode to the end and then leave the files by default). You can resolve this by either cleaning up when done (RemoveLiveStream?) or telling mythtv to only keep so many segments around.

comment:12 Changed 7 years ago by Stuart Auchterlonie

Milestone: 0.28.10.28.2

Moving remaining open 0.28.1 tickets to 0.28.2

comment:13 Changed 6 years ago by Stuart Auchterlonie

Resolution: Won't Fix
Status: acceptedclosed

Closing any remaining tickets for 0.28, if the issue persists, feel free to reopen and align to v29 or master

Note: See TracTickets for help on using tickets.