EDL commercial breaks not detected for H.264 recordings about pvr.mythtv HOT 16 OPEN

Hi, thanks for reporting the issue. Some guys reported this issue in the forum although I haven't reproduced the issue. The original error comes from backend call to retrieve marks. You can try it with your web browser as the following:

http://IP_BACKEND:6544/Dvr/GetRecordedCommBreak?ChanId=1002&StartTime=2017-01-06T00:26:47Z&OffsetType=Duration

where "StartTime" is UTC time. Please let me know what happen with this test.
Issue is related to the recorded map table. Without map none duration conversion is available. Kodi handles only time position for these features (edl and bookmark).

from pvr.mythtv.

To check if you have a recorded map for the recording, query your mythconverg database as the following:

select * from recordedseek where chanid=1002 and starttime='2017-01-06 01:26:47' and type=33;

type=33 means mark for duration (millisec)

from pvr.mythtv.

janbar,

Thanks for your quick response. I queried the backend with the URL you provided (correcting IP_BACKEND accordingly) and got the following results:

<CutList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.0" serializerVersion="1.1"> <Cuttings/> </CutList>

The following SQL query returned no results:

select * from recordedseek where chanid=1002 and starttime='2017-01-06 01:26:47' and type=33;

From what I understand, H.264 videos (or perhaps MP4 program streams... not really sure which aspect is at play here) don't get/need a seektable. When you call mythcommflag -rebuild for a transcoded transport stream MPEG2 file, it actually clears out any entries if you may have had for the recording after it is transcoded to H.242 MP4. Does the addon require the seektable to work?

from pvr.mythtv.

Only fonctionalities EDL and Bookmark require seektable to do conversion. Because kodi speaks with time position. Also it is required to handle recording with variable framerate.
Here I only have h264 recordings from the air. And seektable is always filled. Probably your transcoding erase it without refilling it. It should.

from pvr.mythtv.

This sql query will show you recordings having seek info filled:

select distinct chanid, starttime from recordedseek where type=33;

from pvr.mythtv.

None of my mp4 shows have seek info; only the untranscoded mpegts files contain these rows. Like I said earlier, when I run mythcommflag -rebuild the utility actually removes the seektable entries if they were generated previously before the recording was transcoded. This is a fairly well known behavior for files that have been transcoded into program stream containers like mpg or mp4; the utility removes the recordedseek seektable entries because MythTV doesn't need them. This does not mean that the EDL entries are removed from the recordedmarkup table. They are still maintained there and used quite nicely within the mythfrontend application.

I think that you are seeing seektable entries for your h.264 recordings in recordedseek table because your recordings are delivered a transport stream (mpegts) container. I think that all broadcast mpeg2 or h.264 video streams are delivered in mpegts containers. The transcoded recordings I am working with are stored in standard mp4 program stream containers created by ffmpeg. MythTV has no problem playing them; apparently since they are program streams, they have a fixed length and don't require the additional overhead that the mpegts containers need to help compensate for the unreliable broadcast signals they were designed for.

Does the pvr.mythtv addon use the data rows from within the recordedseek table for the recording or is it simply checking for the presence of timecodes in the table to verify if an EDL is present? I would think that if you want to check for the presence of the EDL marks, a more reliable check would be to query the recordedmarkup table instead since that is where they are stored https://www.mythtv.org/wiki/Recordedmarkup_table. Forgive me; but I am not a C++ coder; I am a C#/Java/JavaScript web dev. I will try looking through the source to see if I can figure out how this recordedseek table data is consumed to see if I can provide more help.

Thanks again for your help!

from pvr.mythtv.

Yes I anderstand the case. The addon doesn t access directly to the database. It uses the mythtv api. For EDL it calls a service to retrieve commbreak list with time offset. This service need marks 33 stored in recordedseek to convert offset to time offset.
Until i knew markup was saved with frame offset position which is unusable by any player other than mythfrontend. And that works only for recording having constant framerate.
Transcode seems to make file with constant framerate and then recordseek rows are no longer required for mythfrontend. That breaks usability with other player uses standard cutlist with time offset.
The solution should to enable cutting of commbreak when transcoding.
Also I willl have a look on recordedmarkup for any change. Maybe it stores time offsets now.

from pvr.mythtv.

IMO the root of the issue is the mythtv api calls for getting commbreak and cutlist (I am the contributor for them), and same thing for api call to set bookmark.
This issue should be addressed to Mythtv. One solution could be check for mark 33 and fallback to constant framerate to compute the position or time offset.

from pvr.mythtv.

I did a little more digging. The Myth API apparently does allow selections between BYTE or DURATION offsets. I retried the API query after correcting the starttime and chanid, and issued the following query:

http://SERVER_IP:6544/Dvr/GetRecordedCommBreak?ChanId=1081&StartTime=2015-09-27T23:30:00Z&OffsetType=DURATION

The results I got were this:

<CutList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.0" serializerVersion="1.1"> <Cuttings> <Cutting> <Mark>4</Mark> <Offset>16746</Offset> </Cutting> <Cutting> <Mark>5</Mark> <Offset>23360</Offset> </Cutting> <Cutting> <Mark>4</Mark> <Offset>50374</Offset> </Cutting> <Cutting> <Mark>5</Mark> <Offset>61297</Offset> </Cutting> <Cutting> <Mark>4</Mark> <Offset>82322</Offset> </Cutting> <Cutting> <Mark>5</Mark> <Offset>89540</Offset> </Cutting> <Cutting> <Mark>4</Mark> <Offset>106066</Offset> </Cutting> <Cutting> <Mark>5</Mark> <Offset>115625</Offset> </Cutting> </Cuttings> </CutList>

So it appears that while I don't have a seektable, I do still have defined comm breaks reported by the API. The DURATION option is case sensitive. I retried with Duration as the OffsetType as in your example and got the following:

<CutList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.0" serializerVersion="1.1"> <Cuttings/> </CutList>

I assume the DURATION is frames since the 720p60 ATSC broadcast video is ~60fps. which seem to line up well with the timeline of the video. With the base framerate, it would be trivial to calculate the mark offset times. Unfortunately, there is nothing in the metadata in the API indicating this. I queried the recordedmarkup table and see that there is a mark of type '32' that provides the framerate of 59940, which would allow calculation of the mark offsets. The other option, BYTES, is probably not what we want either. Does Kodi's own API provide any info on the stream that would give us this missing piece of information to calculate time-based offsets?

from pvr.mythtv.

Jean-Luc,

(Edit - forgot to mark some XML elements as code, causing them to be stripped out when viewed)

I agree that the MythWeb API is a little deficient when it comes to the EDL. When you specify DURATION, the unit is frames; but if the framerate is unknown, the offsets are rather pointless. Either the schema needs to be extended to add an additional element for <DurationFramerate> or something to the <Cutlist>, or the information needs to be added to the <Recording> element in <Program>.

If all you need is duration in the case you describe from mark 33, couldn't you derive the same figure from the existing element in the metadata from the Myth web API? The following elements exist within this element:

<StartTs>2015-09-27T23:30:00Z</StartTs>
<EndTs>2015-09-28T00:01:31Z</EndTs>

The offset could be calculated by simply calculating the offset between and . Is the 1s resolution too coarse for your purposes in this fallback scenario assuming constant framerate?

Tim

from pvr.mythtv.

Assuming constant framerate, you need only the framerate to compute the time offset for mark which is provided as frame offset. The total duration cannot help us without the total frame count, obviously to estimate a framerate.
The addon has a demuxer able to provide the theorical framerate of the recording. I used it to compute EDL mark in the previous version. Now MythTV 0.28 provides EDL marks with the api. So good way is to improve this api. It will take some time. I will propose a patch to fix this case. And you could explain the use case in the track ticket.
I let you know whrn I will post the patch to MythTV dev.

from pvr.mythtv.

I am glad to help in any way I can. I will await your instructions.

from pvr.mythtv.

As a workaround until the MythTV web API is made aware of program stream commercial breaks or in case it is not, do you think it would be possible to create recordedseek table entries from recordedmarkup entries to trick the Myth web API into reporting commbreaks? The byte offsets are lost and would need to be synthesized or just faked if they are not needed. I was thinking of populating type 9 and 33 marks based on type 4, 5 and 33 recordedmarkup rows if we only need seek marks that correspond with the commercial breaks and end of program. Since the addon is using duration (frame count), I would think that the byte offsets are not even necessary. I'm not sure if this would have any unintended consequences.

from pvr.mythtv.

Yes, your workaround will work.
Also I will try to save some times to work on the MythTV patch to fix it. But I am afraid mythtv team won't backport it to 0.28. We will request for backport, considering it is a bug.

from pvr.mythtv.

Yes, the workaround definitely works. I generated seek marks for that show I've been playing with using the following query:

set @fps = (select data/1000
from mythconverg.recordedmarkup
where chanid=1081 and starttime='2015-09-27T23:30:00' and type=32);

replace into mythconverg.recordedseek
select chanid, starttime, mark, floor(mark*1000/@fps ) AS offset, 33 as type
from mythconverg.recordedmarkup
where chanid=1081 and starttime='2015-09-27T23:30:00' and (type=4 OR type=5 OR type=33) ORDER BY mark;

I probably didn't need to use replace into, but I wanted a query that was idempotent without having to clear the table of entries for the chanid, starttime and mark PKs first. Laziness, I guess. I can definitely add this to my transcode script to patch the seektable. Now to figure out a way to patch the ones I've already transcoded in batch...

Thanks for your help.

from pvr.mythtv.

Kirt,
based on your query, figured to make a repo:
https://github.com/jakeapp/myth-h264-edl-for_kodi/blob/master/h264_edl.pl

hoping you could have a look at lines 351 - 358 (not confident it will do the trick as is)

from pvr.mythtv.