roman380 / gdcl.co.uk-mpeg4 Goto Github PK

I setup a bare-minimum win32 project:
https://gist.github.com/LinArcX/b43ce8b1905b165de2314e6f9b202c8c

And i want to use MPEG4-encoder with it instead of MPEG2-Encoder.

I figured that out that i have to include "Module_i.h". And used this code to instantiate the class:

hr = CoCreateInstance(CLSID_MuxFilter, NULL, CLSCTX_INPROC_SERVER, IID_PPV_ARGS(&pMpeg4Encoder));

But it gives me this linker error:

1>Linking...
1>main.obj : error LNK2001: unresolved external symbol _CLSID_MuxFilter
1>D:\workspace\c++\projects\directshow_graph_tester\Debug\directshow_graph_tester.exe : fatal error LNK1120: 1 unresolved externals

Enclose installation script provided by Andrew Griffiths and produced installer

The faulty file is attached: Output-02-0000.zip

There is a problem with tiny MP4 files, supposedly both demultiplexer and multiplexer problem: a 5 frame movie created by mp4mux has a zero "mdhd_duration" as reported by MediaInfo. It is still playable but not with mp4demux.

mp4demux streams zero frames and report EOS, it somehow skips frames because it reads duration of zero from the input file and abort streaming reaching stop time.

No actual reset taking place:

void 
Mpeg4Demultiplexor::DeselectSeekingPin(DemuxOutputPin* pPin)
{
    CAutoLock lock(&m_csSeeking);
    if (pPin == m_pSeekingPin)
    {
        m_pSeekingPin = pPin;
    }
}

Was commented out while doing #8 and never brought back.

While sample delivery is the expected method, implemented is alternate method primarily addressing, as it seems, accurate file closing. The mix of methods does not look accurate and, among other, appears to be a performance bottleneck. It makes sense to revise this part of the multiplexer and elaborate a more accurate solution.

When seeking to key frame the seek operation completes with demultiplexer returning zero position

Raw RGB with MP4 multiplexer is a rare case, but this scenario is basically a supported one and should work out fine. The problem is that RGB subtypes are not generic FourCC subtypes and simplified checking in the multiplexer is incorrect and results in no track and dropped samples.

Multiplexer should be discarding preroll data since otherwise it includes data into output file shifting all time stamps respectively (stream synchronization is however not lost).

The problem takes place, for example, with FFmpeg based AAC encoder which outputs an additional packet with -1 PTS for 0 input PTS. Even though this packet reaches multiplexer as "preroll data -32 ms to 0 ms), multiplexer includes it and shifts video stream so that zero time video frame appears as +32 ms video frame in the file. This is unwanted behavior because the offset jumps in for no reason.

Rather, multiplexer should discard preroll data and assume that upstream connection is otherwise responsible that for proper data markup, e.g. that first non-preroll media sample is a splice point.

Line 465 in SampleToCTS() needs a cast, should be:
LONGLONG tThis = m_tAtBase + TrackToReftime((LONGLONG)(nSample - m_nBaseSample) * nDuration);
Without this change, the multiply is long * long and when you have a lot of samples this wraps around when the product reaches 2^31 (which admittedly takes a while, but I have a customer with a file that is > 1 day long!

Sorry to ask what may be a trivial COM question, but can you point me in the right direction to access GetMediaSampleTimes()? I cannot see it as an interface to the demultiplexor when using GraphStudioNext.

IStreamPtr in MuxOutput::CompleteConnect triggers and exception which needs to be either caught or not generated in first place (in an exception-free project).

...as suggested by Andrew G.

i don not really know how to use a custom filter in my computer?
Does it work when other people 's computer don't register the filter?

Hello,

Not sure if I'm doing something incorrectly, or if this is a feature that is not implemented in the MP4 Muxer. I can't get it to properly record H264 streams when each frame/picture is sliced into multiple NAL units.

For some context, I am working on a Directshow-based application that renders and records up to 24 video and audio feeds, coming from IP cameras, hardware encoders/streamers and/or desktop screen captures encoded using x264. I have the following constraints:

• All feeds have to be synchronized, with minimal latency
• Due to large number of feeds, I cannot transcode them (too much CPU), so I have to mux the signals as they arrive

When I use RTSP/H264, or x264 encoding with single NALU for each frame, the app and muxer works great. However, as I lower the fps on the x264 encoding (say 5 fps), this introduces a 4-5 seconds latency. To get an acceptable latency, I use the "superfast" preset, then I try the tune zerolatency (which includes the sliced-threads option). With sliced-threads, frames are broken down into multiple NALUs (1 per CPU core). The LAV Video Decoder correctly decodes the sliced frames, but the muxer only saves the first slice (bottom macroblock of the picture). The rest of the picture flashes gray/green (in NV12 pixel format), and when I try seeking this recorded video, it crashes my GPU.

I have looked into a few options:

1. Concatenate the NALUs into a single NALU per frame, leaving start codes in between. This works with LAV decoder, but not the muxer.
2. Concatenate the NALUs by removing the start codes and nal_unit_type byte. Doesn't work.
3. Tried multiple options on x264 to mimic the zerolatency tune, but without slicing the frames, but I can't get below a few seconds of latency (at 5 fps)

I'd like to get some help, ideally by having the mp4muxer work with sliced frames, or if you can point me to code/sample on how to concatenate sliced frame NALUs, or if someone knows x264 options that can achieve the lowest latency without slicing frames. I am using very recent versions of ffmpeg and Live555 (from december 2018/january 2019).

Thank you

In MoveiWriter.cpp line 573 in TrackWriter::Close() you have the line:
LONGLONG scaledur = long(Duration() * m_pMovie->MovieScale() / UNITS);
This has unfortunate effects when (with MovieScale() set to 360000) Duration() reaches 6000 seconds (6x10^10 in REFERENCE_TIME) as the cast to long truncates the value - or makes it negative. The fix is to remove the long(...) cast to match line 68:
LONGLONG scaledur = Duration() * m_pMovie->MovieScale() / UNITS;

Default multiplexer behavior is to align time by the earliest media sample - all tracks are in sync adjusted respectively and earliest time stamp has zero time. Additionally, duration is aligned by the last media sample.

For transcoding scenarios I would prefer to be able to keep original times intact (it's okay if there is an initial gap or duration is past last payload data).

While this is optional, and is not so often used, the time is quite helpful with sophisticated playback scenarios where the player needs to resolve original file sample time. See IMediaSample::SetMediaType for details.

Just spent the weekend thinking that I had gone mad as noticed that all timing in files was messed up. I reverted the mp4mux to the last GDCL from the original GDCL before your version of the project and the times are back to the real frame times.

I do not know at which version this happened as I do not know where to look in the source for this. It appears that you are setting the frame times to be the frame index multipled by the nominal stream frame interval... or maybe you have removed all frame times so that the only information is the frame index.

The Microsoft DirectShow doc is very unhelpful as regards frame times... the Get/SetMediaTime() functions is frame indexes, the Get/SetTime() functions are the times.

In my usage, it is vital that the correct frame times are used. The mp4demux seems to be working OK.

You can demonstrate this easily with a web cam. Set it streaming in a reduced light situation so that the frame rate falls significantly, then replay the result.

There should be either a graceful check or ability to accept values while inactive

More detailed description of the problem is here. It is not really a bug of demultiplexer, however it is the problem that bring MP4 demultiplexer down and a small hotfix can resolve the problem.

Especially that the problem takes place with SDK sample source. Ignoring the problem, demulttiplexer attempts to call a method of null interface pointer (causing access violation).

It appears that NAL unit scanning has a bottleneck, which might significantly reduce performance and throughput processing H264 stream.

Presumably, the problem is around H264ByteStreamHandler::WriteData and needs an improvement over there:

The encoder outputs H.264 as byte stream with start codes, and is processed in the Mux filter by H264ByteStreamHandler which replaces the start codes with length values, so that each NALU is length-prefixed (apparently H.264 in an MP4 file must be in length-prefixed format, not start-code format).

In H264ByteStreamHandler::WriteData(), the input H.264 sample buffer is parsed for NALUs, and each NALU ends up being written separately to the file in patm->Append() which calls MuxOutput::Replace().

So patm->Append() is being called multiple times for each input H.264 sample/buffer.

I've encountered a 25fps file approx 30 mins in length that has a CTTS table.

I found that seeking near the start of the file was fine, but the further you seeked into the file the longer it took to complete the seek - e.g. about three-quarters of the way through the file took about 10 seconds on a fairly fast laptop.

I found that all of the time was being consumed inside DTStoSample. As an experiment I removed the call to CTSOffset (on line 387) and the seek was then near instant.

CTSOffset always scans from the start of the CTTS table until it finds the required sample, but it is being called for every sample in DTStoSample's for loop - that seems to be what is taking so many secs to complete.

On the assumtion that CTTS values are probably going to be only 2 or 3 frames duration maximum, I tried only doing the CTSOffset call when the loop gets 'fairly close' to the specified tStart value - I just chose an arbitrary 0.5 sec for 'fairly close'. This seemed to do the trick:

LONGLONG tLimit = m_tAtBase + TrackToReftime(nEntries * nDuration)

if (tStart < tLimit + 5000000)
{
    tLimit += CTSOffset(m_nBaseSample + nEntries);
}

I'm not sure whether some mp4 files CTTS tables might break this, so rather than using an arbitrary 0.5 sec value for 'fairly close', perhaps a better approach would be, when the CTTS is read-in do a quick scan to find the biggest positive value; then use that for the 'fairly close' value?

In Index.cpp line 301 in SampleTimes::Parse(), the expression on the rhs can overflow:

	m_total = 0;
	for (int i = 0; i < m_nSTTS; i++)
	{
		long nEntries = SwapLong(m_pSTTS + 8 + (i * 8));
		long nDuration = SwapLong(m_pSTTS + 8 + 4 + (i * 8));
		m_total += (nEntries * nDuration);	// <- this can overflow
	}
	m_total = TrackToReftime(m_total);

A simple solution is to make nEntries a LONGLONG.
I have a user who hit this problem with a file with all frames at the same interval at 42000(!) seconds into replay from which I deduce that the file has a movie scale of 100000.

Using latest Intel H264 Encoder from Intel INDE (that now contains Intel Media SDK) for a some reason muxer do not work if resolution is 2560x1440 (but work on 1920x1080, looks like maximal resolution supported). With original muxer all ok.

Sample graph - https://yadi.sk/i/qtFtst99hCui7 , used file http://red.cachefly.net/TimeScapes4K2560p.mp4

There is a problem in MuxInput::Receive taking place with concurrent stop/release of the filter, when streaming is stopped without completion and reaching end of stream.

m_pTrack->Add call near the end of the method might be running with null m_pTrack, even though there is a check higher in code. The method requires thread safety update and proper serialization.

Presumably, stsz atom of MP4 file is invalid, the file is not playable with any player. In the same time use of another MJPEG compressor produces a valid file with similar rest of the pipeline.