wang-bin / mdk-sdk Goto Github PK

I would like to use mdk-sdk in a project which could be released with a GPL license (it currently has a more permissive license).

I read:

Free for GPL softwares

Great!
But for that, I need that mdk-sdk source code is provided with GPL license, as the GPL license explicitly stipulates that the whole software must have its source code provided on request and with a license compatible with the GPL (it is actually purpose of GPL: forbidding the usage of GPL code with code having a license incompatible with GPL).

If my code is GPL, mdk-sdk must be compatible with it in order to be able to distribute my software. I checked the packages on Sourceforge, I saw the license “Free for GPL softwares” but without any source code, so mdk-sdk is free for GPL softwares but incompatible with GPL softwares, at least with the public packages I found.

Just to be clear before I ask for a key to the developer: will the source code with a GPL compatible license be delivered with the key? If not, how is it expected that mdk-sdk is compatible with the constraint requested for having the right to use mdk-sdk for free?

mdk version：0.10.3
QT 5.12.0
windows x64

mjpeg decoders is async. plays fine in topoedit. In mdk crash at IMFTTransform.GetStreamIDs()

MDK 0.11.1 (git d71c060) - Multimedia Development Kit. Copyright (c) 2016-2021 WangBin(QtAV author) Build for: Windows0x0600 x86_64; Clang12.0.1 MSVC1911; MSVCRT14.28.29333.0/msvcp140.dll; 02:13:03 May 17 2021

MediaStatus状态一直是loading,本地视频和http的正常播放

I am trying to save some frames samples from a video without starting to play. How can I do that?

player->set(PlaybackState::Paused);
声音也没办法调，静音也是

Hallo,
i try to use Qt QIODevice., but protocol is not supported:

     iodev = dynamic_cast<QIODevice*>(&lfh);
     _player.setMedia(QString("qio:%1").arg(qintptr(iodev)).toUtf8().constData());

log:

qio: Url or protocol is not supported by MediaIO FFmpeg
Try to use default MediaIO

when i try to compile the Qt plugin, compiler said: "mdk/MediaIO.h" not found.
I will try to handle the byte stream for the mdk. In QtAV i can to this with QIODevice, but i need also a solution for the mdk.

Thanks in advance.

Hi, thank you for the great library,
I'm wondering whether it is possible to open a video source from camera stream, or does the new player work only with video files?

I was doing some tests with the example: "qmdkqmlplay.pro"

setting the source in MDKPlayer like this (I'm working under windows):

MDKPlayer {
        id: player
        source: "dshow:video=USB2.0 HD UVC WebCam"
        ...
}

And I changed the MDKPlayer setSource method to this, in order to allow sources that are not local files:

void QmlMDKPlayer::setSource(const QString & s)
{
	internal_player->set(PlaybackState::Stopped);
	qDebug() << "new source = " << s;

        auto filename = QUrl(s).toLocalFile();
        if (filename != "")
	{
		qDebug() << "source is a local file";
		internal_player->setMedia(filename.toUtf8().data());
	}
	else
	{
		auto source = s.toUtf8().data();
		qDebug() << "source is not a local file: "<< source;
		internal_player->setMedia(source);
	}

	auto cb = [this](int64_t position, bool* boost) {
		qDebug() << "Inside prepare callback";

		auto info = this->internal_player->mediaInfo();
		qDebug() << "startTime = " << info.start_time;
		qDebug() << "duration = " << info.duration;
		qDebug() << "bit_rate = " << info.bit_rate;
		qDebug() << "size = " << info.size;
		qDebug() << "format = " << info.format;
		qDebug() << "streams = " << info.streams;
		qDebug() << "metadata = {";
		for (auto &c : info.metadata)
			qDebug() << "metadata ("<< c.first.c_str() << "," << c.second.c_str() <<")";
		qDebug() << "}";

		if (info.video.size()>0)
		{
			auto &v = info.video[0];
			qDebug() << "video startTime  = " << v.start_time;
			qDebug() << "video duration   = " << v.duration;
			qDebug() << "video num_frames = " << v.frames;
			qDebug() << "FPS estimated: " << v.frames / (v.duration/1000.0f);
			qDebug() << "FPS from codec infos: " << v.codec.frame_rate;
			qDebug() << "video codec = " << v.codec.codec;
		}

		// return true to continue without unloading
		return true;
	};

	internal_player->prepare(0, cb);

	//internal_player->prepare();
	play();
}

but nothing seems to happen. Camera led is still off and video area is black.
MediaInfo information that I print in the prepare callback seems to be empty, debug output shows this:
startTime = 0
duration = 0
bit_rate = 0
size = 0
format =
streams = 0
metadata = {
}

I tried to open the same source with ffplay, with the command:

.\ffplay -f dshow -video_size 1280x720 -i video="USB2.0 HD UVC WebCam"

and the video stream is shown correctly.

Am I missing some important steps in order to show an online video stream? Is this feature still supported in mdk player, or this new player only works with offline video files?

Thank you in advance for clarification and help :)
Have a grea day,
Andrea

播放网络的MP3，一直播放，不拖动进度，会在约3分30秒左右不能继续播放。

https://download.niushibang.com/live/teacherFile_test/ed9a3775-1210-4f73-aee7-b7a70c531525/8311033988141615780914724.mp3

这个问题在QtAV也出现了，所以来试试mdk，但也出现了同样的现象...

Qt 5.15.0 MSVC2019 32bit

用的是Qt自己的示例《Media Player Example》，然后改了点代码让播放列表里面存在一个URL。

// main.cpp

/*other code...*/
int main(int argc, char *argv[])
{
    /*other code...*/

    QList<QUrl> urls;
    urls << QUrl("https://download.niushibang.com/live/teacherFile_test/ed9a3775-1210-4f73-aee7-b7a70c531525/8311033988141615780914724.mp3");
    player.addToPlaylist(urls);
   
    player.show();
    return app.exec();
}

补充一下，在QtAV，停下来后，过一阵子会报一个错：
error reading stream data [timeout](FFmpeg ffffffffabb6a7bb: Immediate exit requested)

internal created contexts only

Hi,

I want to use mdk-sdk v0.14.0 in a Qt5 App on Android. I compiled Qt 5.15.2 with the Android NDK r21e. As soon as I start the App I can see a UnsatisfiedLinkError exception in Logcat:

2022-02-09 19:36:43.477 5606-5639/org.qtproject.example E/AndroidRuntime: FATAL EXCEPTION: qtMainLoopThread
    Process: org.qtproject.example, PID: 5606
    java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "__emutls_get_address" referenced by "/data/app/~~E81EpQdeZGcLIpBfGIRNgQ==/org.qtproject.example-SDpDHEObWK1YiOGa1ak5-A==/lib/arm/libmdk.so"...
        at java.lang.Runtime.load0(Runtime.java:939)
        at java.lang.System.load(System.java:1628)
        at org.qtproject.qt5.android.QtNative$3.run(QtNative.java:473)
        at org.qtproject.qt5.android.QtThread$2.run(QtThread.java:87)
        at org.qtproject.qt5.android.QtThread$1.run(QtThread.java:61)
        at java.lang.Thread.run(Thread.java:923)

I suspect that it is because the App is bundled with an older libc++_shared from the NDK r21e than mdk-sdk (NDK r23b). Can you provide a release of mdk-sdk compiled with NDK r21e? Qt5 and 6 can't be compiled with a NDK greater r22.

Thank you

When I build my project on PI4 I get tons of these messages during linking:

/usr/bin/ld: /usr/local/lib/libmdk.so: invalid string offset 32435 >= 29045 for section `.strtab'
/usr/bin/ld: /usr/local/lib/libmdk.so: invalid string offset 32449 >= 29045 for section `.strtab'
/usr/bin/ld: /usr/local/lib/libmdk.so: invalid string offset 32462 >= 29045 for section `.strtab'
...

Do you have any clue what might be causing it?

Can we achieve in the current state of the library a zero-copy decoding on a SFML RenderTexture. If so, would you be able to provide an example?

seems apple M1 is required to decode gbrp h264/5

only 8bit works. 10bit can be decoded in Movies & TV. in mdk I got this error:

I 2020-06-17 22:27:14.234@39124: hr ERROR@535mdk::abi::MFTCodec::processOutput: (80080011) 请求的分配值太大

Took an example from wang-bin/mdk-examples#9 and changed a player source to "rtsp://192.168.1.57:8080/h264_ulaw.sdp" to connect to virtual local ip camera - the android smartphone with running IP Webcam app. It works if sound is turned on in IP Webcam app, but not shown anything if video is streamed without sound.
Vlc plays in both cases.

Windows 10
Qt 5.9.5, MinGW 5.3.0 32bit
MDK 0.11.0 (git 6dd01c9)

There are not signals to connect to in the Player class. there are onMediaStatusChanged and onStateChanged but they are not displayed as signals. connecting to them give error suggesting that
Q_OBJECT is not in the Player class. ("qobject.h:249: error: C2338: No Q_OBJECT in the class with the signal"

这个是由于缓冲原因还是什么原因 还是缺少了哪些参数设置 文档里没找到 新手问问题可能有点蠢 请见谅....

Thank you for your effort
Q?
How to static linkage with Qt on windows ?
Thanks

Was there a reason to bump the minimum supported macOS version to 11+?
MDK sdk v0.13.0 crashes on macOS 10.x with the following message:

  Symbol not found: _VTRegisterSupplementalVideoDecoderIfAvailable
  Referenced from: /Frameworks/mdk.framework/Versions/A/mdk (which was built for Mac OS X 11.6)

MDK sdk v0.12.0 works

ts streams only. need to wait key frame to avoid crash. but may get error kVTVideoDecoderBadDataErr. vlc works, mpv doesn't

Hi, I've stumbled across some weird issue recently.
I have one instance of the Player and I change the url from the list with the following code:

	player.setNextMedia(nullptr, -1);
	player.setState(State::Stopped);
	player.waitFor(State::Stopped);
	player.setMedia(nullptr);
	player.setMedia(file_name);
	player.prepare(0);
	player.setLoop(999);
	player.setBackgroundColor(0, 0, 0, -1);
	player.waitFor(State::Paused);
	auto& codec = player.mediaInfo().video[0].codec;
	player.setVideoSurfaceSize(codec.width, codec.height);
	texture.create(codec.width, codec.height);
	sprite.setTexture(texture.getTexture(), true);
	player.setState(State::Playing);
	// player.setPlaybackRate(1.0);

The first video runs fine, but all the consecutive files run slower ( or faster, depending on the fps of the first video), even the sound pitch reflects that. My videos have all different resolutions and frame rates ie 29.97, 30.00, 59.94, 60.00

Is the order of commands correct, or am I doing something wrong here?

下载qmdkplay示例学习,默认参数播放组播流rtp:// 持续卡顿.
后直接在本机利用vlc搭建rtp流服务,现象依旧
尝试setBufferRange(0, INT64_MAX,true),现象依旧
尝试setBufferRange(0, INT64_MAX,false),可以正常播放10-20秒左右,然后彻底丢失声音并开始花屏卡顿
当drop=true,minMs>0,仅一帧就完全卡住无法继续播放
换不同片源h264,hevc均无明显改善,甚至4k hevc 系统会跳一个驱动程序超时的弹窗.
环境 qt msvc 6.2.2 , win11,mdk-sdk 0.13.0, decoders:MFT:d3d=11,D3D11,DXVA,FFmpeg
同样环境ffplay和potplayer播放完全正常
后又尝试mdk-sdk提供的glfwplay.exe,异常现象完全相同,因此怀疑是mdk-sdk本身的问题
我是音视频新手,也不知猜的对不对🎄
如果不是mdk问题而是rtp流需要额外代码配置,还请作者指教一下😽

Hi and thank you for developing this awesome tool.

I've tried to check the performance of this library on playing network video stream and the spec are based on below:

1. mdk-sdk Version 0.14.0
2. video streamer: IP webcam over android on local wifi network
3. simplePlayer example based on Qt widget 5.12.11
4. Visual Studio 2019 (VS2017 C++ compiler)
5. trying all video decoder provided and supported in OS ( Win 10 64 bit)

Although mdk is really easy to use and out of headache tool but unfortunately from performance point of view it was not efficient enough at least comparing to the VLC player.
VLC was playing videos about 2 seconds faster than simplePlayer with fewer Lags and smoother play.
Then is there any performance option I had ignored or I can do to increase performance?

The repo is empty. Would you please explain why?

ubuntu-18.04.5-desktop-amd64
QT 5.13.2
mdk version :0.10.4

I'm not sure if I understand this correctly. When I set multiple decoders for example

player.setVideoDecoders({"MMAL", "FFmpeg"});

does it mean that when the MMAL decoder is unable to decode one of the videos it should fall back to FFmpeg, or do I need to decide what decoder to use based on info in the file descriptor?
I'm asking because I'm getting a blank screen when playing h264 with yuv444 color space which is not supported by RPI4 hardware decoder. It only accepts yuv420.

Sorry for spamming the issues repeatedly, but I'm trying to make sure there are no unforeseen issues popping up once I implement your player into our app.

I've noticed a memory leak while running the test app I posted in my previous thread
https://github.com/oomek/mdktest

This happens on Windows 10 and Raspberry PI4

4k分辨率av1编码的文件, 播放画面明显一顿一顿,不流畅.
无论mft:d3d=0还是mft:d3d=11 均是如此
1080p av1 是可以流畅播放的
MDK 0.13.0 (git bea9310)
相同环境用ffplay和potplayer均很流畅,尽管是用的不支持硬解av1的显卡
log如下
log-hard.txt
log-soft.txt

windows 10 qt 6.2.2 在切换 opengl渲染时 黑色！
QQuickView::setGraphicsApi(QSGRendererInterface::OpenGL);

Hi, I see that your lib is throwing a little bit of text to the console on start:

MDK 0.10.0 (git 7a075dab) - Multimedia Development Kit. Copyright (c) 2016-2020 WangBin(QtAV author) <wbsecg1 at gmail.com>
Build for: Raspberry Pi ARMv6; Clang8.0.1 (branches/release_80)GCC4.2.1; libc++9000; 02:01:20 Sep 29 2020

I understand that it's a way of advertising yourself, so you would not be keen on adding an option to remove it, but how about making it a little less obtrusive, getting rid of compiler specific info ( set with env var maybe ) and keeping the limit of 80 chars? Would you agree to do it by any chance?

wrong color

I can successfully play a 32bit MOV file (alpha channel) using QtAV. But using MDK-SDK I have problems, the system simply shuts down. Can you try to find out the problem? It is very important for my project to be able to play videos with alpha channel. I am working in Windows 10 64bits and using Visual Studio 2019 IDE.
teste.zip

NVDEC is fine. Maybe not our bug.

I tried your lib on PI4 with sfml-pi under DRM/KMS but I'm struggling with getting a video output, only audio is playing unfortunately.
The console output says:

Build for: Raspberry Pi ARMv6; Clang8.0.1 (branches/release_80)GCC4.2.1; libc++9000; 02:01:20 Sep 29 2020
get Context 0x6467d8/0x547020 has no local data for key 0x5d8058, create 1(0x62ed38)
get Context 0x6467d8/0x547020 has no local data for key 0x62f1e8, create 1(0x62ee08)

MDK_LOG:
mdk_log.txt

How to produce:

glfwplay -metal -c:v VT:format=yuv420p test.mp4

or

glfwplay -metal -c:v FFmpeg:pool=CVPixelBuffer test.mp4

请问目前支持播放7.1声道视频吗？
文档看了没看到，水平低，给您添麻烦了。

For timeline preview, it is useful to have an accurate seek to any frame, in both forward and backward direction. Currently MDK seems to support only forward frame seek, but looks like it's just playing the media until the desired frame is reached.

The way it works in other players is to seek to the nearest keyframe (last keyframe before the desired target), then seek forward frame by frame (but I don't see a reason to render frames while seeking, it should happen internally).

D pushclient: mdk log opening video decoder: AMediaCodec:java=1:copy=0:surface=1:async=1:cache_output=16...
D pushclient: mdk log CodecForMimeType: video/avc, profile: 2, level: -1
D pushclient: mdk log JMI ERROR: java vm is null
D pushclient: mdk log JMI ERROR: java vm is null
D pushclient: mdk log ERROR! failed to setup decoder: video

这个是打印信息，请问java vm is null是什么意思

我的是qt5.12 ndk是r21

怎样去掉二维码？

Please build the mdk libraries with openssl

currently almost all logs are Debug level

Hi,
we are experiencing some playback issues when trying to use the mdk qml player with capture devices as a source.
We tried with streams from pc cameras and from external video grabbers and, in both cases, the playback seems very laggy.
Using the onFrame callback we were able to verify that the frames are received in an non uniform way, with a short period of well distanced frames, followed by playback halts, happening almost once per second.
Is this a known issue, or there is a way improve the playback quality by adusting some of the parameters of the internal mdk player?

Qt正在逐步移除对OpenGL的直接调用，改用RHI这个抽象层。去年发布的Qt 5.14已经添加了RHI的实验性支持，Qt6会全面迁移到RHI。我看了examples里MDK与Qt的对接，目前是直接调用OpenGL的API。未来MDK有支持Qt RHI的计划吗？

After a recent update of HEVC Video Extensions from Microsoft Store. the MFT decoder started crashing when trying to play any HEVC file.

Exception thrown: read access violation.
sample->ptr_ was nullptr.

Call stack:

mdk.dll!mdk::abi::MF::to(mdk::abi::VideoFrame & frame, Microsoft::WRL::ComPtr<IMFSample> sample, int stride_x, int stride_y, int copy) Line 365
	at D:\a\mdk-sdk\mdk-sdk\mdk\video\MFVideoGlue.cpp(365)
mdk.dll!mdk::abi::MFTVideoDecoder::onOutput(Microsoft::WRL::ComPtr<IMFSample> sample) Line 408
	at D:\a\mdk-sdk\mdk-sdk\mdk\video\codec\MFTVideoDecoder.cpp(408)
mdk.dll!mdk::abi::MFTCodec::processOutput() Line 513
	at D:\a\mdk-sdk\mdk-sdk\mdk\base\ms\MFTCodec.cpp(513)
mdk.dll!mdk::abi::MFTCodec::decodePacket(const mdk::abi::Packet & pkt) Line 404
	at D:\a\mdk-sdk\mdk-sdk\mdk\base\ms\MFTCodec.cpp(404)
mdk.dll!mdk::abi::FrameReaderImpl::decodeLoop<0,BlockingFIFO<mdk::abi::TrackFrame<mdk::abi::VideoFrame>,std::list>,std::function<bool __cdecl(mdk::abi::VideoFrame const &,int)>>(int stream, int decoderIndex, BlockingFIFO<mdk::abi::TrackFrame<mdk::abi::VideoFrame>,std::list> & frames, const std::function<bool __cdecl(mdk::abi::VideoFrame const &,int)> & get_cb) Line 1782
	at D:\a\mdk-sdk\mdk-sdk\mdk\core\FrameReader.cpp(1782)
[Inline Frame] mdk.dll!mdk::abi::FrameReaderImpl::videoDecodeLoop(int) Line 741
	at D:\a\mdk-sdk\mdk-sdk\mdk\core\FrameReader.cpp(741)
mdk.dll!mdk::abi::FrameReaderImpl::startDecodeThread::__l2::<lambda>() Line 2012
	at D:\a\mdk-sdk\mdk-sdk\mdk\core\FrameReader.cpp(2012)

Decoder used: MFT:d3d=11

HEVC Video Extensions version: 1.0.42701.0

I confirmed this on two separate systems, one before update, and one after update.
Tested using provided glfwplay example

MDK版本

MDK 0.12.0 (git 013f88e) - Multimedia Development Kit. Copyright (c) 2016-2021 WangBin(QtAV author)
Build for: Windows0x0600 x86_64; MSVC1929; MSVCRT14.29.30038.1/msvcp140.dll; 01:28:25 Jun 28 2021

运行环境：

win10+qt 5.15.2 + msvc2019

这是崩溃时的dump信息

我在UI主线程通过按钮不断的调用以下片段代码，用于切换不同的视频文件播放，以此来实现上一部、下一部功能

在回调函数中通过发送信号方式获取播放器的播放状态，当播放状态 state == 1 时，调用player_->position()方法获取视频总时长，此时出现崩溃问题；

correct image if copy to host.

crash in atemp filter, af_atempo.c line 535
av_assert0(zeros != nsamples);

I've used QtAV in an open source project and used the PPA to install the required libraries and even donated to keep it going but unfortunately it looks like development has been abandoned :(

I believe the MDK is it's replacement but I don't see any mention of a PPA for it. Since the MDK is not open source I'm guessing you can't create a PPA for it? Are there any open source projects using the MDK? How do/would they use it in an open source project and still allow there project to be packaged for use in Ubuntu?

MediaFormat: mime: string(video/raw), stride: int32(3840), slice-height: int32(2160), color-format: int32(21), image-data: data, crop: Rect(0, 0, 3839, 2159), hdr-static-info: data, width: int32(3840), height: int32(2160), color-standard: int32(6), color-transfer: int32(3)}

it's a bug in mediacodec. but we can workaround in our code