kokorin / jaffree Goto Github PK

com.github.kokorin.jaffree.ffprobe.Stream#getCodecType

ffmpeg -ss 0 -t 1 -i short.wav -y -f null -c copy NUL
gives the following output:

size=N/A time=00:00:00.00 bitrate=N/A speed=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)

-c:v copy or removing -t 1 fix output

Frame reuse (as well as Image or samples) can lead to errors while producing video.

Immutable will make it more clear.

It must be BaseOutput<T extends BaseOutput & Output>

Add Process.isAlive() checks to wating loop

Some properties (like com.github.kokorin.jaffree.ffprobe.Stream#avgFrameRate) have type Stream, but they represent Rational (like 10000000/33333).

Note that Rational can extend Number.

I try to trim videos and concat them. Doesn't put quatation marks and null=concat instead of only using concat. And [0:v]trim=start=0.0:end=2.0:setpts:PTS-STARTPTS[0]; after could be setpts=PTS-STARTPTS but couldn't. I want to create the following command like that
./ffmpeg -i ../Desktop/ramb.mp4 -filter_complex "[0:v]trim=start=0:end=5,setpts=PTS-STARTPTS[0]; [0:v]trim=start=10 : end=15,setpts=PTS-STARTPTS[1]; [0][1]concat[2]; [0:v]trim=start=25,setpts=PTS-STARTPTS[3]; [2][3]concat [out1]" -map [out1] ../Desktop/cut_ramb2.mp4

but command constructed like this:
ffmpeg -i /var/www/contents/rawVideo/30_Rambo.mp4 -y -filter_complex [0:v]trim=start=0.0:end=2.0:setpts:PTS-STARTPTS[0]; [0:v]trim=start=5.0:end=15.0:setpts:PTS-STARTPTS[1]; [0][1]null=concat[2]; [0:v]trim=start=40.0:end=45.0:setpts:PTS-STARTPTS[3]; [2][3]null=concat[4] -map [4] /var/www/contents/trimmedVideo/trimmed_30_Rambo.mp4

and I am using Jaffre-wrapper in Java Code like this:

FilterChain filterChain = new FilterChain();

            filterChain.addFilter(
                    Filter.fromInputLink(StreamSpecifier.withInputIndexAndType(0,StreamType.VIDEO)).
                            setName("trim").
                            addArgument("start",start.toString()).
                            addArgument("end",end.toString()).
                            addArgument("setpts").
                            addArgument("PTS-STARTPTS").
                            addOutputLink(String.valueOf(outIndex++))
            );
            filterChainList.add(filterChain);

            if(i != 0){
                FilterChain filterChain1 = new FilterChain();
                filterChain1.addFilter(
                        Filter.fromInputLink(String.valueOf(outIndex-2)).
                                addInputLink(String.valueOf(outIndex-1)).
                                addArgument("concat").
                                addOutputLink(String.valueOf(outIndex++))
                );
                filterChainList.add(filterChain1);
            }`

return FilterGraph.of().addFilterChains(filterChainList);

FilterGraph filterGraph = createFilterChain();

    FFmpegResult result = FFmpeg.atPath(ffmpegBin)
            .addInput(UrlInput.fromPath(input))
            .setOverwriteOutput(true)
            .setFilter(filterGraph.getValue())
            .addArgument("-map")
            .addArgument("[" + out + "]")
            .addOutput(UrlOutput.toPath(output))
            .execute();`

Thanks for this project. This project helped me a lot.

Save instance of main created Process as FFMpeg class field, than provide a way to developer call destroy/destroyForcibily.

Related with: #20

Is there a way to use hardware acceleration in your code?

Regards

It seems that despite an exception is thrown if stream IDs do not start with zero, this exception isn't rethrown correctly. Which leads to

INFO writer-stderr com.github.kokorin.jaffree.ffmpeg.FFmpegResultReader - [nut @ 00000000011924a0] No main startcode found. INFO writer-stderr com.github.kokorin.jaffree.ffmpeg.FFmpegResultReader - tcp://127.0.0.1:54679: Invalid data found when processing input DEBUG writer-stdout reader com.github.kokorin.jaffree.process.ProcessHandler - StdOut thread has finished WARN writer-stderr com.github.kokorin.jaffree.process.ProcessHandler - Failed to process stderr

FFmpegResultReader does not parse (and subsequently does not consume line) progress from ffmpeg output if ProgressListener isn't set.

It means that errorMessage will be set and an Exception will be thrown

com.github.kokorin.jaffree.ffprobe.Stream.getTag() is empty for video stream having metadata.
ex:

Stream #0:0(und): Video: hevc (hvc1 / 0x31637668), yuv420p(tv, bt709), 1920x1080, 7709 kb/s, 29.97 fps, 29.97 tbr, 600 tbn, 600 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2019-03-18T07:54:08.000000Z
      handler_name    : Core Media Data Handler
      encoder         : HEVC
    Side data:
      displaymatrix: rotation of -90.00 degrees

getTag() had "rotate" metadata in tag list before (version 0.8.0).

Now we must rely only on getSideDataList() -> x.getRotation() for video rotation detection (then real width / height after ffmpeg conversion).

Not sure this is universal.

Hi @kokorin,

I'd love to have support in Jaffree for piping raw video in and out of my own custom process, specifically to be able to do something similar to this shell command:

ffmpeg -i input.mp4 -f rawvideo -pix_fmt yuv444p pipe: | ./custom-video-processor | ffmpeg -f rawvideo -r 29.97 -s 320x240 -pix_fmt yuv444p -i pipe: -f mov output.mp4

I've made my own hacky implementation by copying your FFmpeg class as a proof of concept. It gets the job done, but it would great to either see a proper implementation that allows the above or at least it would be nice if you could also invoke setStdInWriter as part of creating ProcessHandler in execute(), which would allow people to subclass FFmpeg easily for a use case like this.

If i try to delete the input file provided to the FFmpegResult instance after its execution will result java.nio.file.FileSystemException: The process cannot access the file because it is being used by another process.
How can we gracefully release the resources programmatically once the job is done ?

video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

Is there a way to not append -n to the command that's generated. I actually want to append -y
.

Java codfe

FFmpeg.atPath(FFMPEG_BIN)
        .addInput(UrlInput.fromPath(VIDEO_MP4_IN))
        .addOutput(UrlOutput.toPath(VIDEO_MP4_OUT)
            .addArgument("-an").addArguments("-vf", "scale=320:240,format=gray")
            .addArguments("-pix_fmt", "yuv420p").addArguments("-c:v", "h264")
//            .addArgument("-y")
        )

Resulting ffmpeg command

/usr/local/Cellar/ffmpeg/4.0/bin/ffmpeg -loglevel 32 -i src/test/resources/TruePositive3.mp4 -n -an -vf scale=320:240,format=gray -pix_fmt yuv420p -c:v h264 src/test/resources/TruePositive3_reduced.mov

BTW, most if not all of the sample code snippets in the README have compile errors, and seem to be using an outdated API.

My use case is transcoding an hour long audio file (wav to aac) using PipeOutput.

FFmpeg.atPath(Paths.get(ffmpegBinDirectory)) .addInput(PipeInput.pumpFrom(input)) .addOutput(PipeOutput.pumpTo(output).setFormat("mp4")) .addArguments("-movflags", "frag_keyframe+faststart") .addArguments("-c:a", "aac") .execute();

The file is totally unplayable when using PipeOutput, however if I use UrlOutput then the file is playable.

In issue #44 it appeared that an exception thrown inside of ProgressHandler wasn't logged with WARN level.
Reproduce and improve.

Currently it's only possible to stop ffmpeg by throwing an exception in ProgressListener.

Code generated from ffprobe.xsd lacks TimeUnit aware getters. Check JAXB delegate plugin.

https://github.com/mklemm/jaxb-delegate-plugin

I came across a file which seems to have tags that contain values that span multiple lines. This seems to cause the DataParser to error:

Caused by: java.lang.RuntimeException: key=value was expected but got: 
	at com.github.kokorin.jaffree.ffprobe.data.DataParser.parseLine(DataParser.java:54)
	at com.github.kokorin.jaffree.ffprobe.data.DataParser.parse(DataParser.java:151)
	at com.github.kokorin.jaffree.ffprobe.FFprobeResultReader.read(FFprobeResultReader.java:40)
	at com.github.kokorin.jaffree.ffprobe.FFprobeResultReader.read(FFprobeResultReader.java:32)
	at com.github.kokorin.jaffree.process.ProcessHandler$3.run(ProcessHandler.java:186)
	at com.github.kokorin.jaffree.process.Executor$1.run(Executor.java:49)

with showFormat enabled:

ffmpeg.setShowFormat(true)

running ffprobe with options ffprobe -loglevel 16 -show_format -show_private_data shows this output (long text shortened with ...):

[FORMAT]
filename=/tmp/cvma7ou0kk5infmok4403378daciqkia/cvma7ou0kk5infmok4403378daciqkia.mp3
nb_streams=2
nb_programs=0
format_name=mp3
format_long_name=MP2/3 (MPEG audio layer 2/3)
start_time=0.011995
duration=122.044082
size=2029181
bit_rate=133012
probe_score=51
TAG:comment=The most popular...way.

Samples...provide.

Then,...format
TAG:encoder=Lavf56.40.101
[/FORMAT]

I was thinking about modifying the dataParser so that it if it does not find a new tag, it will add the content to the previous value. However, that probably will blow up as easy as the multiline value containing a =. Maybe a better approach would be to only parse values we know are relevant for ffprobes result and ignore everything else? Or we just ignore multiline tags, by simply returning instead of throwing an error? what do you think?

The file is available here https://www.dropbox.com/s/wsczg7umwfx0isc/multiline.mp3?dl=0

I was also having some issues running the tests locally, mostly because I was unable to find the input files required for testing. Are they available for download somewhere?

java.lang.NumberFormatException: For input string: "N/A" at com.github.kokorin.jaffree.Rational.valueOf(Rational.java:165) at com.github.kokorin.jaffree.ffprobe.data.DBase$RationalConverter.convert(DBase.java:158) at com.github.kokorin.jaffree.ffprobe.data.DBase$RationalConverter.convert(DBase.java:140) at com.github.kokorin.jaffree.ffprobe.data.DBase.getValue(DBase.java:44) at com.github.kokorin.jaffree.ffprobe.data.DBase.getRatio(DBase.java:82)

if (value != null && !value.isEmpty() && !value.equals("0/0")) may add the "N/A" judge

Currently it's not possible to set position to 2.5 seconds:
urlInput.setPosition(2.5, TimeUnit.SECONDS)
The above method only accepts long values.

NUT can speed up programmatic video creation and consumption, due to NUT can handle RGB24 pixel format, not YUV420<->RGB24 conversion in Java is needed.

Caused by: java.lang.RuntimeException: Process has ended with no result and non zero status: 1
	at com.github.kokorin.jaffree.process.ProcessHandler.execute(ProcessHandler.java:220)
	at com.github.kokorin.jaffree.ffmpeg.FFmpeg.execute(FFmpeg.java:143)
	at commands.Commands.webToGifConv(Commands.java:61)
	at commands.Commands.gifInputStream(Commands.java:88)
	at commands.Commands.onGifCommand(Commands.java:129)
	... 11 common frames omitted

with the setttings of

Path BIN = Paths.get("C:/ffmpeg/bin/");
		Path INPUT = Paths.get(path+"temp.webp");
		Path OUTPUT = Paths.get(path+"temp.gif");
		FFmpeg result = FFmpeg.atPath(BIN)
				.addInput(UrlInput.fromPath(INPUT))
				.addOutput(UrlOutput.toPath(OUTPUT)
						.setCodec(StreamType.VIDEO, "libx264"));
		result.execute();

I'm probably doing something incorrectly, which is pretty sstraightfoward, butt if anyonee could help me i'd be nice.

com.github.kokorin.jaffree.ffmpeg.BaseOutput#setOutput

Hi @kokorin, tks for this amazing library!
I'm using it to create a dynamic generator video/sound streamer:

public FFmpegResult startAndWait() throws IOException {
        FFmpeg ffmpeg = FFmpeg.atPath(Paths.get(System.getProperty("natives.ffmpeg.path"))) //
                .addInput(FrameInput.withProducer(new VideoProducer()).setFrameRate(framerate).setReadAtFrameRate(true)) //
                .addInput(FrameInput.withProducer(new AudioProducer()).setFrameRate(framerate).setReadAtFrameRate(true)) //
                .addOutput(//
                        UrlOutput.toUrl("udp://127.0.0.1:14667?pkt_size=1316")//
                                .setPixelFormat("yuv420p")//
                                .setFrameRate(framerate)//
                                .addArguments("-preset", "ultrafast")//
                                .addArguments("-tune", "zerolatency")//
                                .addArguments("-b", "900k")
                                .addArguments("-g", String.valueOf(2 * framerate))//
                                .setCodec(StreamType.VIDEO, OSDetector.isMac() ? "h264_videotoolbox" : "libx264")//
                                .setFormat("mpegts")//
                );

        return ffmpeg.execute();
    }

This code is working fine, I can consume it on VLC with address udp://@:14667 without drop frames and with good quality of video and sound :), thank you again \o/ \o/ \o/.

But I can't make a "dynamic" input work, let me explain my idea:

If I include a new UDP input, like:

addInput(UrlInput.fromUrl("udp://localhost:14666"))

And after that I create another instance of ffmpeg with output to this address (external source):

FFmpeg ffmpeg = FFmpeg.atPath(Paths.get(System.getProperty("natives.ffmpeg.path"))) //
        .addInput(UrlInput.fromPath(Paths.get("./sound.mp3")).setReadAtFrameRate(true)) //
        .addOutput(//
                UrlOutput.toUrl("udp://127.0.0.1:14666?pkt_size=1316")//
                        .addArguments("-tune", "zerolatency")//
                        .setFrameRate(25)
                        .setFormat("mpegts")//
        );

ffmpeg.execute();

The resulting of first ffmpeg drops a lot of frames, delay some seconds to show first frame and play only some parts of external sound (sound.mp3).

My question: Can ffmpeg "ignore" no data on an input? Or... Can I tell to ffmpeg to don't use this input if no data (bytes received) and only use it if data is present? (better if input fps is ignored too).

I am trying to play 64 video feeds simultaneously, but the performance degrades as the number of streams played increases. Is there an optimization method for the same?

It's required to profile next cases:

1. Simple converting video from one format to another
2. Programmatic video production with Jaffree
3. Programmatic video consumption with Jaffree

https://github.com/takari/maven-wrapper

It must distinct Operation System ( /dev/null in Linux and Nul in Windows).
Also it must allow to set video and audio codecs.

input is java.io.file.Path, how to input rtmp stream to check

com.github.kokorin.jaffree.ffmpeg.Stream

Hi @kokorin,

I've noticed that DBase logs the error below when I'm attempting to get a display aspect ratio which is not set for a particular video file. Would it make sense to make an explicit check for "N/A" and just return null before attempting to parse?

11:22:56.682 [main] WARN com.github.kokorin.jaffree.ffprobe.data.DBase - Failed to parse rational number: N/A
java.lang.NumberFormatException: For input string: "N/A"
	at com.github.kokorin.jaffree.Rational.valueOf(Rational.java:165)
	at com.github.kokorin.jaffree.ffprobe.data.DBase$RationalConverter.convert(DBase.java:158)
	at com.github.kokorin.jaffree.ffprobe.data.DBase$RationalConverter.convert(DBase.java:140)
	at com.github.kokorin.jaffree.ffprobe.data.DBase.getValue(DBase.java:44)
	at com.github.kokorin.jaffree.ffprobe.data.DBase.getRatio(DBase.java:82)
	at com.github.kokorin.jaffree.ffprobe.Stream.getDisplayAspectRatio(Stream.java:121)

I'm doing a stream.getSection().getString("display_aspect_ratio").equalsIgnoreCase("N/A") check in my own code for now, but IMO it would be nice if Jaffree handled this.

I found Maven version 0.3 is dated Oct. Is current change significant to update to a newer version?

Thanks!

Check how programmatic consumption works with -vframes option

2019-03-28 07:48:07.564 WARN [FFmpeg-async-runner][] c.g.k.j.p.ProcessHandler - Waiting for process has been interrupted
java.lang.InterruptedException: null
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:502)
	at java.lang.UNIXProcess.waitFor(UNIXProcess.java:395)
	at com.github.kokorin.jaffree.process.ProcessHandler.interactWithProcess(ProcessHandler.java:112)
	at com.github.kokorin.jaffree.process.ProcessHandler.execute(ProcessHandler.java:85)
	at com.github.kokorin.jaffree.ffmpeg.FFmpeg.execute(FFmpeg.java:161)
	at com.github.kokorin.jaffree.ffmpeg.FFmpeg$1.call(FFmpeg.java:176)
	at com.github.kokorin.jaffree.ffmpeg.FFmpeg$1.call(FFmpeg.java:173)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:745)
2019-03-28 07:48:07.564 WARN [FFmpeg-async-runner][] c.g.k.j.p.Executor - Interrupting ALIVE thread: StdOut

Probably problem parsing ffmpeg output being (for audio only):
size= 11986kB time=00:24:55.80 bitrate= 65.6kbits/s speed=55.8x

How to interrupt main running thread to stop instance .
it's returning FFmpegResult object, how to access ProcessHandler class method.
I want to close video programatically rather than closing the window.

https://developers.italia.it/en/reuse/publication

Today I found that it's possible to have the following in playable MP4 video
avg_frame_rate=0/0
Last Jaffree version throws exception

Some iPhone video are rotated 90°. FFProbe output is as follow:

<stream>
<side_data_list>
                <side_data side_data_type="Display Matrix" displaymatrix="
00000000:            0       65536           0
00000001:       -65536           0           0
00000002:     70778880           0  1073741824
" rotation="-90"/>
</side_data_list>
</stream>

Jaffree can't get this rotation info due to current Jaffree XML schema definition.

FFMpeg is using this attribute when transcoding and inverts height and width automatically. If we force scale filter, it is not auto inverting height and width (as expected) but still rotates the frames. We should know this rotation parameter to specify the correct scale dimensions.

Is it possible to add Javadoc? Some methods are difficult to understand what they do.
Thanks.

PS: This is nice FFMPEG wrapper.

First off, thanks for creating this project. I came here from ffmpeg-cli-wrapper and finally found most of the flexibility I need when using FFmpeg. However, I came along one thing I am missing:

The ability to retrieve and parse FFmpeg command line output (usually errout). For instance when using the loudnorm filter, which essentially is a two pass job that requires to retrieve and parse the parameters from the first run and use them for the second run.

I was thinking about adding a capability to add a custom OutputParser to FFmpegResultReader:

public FFmpegResult read(InputStream stdOut) {
        //just read stdOut fully
        BufferedReader reader = new BufferedReader(new InputStreamReader(stdOut));
        String errorMessage = null;

        String line;
        FFmpegResult result = null;

        try {
            while ((line = reader.readLine()) != null) {

                if (outputParser != null) {
                    outputParser.parse(line);
                }
...

OutputParser is just an interface:

public interface OutputParser {
    public void parse(String line);
}

in the end this will be invoked like:

LoudnormParser loudnormParser = new LoudnormParser()

FFmpegResult result = FFmpeg.atPath()
                .setOutputParser(loudnormParser)
                .addInput(UrlInput.fromPath(Paths.get("/Users/steve/Desktop/3-part Harm 1b_keyCmin_56bpm.wav")))
                .addArguments("-af", "loudnorm=I=-16:TP=-1.5:LRA=11:print_format=json")
                .addOutput(new NullOutput())
                .execute();

loudnormParser.getInputThreshold();

where LoundnormParser is an implementation of OutputParser.
When the command has been executed, one can obtain parsed values from the OutputParser implementation.

I will be happy to submit a PR if this sounds like a reasonable thing to add to you. And please let me know in case you come along better names 😸

This will allow

1. Remove JAXB dependency
2. Remove ffprobe classes generation
3. Parsing of non-standard or format-specific properties.