To make it easier to manage resources and usage in AWS we would like to be able to tag resources with customer info, i.e:

Customer=<CUSTOMER>

This also applies to the API. https://github.com/Eyevinn/autovmaf-api.

AWS JavaScript SDK (v2) will be put into maintenance mode in 2023.

The current algorithm for generating a bitrate ladder only considers bitrates that were actually measured on for inclusion in the bitrate ladder. It would make sense to tweak the algorithm to use linear interpolation between measurement points to allow it to select more optimal bitrates.

The readme needs to contain much more information. One should be able to use autoabr without going through the code base.

Create an encore instance using Eyevinn OSaaS platform, run the jobs and then tear the instance down

When exporting results to CSV, only the cpu time used by the first pass is included. The CSV should contain the sum of the cpu time for all passes.

There might be an issue related to hardcoded paths in Autovmaf see: https://github.com/Eyevinn/autovmaf/blob/e17167dff563b5d80e60eb8dcc26973d1ab52140/src/pipelines/local/local-pipeline.ts#L76C67-L76C76 and running the CLI/local jobs on Windows.

(node:448) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy
(Use `node --trace-warnings ...` to show where the warning was created)
info: Creating job MyVMAFMeasurements.
info: ffmpegOptions: ["-vf","scale=640:360","-c:v","libx264","-b:v","150000","-maxrate","150000","-bufsize","300000"]
autovmaf <source>

run transcode and vmaf for videofile source

Commands:
  autovmaf <source>                 run transcode and vmaf for videofile source
                                                                       [default]
  autovmaf suggest-ladder <folder>  Suggest bitrate ladder given vmaf results

Positionals:
  source  SOURCEFILE                                                    [string]

Options:
  --help            Show help                                          [boolean]
  --version         Show version number                                [boolean]
  --resolutions     List of resolutions, ie 1920x1080,1280x720...       [string]
  --bitrates        List of bitrates, ie 800k,1000k,...                 [string]
  --name            Name for this autovmaf run
                                        [string] [default: "MyVMAFMeasurements"]
  --models          List of VMAF Models to use          [string] [default: "HD"]
  --ffmpeg-options  List of options to pass to ffmpeg, on the form
                    key1=value1:key2=value2                             [string]

Error: ffmpeg exited with code 4294967294: Error opening output file /dev/null.
Error opening output files: No such file or directory

    at ChildProcess.<anonymous> (C:\Users\kalle\AppData\Roaming\nvm\v18.18.0\node_modules\@eyevinn\autovmaf\node_modules\fluent-ffmpeg\lib\processor.js:182:22)
    at ChildProcess.emit (node:events:517:28)
    at ChildProcess.emit (node:domain:489:12)
    at ChildProcess._handle.onexit (node:internal/child_process:292:12)
PS C:\Users\kalle\Documents\Works\Gigset\AutoVMAF>

Currently all bitrates will be encoded and measured for all resolutions. It would be nice to be able to set a bitrate range for a resolution (optional).

Proposal:

{
    name: "MyVMAFmeasurements",
    pipeline: "pipeline.yml",
    encodingProfile: "profile.json",
    reference: "reference.mp4",
    models: ["HD", "PhoneHD"],
    resolutions: [
    { 
       width: 1280, 
       height: 720,
       range: {
         min: 500000,
         max: 600000
       }    
    }],
    bitrates: [500000, 600000, 800000],
    method: "bruteForce"
}

When running locally, it would be useful if there would be more flexibility in configuring the ffmpeg command.

• Support 1-pass encoding
• Support for not adding -b:v, -maxrate, -bufsize to the ffmpeg commandline

Currently we always suggests a ladder when the brute force calculations have finished. This should be optional.
https://github.com/Eyevinn/autoabr/blob/bce73d12ef4fb1ec3a3388fe7639fe28a80f2d6f/src/analysis/brute-force.ts#L164-L173

As it is now the only way to know if a MediaConvert job has failed is to wait for waitUntilObjectExists to throw an error. We should instead with a specific interval check the status of a MediaConvert job.

https://github.com/Eyevinn/autoabr/blob/e4992725165869d5502e782470358bc25394dd94/src/pipelines/aws/aws-pipeline.ts#L124

easyvmaf_s3 should have its own repo.

OSAAS encore api profile parameter description
User can provide a url to a profiles file on instance creation. If no such url is provided a url to default profiles is used.

If this is the case, then the user is required to have the profile hosted on http? Right now the default profiles are on github. I assume similar hosting would be required if one was to use their own profile.yml. As autovmaf is designed today, the user expects to have the ability to affect the resolution and bitrate of the transcoding output through the autovmaf interface. How do we want to solve this?

Local profile.yml pushed somewhere where the OSAAS encore api can access it?

Deliverable
Should be able to start a job in exact same process flow as local and aws pipelines. This likely implies that we will need some way for the autovmaf job parameters to be pushed to the hosted encore profile. Current control flow is that a .yml profile is stored on github or other http host, thereby accessible by the encore OSAAS api?

https://svt.github.io/encore-doc/#profiles
As described by the SVT encore documentation
A Profile can be seen as a general abstraction of an FFmpeg configuration - example, bitrate to use, thumbnail generation, the codec to use.
A Profile specifies a big part of the configuration used by an Encore Job - metadata, FFmpeg configuration and specific codec configuration.
A Profile is specified in the yaml-format.

To make it easier to use AutoVMAF to measure on already encoded content it would be nice to be able to easily upload content to an S3 bucket and skip the MediaConvert step.

Currently there is no way of passing sync window and sync start time to easyvmaf. We should add this functionality.

Currently we do not check if an output json file for a specific resolution/bitrate already exists before starting an ECS task. There should be a check so that we do not run the same VMAF measurement task multiple times.

When transcoding locally with ffmpeg, it would be useful to measure cpu usage. Easiest solution is probably to run the ffmpeg command through gnu time.

There might be some S3 paths that are hard coded. These should be changed and taken from pipeline.yml.

Hi Devs,

Many Kudos for this excellent piece of software!

I'd like to create a Dockerfile for running autovmaf in a container, but currently it fails, hopefully you can help me?

Dockerfile:

FROM python:latest

COPY --from=mwader/static-ffmpeg:5.1.2 /ffmpeg /usr/local/bin/
COPY --from=mwader/static-ffmpeg:5.1.2 /ffprobe /usr/local/bin/

RUN apt update && sh -c 'curl -sL https://deb.nodesource.com/setup_19.x | bash -'
RUN apt install nodejs -y
RUN npm install -g @eyevinn/autovmaf

RUN git clone https://github.com/gdavila/easyVmaf.git
RUN pip3 install ffmpeg-progress-yield

ENV EASYVMAF_PATH=/easyVmaf/
ENV FFMPEG_PATH=/usr/local/bin/
ENV PYTHON_PATH=/usr/bin/

Building image:

docker buildx build --platform linux/amd64,linux/arm64 . -f Dockerfile -t r.gl-systemhaus.de/tools/autovmaf:latest --push

Running container fails:

docker run  -v /home/martini/:/tmp -it r.gl-systemhaus.de/tools/autovmaf:latest /bin/bash
autovmaf --resolutions 1920x1080,1280x720,960x540 --bitrates 500k,800k,1200k,1600k,2000k,3000k,4000k --name my-autovmaf-test1 /tmp/1080P50.mp4

Error: spawn /usr/local/bin/ EACCES
    at ChildProcess._handle.onexit (node:internal/child_process:285:19)
    at onErrorNT (node:internal/child_process:483:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -13,
  code: 'EACCES',
  syscall: 'spawn /usr/local/bin/',
  path: '/usr/local/bin/',
  spawnargs: [ '-formats' ]
}

=> I'm not in expert in node, so I don't know, how to fix this error Error: spawn /usr/local/bin/ EACCES

During early testing there was a problem with the encore api where jobs got stuck when queueing from multiple input sources.

Reproduction steps:
Add more than one source in the inputs[] parameter in the pipeline configuration and run createJob() like in the example.

Currently when running a vmaf analysis the the program is running locally. This however can cause issues if for example the internet connection is lost or the computer goes to sleep etc.

It would therefore be nice to have an option to run it as a lambda or something similar so it isn't dependant on a local setup.

Proposed new file structure:

bucket-name (or maybe S3 prefix)
├── encoded-files
│   └── job-name
│       └── HxW
│           ├── HxW_b.mp4
│           └── HxW_b2.mp4
├── reference-files
│   └── high-res-reference.mxf
└── results
    └── job-name
        └── model
            ├── HxW_b2_vmaf.json
            └── HxW_b_vmaf.json

I looked into how we are measuring CPU usage with our fluent-ffmpeg fork, and I can see some issues with it. For starters, time is a shell builtin in bash, zsh and fish, and then the binary is not needed, and if not installed the local pipeline will fail. It will also fail for Windows. Maybe it should degrade gracefully instead so we don't get any CPU measurements, but I think we still can get them with pidusage (which should work cross platform).

This only works if we have access to the pid. From what I have seen fluent-ffmpeg doesn't expose the pid or the process, but it's really not that hard to write a cli wrapper for ffmpeg. I have already done that previously (this doesn't help "build" the command for you unlike fluent-ffmpeg, making it much lighter and easier to maintain, it just helps you run the command and get the progress updates).

The pidusage package also recommends process.cpuUsage, which also works cross platform, but it's is not available on the child proccess, and I'm not sure if it's possible to use it with child processes). Perhaps you can if you fork the process, then spawn a command from there, but I'm not sure if it measures the child process cpu usage then?

The current maxWaitTime for ECS tasks and MediaConvert jobs are set to 3600 seconds. This should be increased and we should not allow the service to crash if a timeout happens. Instead a retry should occur so that the job doesn't need to be restarted.

We should add support to run all measurements locally. Most of the pieces are already in place.

A good README.md and some awesome 🌈 documentation. How hard could it be?

MkDocs is nice.

We should have a GH action to build and push an npm package.

There is a control api available here: https://autovmaf.docs.eyevinn.technology/getting-started/api.html

This should also be able to start encore pipeline jobs.