ashwhall / vidalign Goto Github PK

A tool to perform multi-camera video temporal alignment and clip extraction.

Install Anaconda here or Miniconda here, then run the below command:

conda env create -f environment.yml;

There are likely some system dependencies that need to be installed prior to this setup procedure. Raise an issue if you encounter any such problems, or create a pull request if you know how to fix them.

conda activate vidalign;
python -m vidalign;

1. Video Player: Basic video playback and frame seeking, with support for panning and zooming using the mouse. Take note of the keyboard shortcuts by hovering on the playback buttons, as they'll be your best friend. There are also some cropping tools available, see cropping below.
2. Video Dropper: Drop video files and folders here to import them. Directories will be descended recursively looking for video files. At the time of writing the supported extensions are: {mp4, avi, mkv, mov, mxf}. You can optionally filter the video filenames using the text box, with an asterisk (*) acting as a wildcard (e.g. left* will match any filename starting with "left").
3. Video Info: Information about the currently selected video, with buttons to set sync frames and aliases. See workflow below for more detail.
4. Video List: List of imported videos. If there are no empty cells here, you've done what's needed for the videos. Ensure you don't set the same alias for multiple videos!
5. Clip Info: Information about the currently selected clip, with buttons to set the start and end frames, and other useful things. See workflow below for more detail.
6. Clip List: List of clips. If there are no empty cells here, you've done what's needed for the clips. Ensure you don't set the same clip name for multiple clips!
7. Video/Clip File Utilities: Set the output directory for your exported clips, and save/load/reset the videos and clips. Handy if you need to save your progress and return to it later, or just for reproducibility.
8. Encoder Settings: Here you can choose your encoder implementation (either FFmpeg or PyAV), configure some of the encoding parameters, and save/load/reset them. If you figure out parameters that you like, export them and load them at the start of each session.

1. Load some videos by dragging them onto the video dropper.
   1. You can optionally filter the video filenames using the text box.
2. For each video:
   1. Select it and click Set alias. Give it a recognisable name like "left_camera".
   2. Scrub through the video and find a frame with a clearly visible event which can be used for time-synchronisation across cameras. Click Set sync frame. Typically you would use the flash of a light or a clap.
3. For each desired clip:
   1. Click New clip. The clip's start frame will be automatically set at the current frame, but this can be changed by navigating to the desired frame and clicking Set; same goes for the end frame.
   2. Rename the clip to something recognisable like "calibration" or "run_1".
4. Set your output directory with the button on the right. The videos will be output here with filenames like {output_dir}/{clip_name}_{video_alias}.{ext}
   • Save the videos/clips configuration to file if you want to be safe.
5. Open up the encoder options and check that you're happy with them.
   • Instead you can load a previous encoder configuration from file if you have one prepared.
   • I like using PyAV for encoding, but FFmpeg is also a good choice as it's more tried and tested.
   • Note that the PyAV implementation currently has a memory leak, so you may need to restart the app after encoding many clips.
6. View the encoder commands to check that everything is as expected.
   • You can copy-paste them elsewhere if you want to make any changes that the app can't facilitate.
   • This is only available for FFmpeg at the moment.
7. Run the encode commands by clicking Run encode commands.
8. Have a coffee and come back when it's done!

This tool also supports spatially cropping a video over time. This is visually editable by interacting with the video player.

• Move the bounding box by holding , clicking and dragging it.
• Resize the bounding box horizontally by holding and scrolling the mouse wheel.
• Resize the bounding box vertically by holding + and scrolling the mouse wheel.

A red bounding box indicates that a bounding box has been set on the current frame. A yellow bounding box indicates that the current bounding box location is being interpolated between the previous and next annotated frames.

Right-clicking on the video player will clear the bounding box for the current frame.

Please feel free to open an issue or pull request if you have any suggestions or find any bugs.