AlvaAR is a realtime visual SLAM algorithm running as WebAssembly, in the browser. It is a heavily modified version of the OV²SLAM and ORB-SLAM2 projects. SLAM is the core building block of Augmented Reality applications focusing on world tracking.
The examples use ThreeJS to apply and render the estimated camera pose to a 3d environment.
Video Demo: A desktop browser version using a video file as input.
Camera Demo: The mobile version will access the device camera as input.
To run the examples on your local machine, start a simple http server in the examples/ folder:
$: python 2: python -m SimpleHTTPServer 8080 or
$: python 3: python -m http.server 8080 or
$: emrun --browser chrome ./
Then open http://localhost:8080/public/video.html in your browser.
To run the examples on another device in your local network, they must be served via https. For convenience, a simple https server was added to this project – do not use for production.
$: cd ./AlvaAR/examples/
$: npm install
$: cd ./AlvaAR/examples
$: mkdir ssl/
$: cd ssl/
$: openssl req -nodes -new -x509 -keyout key.pem -out cert.pem
$: cd ./AlvaAR/examples/
$: nvm use 13.2
$: npm start
Then open https://YOUR_IP:443/video.html in your browser. If met with a ERR_CERT_INVALID error in Chrome, try typing badidea or thisisunsafe directly in Chrome on the same page. Don’t do this unless the site is one you trust or develop.
This code shows how to send image data to AlvaAR to compute the camera pose.
import { AlvaAR } from 'alva_ar.js';
const videoOrWebcam = /*...*/;
const width = videoOrWebcam.width;
const height = videoOrWebcam.height;
const canvas = document.getElementById( 'canvas' );
const ctx = canvas.getContext( '2d' );
canvas.width = width;
canvas.height = height;
const alva = await AlvaAR.Initialize( width, height );
function loop()
{
ctx.clearRect( 0, 0, width, height );
ctx.drawImage( videoOrWebcam, 0, 0, width, height );
const frame = ctx.getImageData( 0, 0, width, height );
// cameraPose holds the rotation/translation information where the camera is estimated to be
const cameraPose = alva.findCameraPose( frame );
// planePose holds the rotation/translation information of a detected plane
const planePose = alva.findPlane();
// The tracked points in the frame
const points = alva.getFramePoints();
for( const p of points )
{
ctx.fillRect( p.x, p.y, 2, 2 );
}
};Ensure Emscripten is installed and activated in your session.
$: source [PATH]/emsdk/emsdk_env.sh
$: emcc -v
Alva makes use of C++11 features and should thus be compiled with a C++11 or higher flag.
| Dependency | Description |
|---|---|
| Eigen3 | Download Eigen 3.4. Find all releases here.This project has been tested with 3.4.0 |
| OpenCV | Download OpenCV 4.5. Find all releases here.This project has been tested with 4.5.5. |
| iBoW-LCD | A modified version of iBoW-LCD is included in the libs folder. It has been turned into a static shared lib. Same goes for OBIndex2, the required dependency for iBoW-LCD. Check the lcdetector.h and lcdetector.cc files to see the modifications w.r.t. to the original code. Both CMakeList have been adjusted to work with Emscripten. |
| Sophus | Sophus is used for SE(3), SO(3) elements representation. |
| Ceres Solver | Ceres is used for optimization related operations such as PnP, Bundle Adjustment or PoseGraph Optimization. Note that Ceres dependencies are still required. |
| OpenGV | OpenGV is used for Multi-View-Geometry (MVG) operations. |
For convenience, a copy of all required libraries has been included in the libs/ folder. Run the following script to compile all libraries to wasm modules which can be linked into the main project.
$: cd ./AlvaAR/src/libs/
$: ./build.sh
Run the following in your shell before invoking emcmake or emmake:
$: [PATH]/emsdk/emsdk_env.sh
Then, run the following:
$: cd ./AlvaAR/src/slam
$: mkdir build/
$: cd build/
$: emcmake cmake ..
$: emmake make install
- Improve the initialisation phase to be more stable and predictable.
- Move feature extraction and tracking to GPU.
- Blend visual SLAM with IMU data to increase robustness.
AlvaAR is released under the GPLv3 license.
OV²SLAM and ORB-SLAM2 are both released under the GPLv3 license. Please see 3rd party dependency licenses in libs/.
Alan Ross: @alan_ross or [email protected]
Project: https://github.com/alanross/AlvaAR
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent