Stitched Video

Stitched Video is an entity that can calibrate a camera rig containing one or more cameras. The cameras can then be stitched together during runtime using different techniques. Common uses cases for the Stitched Video are to remove the fish-eye effect for one camera or to make a larger stitched video by combining multiple cameras.

Another way to think about the use case for Stitched Video is that where a 2D Video presents video like one would see video on a TV or phone screen the Stitched Video allows to look out from the camera like if the viewer were at the center of the camera rig. In this way of thinking about it, the purpose of a Stitched Video is to visualize the world as correctly as possible based on the cameras that are part of it.

A Stitched Video requires calibration to work properly.

Stitching

The Stitching tab contains settings for controlling the stitching of the video.

Blend Distance

Given two cameras with some overlap between them, Blend Distance sets how much of the overlap is visible. Value goes from 0.0-1.0 where 0.0 is a sharp line (no visible overlap) and 1.0 shows the full overlap.

Blend Curve

The overlap zone is the area where video is shown from both cameras. Blend Curve describes the blending function across the overlap zone.

Type

The stitching can be viewed on different surfaces. The currently supported types are:

Sphere
GPU Auto
Model

Show Stitch Zone

When enabled, it will show where the stitch between different video feeds is located, can be helpful to have enabled when changing stitch settings.

Type specific settings

Some specific settings are only visible when a type is selected.

Radius(Sphere specific)

Sets the size of the sphere which the video is displayed on. It recommended to set the size to the distance of the object most likely to be look at.

Lock Auto-Stitch(GPU Auto specific)

Reset(GPU Auto specific)

Add (Model specific)

Adds one or more of the selected primitives, the following are supported:

Box
Sphere
Quad
Model From File

All model types have settings regarding transform, scale, and rotation. There are other settings that are specific for this type called Entity transform which can connect this primitive to other entities transform.

Quad has a button called Set As Calibrated Floor, if a horizon calibration has been made it will automatically set the quad to the height of the calculated floor.

Manual Geometry Adjustment

Sector Regions

Add
Angle
Size
Blending
Distance

Ground Plane

Active
Angle
Height
Blending

Circular Placed Regions

Add

Preview Depthmap Mod

Advanced

Light Field Render
Eye Separation

Color

Color is described in Color Handling.

Chroma Key

Chroma Key is described in Chroma Key.

Matte Key

Matte Key is described in Matte Key.

Horizon

Horizon can change the rotation of the stitched video to better align the video with the real world if the cameras are tilted. The horizon can be calibrated with the use of an aruco marker. For the calibration, the marker on the backside of the calibration board can be used or it can be printed.

When printing a marker, make sure it is printed on a flat stable surface!

At least three markers need be captured for the calibration, but we recommend using more captures to get a better calibration. When three markers have been captured the button "Calibrate" will be available, when pressed, it will rotate the stitched video. The video might not be centered where the user wants it, the manual horizon values can then be used to make fine-tunings.

Angle: The angle of each axis that the stitched video will be rotated around.
Bake Angle: This button will take the horizon angles and bake them into the camera rig settings. This is recommended after horizon calibration has been made.
Reset: Resets the applied horizon.
Drag To Adjust: When enabled, make it possible to change the horizon by clicking and dragging the view.
Capture marker: When pressed, captures a visible marker.
Calibrate: Runs the calibration algorithm and changes the horizon. At least three markers must have been captured.
Marker Position: Each marker that has been captured will be shown here along with the position. The maker can be deleted with the Delete button.

Horizon Calibration Settings

Show marker: When enabled it will show the captured marker in the 3D world.

The rotation may not always be accurate, however the rotation is not used in the algorithm so it can be discarded.

Marker Size: The size of the physical calibration board.
Dict Size: The size of the dictionary with markers that will be created.
Marker Bits: The number of bits in each dimension for the marker.
Marker ID: The ID of the marker in the dictionary.
Reset To Default Values: Resets to the default values.
Save as .png: Saves the marker to a .png file.

Calibration

Calibration is used to calibrate the camera positions, this can be done with one camera or multiple.

Figure 1. The image shows how a failed calibration look like. It can be seen that 'Image Set 4' has a large error, the image would need to be retaken.

Capture Image Set: Capture the current frame to use for the calibration.
Do Calibration: Does a camera rig calibration with the currently captured images. It is available after at least one image is captured.
Delete Last Capture: Delete the last calibration image taken, only appears after a minimum of 1 image has been taken.
Capture Image Sets: This tab contains all the images taken of the calibration board and the pixel error for each image. The Images can also be inspected and deleted.

Advanced

Advanced settings for the calibration, recommended is to let them be the default value.

Look For Markers: If enabled, the software will look for calibration boards and single markers.
Show Board: Shows the captured location of the calibration board.
Show Cameras: Shows the positions of the cameras in the 3D space in relation to each other.
Locked Camera: Lock one of the cameras so the rest moves in relation to that camera.
Locked Forward: Lock rotations of the camera selected in the above setting.

Calibration Board

Settings for how the calibration should look like, the supplied calibration board from Voysys use the default values.

Square Length

The width of each square on the calibration board.

If a calibration board has been printed it is important to measure the square size very accurately.

Cols

The number of columns of aruco markers.

Rows

The number of rows of aruco markers.

The displayed calibration board might be flipped so the columns and rows may be inverted.

Fill Ratio

Set how much of the white area the aruco marker should cover.

Dictionary

Set which pre-made dictionary to use.

Reset To Default Values

Sets all values to the default.

Save as .png

Save the calibration board image to a .png file for printing.

Single marker

Settings for the single marker detection, this marker is used for both calibrations and check calibrations.

Marker Size: The size of the physical calibration board.
Dict Size: The size of the dictionary with markers that will be created.
Marker Bits: The number of bits in each dimension for the marker.
Marker ID: The ID of the marker in the dictionary.
Reset To Default Values: Resets to the default values.
Save as .png: Saves the marker to a .png file.

Calibration Check

Calibration check can be used to analyse how good the calibration is, this is done by using the single marker on the backside of the calibration board. If no calibration board is available, it is possible to print one. A .png file can be downloaded from the Single Marker tab. Most of the settings are only visible after a calibration check has been made.

Check Calibration Using Marker: Pressing this button while having an aruco marker visible by two or more cameras will give a value on how good the calibration is.
Force Non-Linear: Force the solver to use Non-Linear equations to solve the problem.
Show Marker: Show a marker where the software has detected them in the video.
Marker Images: Show an image of the marker on the place the marker was detected.
Show rays: Show the rays which were used to detect the marker.

Result

Marker ID: Show which ID in the dictionary the detected marker has.
Result Computed Using Non-Linear Optimization
Distance to Marker Center: The distance to the marker center.
Marker Position: The calculated position of the marker.
Residual error: The error of the captured marker, larger error means that the calibration is not good.
Seen in Camera: Shows which cameras that detected the marker during the check.

Geometry Control

Optimazation Options

Use AprilTag2: Uses the AprilTag 2 algorithm to detect the markers which can give a better result, however, will take more computation time.

Rig Parameters

Very advanced settings about how the camera rig is set up and should not be tampered with if you do know exactly what you are doing.

Reset: Resets the rig parameters.
Rig Rotation: Rotates the whole rig which includes all cameras.

Camera N

Position
K
Offset
Rotation
Max Theta
Horizontal FoV
Vertical FoV
Diagonal FoV

Calibration Manipulation

Center Position On Camera N
Center Camera N Rotation on New Forward
New Forward

Stored Calibrations

The software ships with some premade calibration for common cameras, by selecting one in the dropdown menu it sets all camera rig parameters.

More calibrations can be added to the dropdown menu by specifying a path to a folder containing '.camera' files. More information is described in Application Settings under "Camera Calibration".