XR - Latency Calibration

Automatically evaluate the amount of frame delay between sending a picture to the stage and receiving it back through the camera.

LatencyCalibration LatencyCalibration LatencyCalibration LatencyCalibration

XR - latency calibration needs a specific XR/AR license. If you’d like to try out these features to decide whether you want to purchase an XR/AR license, please contact us.

1) Theory

This step calculate a frame number of delay between the content broadcast in the Screens and the reception of it in the camera. It set a FrameDelayTextureModifier Icon Frame Delay according to the result. Indeed, there is delay between the moment Smode is broadcasting a content in the screens and the moment this pictures is actually get back as video input. This is the close loop latency.

XR-Delay XR-Delay

The main objective of this part is to count the number of frames that there is between these 2 moments. In order to do it, Smode will a broadcast a White Glitch in each screens and will wait to see it in the video input.

The difference in time between the moment of emission and reception is equals to the delay you will have to apply on the set extension and content broadcasted in the screens.

The example below is a simplified illustration of a XR set-up with 6 frames of delay.

DelayCalculation DelayCalculation


2) Before Starting a Latency Calibration

Before launching the latency calibration, you must ensure that:

  • Your Smode is fully optimized in graphical performances. Use the Profiling feature if needed.

  • The frame rate is stable. If not, it might be because your set-up isn’t fully genlocked

  • You are filming every impacted screens

  • the camera does not move

  • Alert people not to pass in front of the camera

Once you ensure the previous points, you can start a calibration.


3) UI Check-out

  1. Viewport : Display the stream of the XRCalibrator Icon XR Calibrator VideoInputTextureGenerator Icon Video Input

  2. Tracker information : Display the current position orientation of the Tracker Icon Tracker of the PhysicalCamera Icon Physical Camera such as the deviation. A positive value of Position Deviation and Orientation Deviation means that your tracker is currently moving.

  3. Added delay : (Frame) delay added for each screen in result of the XR - Latency Calibration , if some of your screen are faster than other.( if not all screen are at 0 Added delay you may have to check your setup, and look for some hardware that add delay on some of the screen)

  4. Delay in frames : (Frame) delay calculated by the algorithm.

  5. Delay Mean : (ms) Average time between the emission of a white content and its reception.

  6. Delay Standard Deviation : actual difference between the delay mean (which is an average) and the actual time between emission and reception. If this value is not 0 your setup is probably not genlocked


4) Calibration process

To launch this calibration part, simply press “calibrate” on the CameraAnalyzerWorkspaceComponent

Latency-Calibrate Latency-Calibrate

This is what should happen:

DemoLatencyCalibration DemoLatencyCalibration

Once you have the Delay you can apply it into your Extension and AR in the VFX processor, as explained here: XR - Extended and augmented reality


TroubleShoots

If you have a Standard Deviation not at 0 for all screen:

LatencyTroubleShoot01 LatencyTroubleShoot01

  • Your setup is not genlocked ?

  • Check the EDIDs of each screens.

  • Smode is dropping ?

stableFps stableFps

  • Smode is not optimized, use the Profiling feature

  • First of all you must ensure you camera and led screen are at same refreshRate

  • Smode requested Frame-rate is the same as your screen refreshRate

RequestedFps RequestedFps

If you have all your Standard Deviation a 0 you can ignore the warning “Clock Synchronization was not perfect”


The second latency you need to adjust: Camera Tracking .

Next step: XR - Geometry Calibration .