How to use Illumetry SDK in Unity

Starting with version 2021.2 there is an ability to automatically move the application window to the ILLUMETRY IO display.

To create an Illumetry display in the scene, open the Illumetry menu and click on the Add Illumetry Display to Scene option. This will generate an IllumetryDisplayHandle object with all the required components.

RequiredSettingsApplyer sets the specified settings when OnEnable is called.

You can disable the component, or set Disable Applying Settings to true to ignore the following settings (optimal values are already set by default in this component):

1. Target Frame Rate specifies the frame rate at which Unity tries to render your game. On standalone platforms, the default frame rate is the maximum achievable frame rate. To use the platform's default frame rate, set Application.targetFrameRate to -1.
2. V Sync Count is the number of vertical syncs that should pass between each frame.
3. Max Queued Frames is the maximum number of frames queued up by the graphics driver.

The description of these three parameters is taken from the official Unity documentation.

Display Handle draws the display borders relative to the specified origin point, sets the local size of the display relative to the Scale Mode, and the current size of the DisplayHandle.

1. The Origin X and Origin Y parameters set the offset for the Display Rect. They have values from -1 to 1.
2. The Scale Mode parameter sets the size of the Display Handle (virtual display). It has three value options:

• Real Size makes the size of the virtual display equal to the real size of the Illumetry monitor in meters.
• Width Is One sets the width of the virtual display to 1 meter, and the height is scaled relative to the width in the standard 16:9 aspect ratio.
• Height Is One means that the height of the virtual display will be 1 meter, and the width will be scaled according to the 16:9 aspect ratio.

3. The Show Display Border parameter draws the display border.
4. Show Ruler shows the ruler (in meters) perpendicular to the display to help gauge the object depth.

After selecting the Scale Mode, you can scale the virtual display using the Scale parameter in the current Transform component. All values (XYZ) of the Scale parameter must always be equal.

For example, to make a virtual display 16 meters wide and 9 meters high, select the Width Is One option in the Scale Mode parameter, and then set the values 16 (16,16,16) in the Scale parameter of the Transform component.

Device Network Provider is the communication link between the application and connected Antilatency devices, helps to monitor changes in connected devices, and provides access to nodes of the device tree.

Default Screen Resolution sets the desired resolution for the Illumetry monitor for mono or stereo mode. This component also determines which monitor the application is currently displayed on. If the application is not on the Illumetry IO display, then it is automatically moved there.

It has the AllowReturnToFirstParamsOnDisabled parameter with a default value of true. When true, at the moment the DefaultScreenResolution component is disabled, the initial resolution and fullscreen mode settings will be returned, and the application will move back to the monitor it was originally launched on.

Starting with version 2021.2 there is an ability to automatically move the application window to the Illumetry IO display.

Glasses finds the node corresponding to the glasses, and matches the virtual camera position with the position of the glasses relative to the display.

It has a parameter called Pupillary Distance, which is the distance between a person's pupils. On average, it is 0.06 m, which is set by default.

Renderer LCD is responsible for rendering the image in stereo mode.

It has the following parameters:

1. Render Target Format. Select R16G16B16A16_S Float or R32G32B32A32_S Float format in Inspector if HDR rendering is required.
2. Common Gamma controls gamma correction. If the Color Space parameter of Project Settings has the Gamma option, leave the value "1"; if Linear is selected, an approximate optimal value is "0.43".

Mono Rendering Tracker finds the socket with the tracker which the camera will now be positioned relative to. This is used for mono mode.

1. The Tracker Socket Tag parameter should have the same value as the Tag property of the socket’s node. The Tag property of the socket can be set, for example, in AntilatencyService.

Styluses Creator finds stylus nodes and creates Stylus GameObjects, depending on the number of connected devices.

It has the following parameters:

1. Stylus GO Template contains the stylus prefab, which can have its own logic. The prefab must contain the Stylus component.
2. Required Tags allow adding custom styluses (assembled by the user) based on their Tag properties. These values will be used to search for custom styluses’ nodes.

About custom styluses:

• An unlimited number of styluses can be connected.
• A custom stylus can be assembled, for example, using a Hardware Extension Module and a socket with a tracker. The stylus must have a non-empty Tag property, which can be set in AntilatencyService. In the “Styluses Creator” component, the same Tag must be added to Required Tags for the stylus to be found by the application.

RendererLCD:

1. event Action OnBeforeRendererL is called before rendering the first frame (for the left eye). It will be called if stereo mode is active.
2. event Action OnBeforeRendererR is called before rendering the second frame (for the right eye). It will be called if stereo mode is active.

It is useful to subscribe to these events if you need to update the state of something for the left and right eyes. This is because these two frames will be rendered with some delay, and the state of something when rendering the first frame in the future may already be outdated for the second frame.

Example: a mirror - we render with one projection for the left eye, and with another for the right.

MonoRendererLCD:
OnBeforeRenderer is called before rendering the frame. It will be called if mono mode is active.

It is worth subscribing if you need to perform any tasks immediately before rendering the frame.
StylusCreator:

1. event Action OnCreatedStylus<Stylus> is called when a new stylus GameObject is created.
2. Stylus[] Styluses {get;} returns an array of active styluses.

Stylus:

1. event Action<Stylus,bool> OnUpdatedButtonPhase is called with each new state update, even if the state stays the same. If the stylus is disconnected while the button is pressed, then it signals the released state.
2. public event Action<Pose, Vector3,Vector3> OnUpdatedPose is called after the stylus pose is updated, has the parameters:

• Pose - the position and rotation of the stylus in world space,
• world velocity,
• angular velocity.

3. event Action<Stylus> OnDestroying is called when the stylus GameObject is deleted.
4. Pose ExtrapolatedPose {get;} is the last extrapolated pose received.
5. Vector3 ExtrapolatedVelocity {get;} is the last extrapolated world velocity value received.
6. ExtrapolatedAngularVelocity {get;} is the last extrapolated angular velocity value received.

An example of working with the SDK can be found here, as well as a demonstration of the stylus operation and several options for implementing its logic.

The default Unity UI render mode is “overlay”. You should select the “World Space” value of this parameter to achieve the most correct operation of the UI with the Illumetry display. This selection has two reasons:

1. Unity sets the UI Canvas to the full size of the application window, which is 1920 × 2760 pixels in stereo mode. Since Illumetry uses frame stacking technology, the top of the canvas will be displayed in the area for the left eye, and the bottom in the right eye area. Therefore, when the user looks at the screen in “overlay” render mode, some parts of the interface will only be visible to the left or right eye.
2. Moreover, the “overlay” render mode increases ghosting effect, since no post-processing (pixel overdrive and viewing angle correction) will be applied to the interface elements.

Cases:

• the cursor is visible with only one eye;
• the cursor seems to be over the UI element, but nothing happens when clicked.

This is because the image for 3D mode (frame overlay) has a specific resolution of 1920 × 2760 px.. It contains a frame for the left eye at the top, a frame for the right eye at the bottom, and an intermediate blank in the middle. So the system cursor can be either in the left or in the right frame.

There are two solutions.

1. The working area of the system cursor is a rectangle from x=0; y=0; up to x=1920; y=1080;. So you should set constraints or automatic return of the system cursor to this working area.
2. You can disable the system cursor in your project and draw your own cursor, which will be rendered 2 frames apart from the system one. For a better user experience, we recommend using this approach.