AR Data Format & API

AR Coordinate System

Coordinate Frame:

• X-axis: Right

• Y-axis: Up

• Z-axis: Forward

Pose : $[t_x, t_y, t_z, q_x, q_y, q_z, q_w]$

• Translation (first 3 values): Position in meters

  • $t_x, t_y, t_z$: X, Y, Z coordinates

• Rotation (last 4 values): Orientation as quaternion

  • $q_x, q_y, q_z$: Imaginary part

  • $q_w$: Real part

Difference between local_pose and global_pose:

• local_pose: Position relative to device starting point

• global_pose: Position in shared world coordinate (when multiple devices collaborate)

Note: When using a single device, local_pose and global_pose are identical

Velocity

Format: $[v_x, v_y, v_z]$

• Calculated as: $\vec{v} = \frac{\Delta \vec{p}}{\Delta t}$

• Unit: meters per second (m/s)

• In the same coordinate frame as pose

Camera Intrinsics

Format: $[f_x, 0, 0, 0, f_y, 0, c_x, c_y, 1]$

Represents 3×3 matrix:

$$K = \begin{bmatrix} f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix}$$

Where:

• $f_x, f_y$: Focal length in pixels

• $c_x, c_y$: Principal point (optical center) in pixels

Attention

Camera intrinsics scaling required

The camera intrinsics are originally computed for the original image size 1920 × 1440.

In the current app, the streamed color image size is 640 × 480.

If you use the intrinsics with the streamed images, you must scale the intrinsics to match the new resolution.

(You can follow the method below for implementation.)

from asmagic import ARDataSubscriber
import numpy as np

sub = ARDataSubscriber("192.168.1.100")

for data in sub:
    K = np.array(data.camera_intrinsics).reshape(3, 3, order='F')
    scale_x = 640 / 1920  # = 1/3
    scale_y = 480 / 1440  # = 1/3
    
    K[0, 0] *= scale_x  # fx
    K[1, 1] *= scale_y  # fy
    K[0, 2] *= scale_x  # cx
    K[1, 2] *= scale_y  # cy
    
    print(f"Scaled intrinsics:\n{K}")

Depth Image

• Format: 16-bit unsigned integer (uint16)

• Unit: $10^{-4}$ m (0.1 mm, scaled by 10000 from meters)

• Conversion to meters: $d_{meters} = \frac{d_{raw}}{10000}$

• Range: $0 \leq d_{raw} \leq 65535$ → $0$ to $6.5535$ m

• Access: Use data.depth or data.depth_array to get numpy array

Image Data Format

Color Image: Two formats available

• color_bytes: JPEG compressed bytes

• color_array: RGB numpy array (640×480×3)

• color: Shortcut for color_array

Depth Image: Numpy array only

• depth_array: uint16 numpy array (256×192)

• depth: Shortcut for depth_array

API Reference

ARDataSubscriber

Constructor:

ARDataSubscriber(ip, port=8000, hwm=1, conflate=True, verbose=False)

Parameters:

• ip (str): iPhone's IP address

• port (int): Port number (default: 8000)

• hwm (int): High water mark (default: 1, keeps only latest message)

• conflate (bool): Message conflation (default: True)

• verbose (bool): Print connection info (default: False)

Usage:

# Create subscriber
sub = ARDataSubscriber("192.168.1.100")
​
# Continuously receive data
for data in sub:
    print(data.timestamp)
    print(data.velocity)
​
# Close when done
sub.close()

Main Methods:

Method	Returns	Description
get()	ARFrame or None	Get latest data frame
get_timestamp()	float or None	Get timestamp only
get_velocity()	np.ndarray or None	Get velocity only
get_local_pose()	np.ndarray or None	Get local pose only
get_global_pose()	np.ndarray or None	Get global pose only
get_camera_intrinsics()	np.ndarray or None	Get camera intrinsics only
get_color_image()	bytes or None	Get color image bytes only
get_depth_image()	np.ndarray or None	Get depth array only
close()	None	Close connection

Note:

• The subscriber is iterable, so you can use for data in sub: to receive frames continuously.

• All get_*() methods accept an optional timeout parameter (default: 1000ms).

ARFrame

Data object returned by get() or when iterating. For data format details, see AR Data Format Reference above.

Properties:

Property	Type	Description
timestamp	float	Unix timestamp in seconds
velocity	np.ndarray	Velocity $[v_x, v_y, v_z]$ in m/s
local_pose	np.ndarray	Local pose $[t_x, t_y, t_z, q_x, q_y, q_z, q_w]$
global_pose	np.ndarray	Global pose $[t_x, t_y, t_z, q_x, q_y, q_z, q_w]$
camera_intrinsics	np.ndarray	Camera intrinsics (3×3 flattened)
Color Image
color_bytes	bytes	JPEG image bytes (for saving/forwarding)
color_array	np.ndarray	Decoded RGB image array (640×480×3)
color	np.ndarray	Shortcut for color_array
Depth Image
depth_array	np.ndarray	Depth image array (uint16, 256×192)
depth	np.ndarray	Shortcut for depth_array
depth_width	int	Depth image width (256)
depth_height	int	Depth image height (192)
Helpers
has_color_image	bool	Check if color image exists
has_depth_image	bool	Check if depth image exists

Methods:

Method	Returns	Description
show_color(window_name)	bool	Display color image with OpenCV
show_depth(window_name, colormap)	bool	Display depth image with colormap
show_images(show_color, show_depth)	tuple	Display both images side by side