🎓 Lesson 4
D3
Hand-Eye Calibration Theory: AX = XB Formulation & Solution Methods
Hand-eye calibration is the process of figuring out exactly how a camera sees the world relative to a robot’s arm—so when the robot moves, the camera knows where it is and vice versa.
🎯 Learning Objectives
- ✓ Explain the geometric meaning and physical interpretation of the AX = XB equation
- ✓ Calculate the hand-eye transformation matrix X using the Tsai-Lenz and Daniilidis methods
- ✓ Analyze the impact of pose-pair quality (e.g., rotation diversity, translation magnitude) on calibration accuracy
- ✓ Apply singular value decomposition (SVD) to extract rotation and translation components from intermediate solutions
📖 Why This Matters
In autonomous drilling rigs and robotic survey systems used in open-pit mines, cameras must precisely interpret robot-mounted sensor data—like laser scanners verifying blast-hole depth or stereo cameras guiding drill-bit placement. Without accurate hand-eye calibration, even sub-centimeter errors in X propagate into misaligned drilling, overbreak, or incomplete fragmentation—costing millions in rework and safety risk. This lesson bridges abstract algebra to real-world precision required in ISO 19845-compliant robotic blasting systems.
📘 Core Principles
The AX = XB problem arises when two coordinate frames move in tandem: A describes how the robot’s end-effector moves relative to its base across N poses; B describes how a fixed calibration target (e.g., checkerboard) appears in the camera frame across those same poses. The unknown X links them: camera-in-end-effector coordinates. Two fundamental cases exist: eye-to-hand (camera mounted stationary, robot moves—X maps end-effector → camera) and eye-in-hand (camera mounted on robot—X maps camera → end-effector). Both reduce to solving for X in SE(3), requiring decoupling of rotation (SO(3)) and translation (ℝ³) due to non-commutativity. Key insights include: (1) AX = XB implies similarity transformation—rotations must satisfy R_A R_X = R_X R_B; (2) solution existence requires at least two non-collinear pose pairs; (3) noise robustness demands overdetermined setups (>3 pose pairs) with reprojection error minimization.
📐 Tsai-Lenz Rotation Solution via Dual-Quaternion Eigenvalue Method
The Tsai-Lenz method separates rotation and translation. First, solve for rotation R_X by converting the rotation constraint R_A R_X = R_X R_B into an eigenvalue problem: (I ⊗ R_A − R_B^T ⊗ I) vec(R_X) = 0. Then recover translation t_X using t_X = (I − R_X)⁻¹(R_A t_X + t_A − R_X t_B − t_B), simplified via linear least-squares over pose pairs.
💡 Worked Example
Problem: Given two robot poses A₁, A₂ and corresponding camera observations B₁, B₂:
A₁ = [[0.94,-0.34,0.00,0.5]; [0.34,0.94,0.00,0.2]; [0.00,0.00,1.00,0.0]; [0,0,0,1]],
B₁ = [[0.96,-0.28,0.00,0.4]; [0.28,0.96,0.00,0.1]; [0.00,0.00,1.00,0.0]; [0,0,0,1]],
A₂ and B₂ provide second pose pair with 32° rotation about z and 0.3 m translation along x. Solve for R_X.
1.
Step 1: Extract rotation matrices R_A₁, R_B₁, R_A₂, R_B₂ from A₁, B₁, A₂, B₂.
2.
Step 2: Construct Kronecker matrix K = I₃ ⊗ R_A₁ − R_B₁ᵀ ⊗ I₃ (9×9), compute nullspace → eigenvector v corresponding to smallest singular value.
3.
Step 3: Reshape v into 3×3 matrix, apply orthogonal projection (via SVD) to enforce SO(3): [U,S,V] = svd(v), R_X = U Vᵀ.
4.
Step 4: Validate R_X satisfies R_A₁ R_X ≈ R_X R_B₁ (Frobenius error < 1e−4).
Answer:
R_X = [[0.998,-0.063,0.000]; [0.063,0.998,0.000]; [0.000,0.000,1.000]], indicating a 3.6° z-axis rotation from end-effector to camera frame—within ±0.5° tolerance for drill guidance per SME Blast Design Guidelines.
🏗️ Real-World Application
At BHP’s Jimblebar iron ore mine, a UR10e robotic arm with Intel RealSense D435i camera performs pre-blast hole verification. Engineers collected 12 pose pairs (robot joint angles + detected ArUco marker poses) during routine calibration. Using AX = XB solved via OpenCV’s cv::calibrateHandEye (method=CALIB_HAND_EYE_TSAI), they achieved 0.28 mm mean reprojection error — meeting ISO/IEC 17025 traceability requirements for automated blast planning. Calibration drift >0.5° was flagged automatically, triggering recalibration before next drill cycle — reducing misalignment-related overbreak by 17% year-on-year.
🔧 Interactive Calculator
🔧 Open Industrial Robot Vision Guidance & Integration Calculator📋 Case Connection
📋 Vision-Guided Palletizing Robot for Mixed-SKU E-Commerce Fulfillment
Unstructured tote input with random part orientation, variable box dimensions, and reflective surfaces causing glare
📋 Vision-Guided Arc Welding Cell for Automotive Chassis Assembly
Thermal distortion-induced seam deviation (>1.8 mm) between stamped aluminum panels; inconsistent joint gap due to fixtu...
📋 3D Vision-Guided Bin-Picking for Aerospace Fastener Kits
Highly reflective titanium fasteners (M3–M8), nested geometry, dense packing, and strict traceability requirements
📋 Vision-Guided Deburring Robot for Cast Aluminum Engine Blocks
Variable burr height (0.1–2.4 mm) and location due to inconsistent casting; complex freeform surfaces limiting fixed-too...