CIE53 | I03

Intelligent Robotics and Human Motion Analysis

Wednesday, 21 October | 10:45-12:15 GST

Room 1 (provisional) | In person

Chair: Maher Maalouf

Conference schedule

Paper #65

Retrieval-grounded robot program generation and simulation-based correction via Model Context Protocol

Zhichao Zhou, Siyuan Chen, Omkar Salunkhe, Ebru Turanoglu Bekar, Johan Stahre and Anders Skoogh

Presenting author: Siyuan Chen

Abstract

Flexible manufacturing requires industrial robots to be reprogrammed rapidly as product variants change. However, existing natural-language robot programming workflows often stop at text-level generation or retrieval grounding, leaving execution-level failures in vendor-specific simulation environments insufficiently addressed. This paper presents a language-model-based workflow that generates, validates, and iteratively corrects ABB RAPID robot programs from natural language task descriptions. A dual-stream retrieval-augmented generation (RAG) pipeline grounds code generation in verified technical documentation and production templates, substantially reducing domain-specific errors produced by ungrounded language models. A custom Model Context Protocol (MCP) server connects the language-model client directly to ABB RobotStudio for automated code upload, simulation execution, and diagnostic feedback. The evaluation combines a 30-query retrieval benchmark, scoped code-generation checks, and RobotStudio case studies in a simulated pick-and-place manufacturing cell. The simulation loop exposes execution failures that static and semantic checks alone cannot catch, including suction release-height errors, unreachable placement targets, and configuration-dependent recovery motions. The results show how RAG and MCP can connect grounded code generation with executable feedback from industrial robot simulation software, reducing domain-specific errors and iteration effort while retaining expert setup and final supervision as necessary boundaries.

Paper #251

Risk-Aware Obstacle Prioritization Framework for Indoor Robots Using Deep Learning

Farhana Ahmed, Rosli Besar and Nor Hidayati Abdul Aziz

Presenting author: Farhana Ahmed

Abstract

The current object detection algorithm can detect objects accurately but fails to determine the riskiest object among all obstacles. The lack of prioritizing obstacles according to risk leads to less efficient decisions made in the indoor setting. This paper proposes RAOP (Risk-Aware Obstacle Prioritization); an enhancement of the state-of-the-art detection framework, YOLOv11, by adding a post-detection Risk Assessment engine based on distance between obstacles, object size, semantics of the obstacles, and object detection confidence. To enhance the performance of the system under challenging indoor lighting conditions, CLAHE is added as a preprocessing step. The proposed framework categorizes risk levels to four actions: STOP, SLOW DOWN, AVOID, and SAFE. Performance results show that the proposed method outperforms its counterparts, providing improvements of 6.49%, 0.81%, and 16.25% in precision, mAP@50, and low-light object detection accuracy, respectively. Finally, a GUI-based standalone software implementation verifies its effectiveness for indoor robotic navigation tasks.

Paper #79

AaSoM: A Novel Unified Spatial-Temporal Motion Framework using Multi-Head Attention and BiGRU for Indian Sign Language Recognition

Priyanka Suresh, Hari Kumar Reddy B, Vansh Gupta, Madhusmita Das and K L Sai Prasad

Presenting author: Priyanka Suresh

Abstract

Sign language is considered a spatial-temporal representation of patterns, where curvature, acceleration, and dynamic intensity play major roles. However, existing self-supervised skeleton-based sign language recognition (SLR) methods primarily depend on static joint configurations and first-order derivatives, limiting motion-capture from higher-order motion derivatives. In this paper, we propose an Attention-adaptive Second-order Motion (AaSoM) framework, a hybrid Attention-BiGRU architecture for Indian Sign Language (ISL) recognition that operates on three-stream skeletal kinematics (pose, velocity and acceleration), extracted using MediaPipe and concatenated to a unified spatial-temporal representation. Our proposed approach enriches skeletal representation and captures salient inter-joint dependencies that demonstrate robust temporal modeling with strong generalization across the datasets used. The performance of our AaSoM framework is evaluated using loss function, and accuracy metric. Our experimentation achieved a testing accuracy of 89.14% and reduced loss across 50 epochs. Along with this we have compared our framework with other state-of-the-art models. Thus our findings highlight the need for a higher-order dynamics (second-order) in skeleton-based (spatial-temporal) self-supervised SLR learning.

Paper #110

Real-Time Golf Swing Coaching via Pose Estimation and Reference-Based Quality Scoring on an Edge AI Device

Fatma Ramadan, Maryam Alkharji, Mariam Alratrout, Nadin Waleed, Kin Poon and Ubaid Ahmad

Presenting author: Kin Poon

Abstract

This paper presents a golf swing coaching system that evaluates movement quality and delivers real-time corrective feedback. The system was implemented entirely on an edge AI platform, eliminating reliance on cloud-based infrastructure to ensure low latency, improved data privacy, and reliable operation in offline environments. A structured computer vision pipeline is used where human pose data is extracted from live video using landmark detection, normalized to ensure consistency across users with different body proportions, and transformed into joint-angle representations for analysis. Movement quality is evaluated through comparison with multiple reference poses, enabling robust handling of natural variation in user performance. A finite-state model is used to segment motion into distinct golf swing phases, allowing context-aware feedback generation. By combining efficient geometric computations with lightweight models, the system demonstrates the practicality of a standalone Edge based solutions for real-time motion analysis and coaching applications.

Paper #171

A Multi-Criteria Decision Framework for Selecting Video-to-3D Reconstruction Software in Forensic Engineering

Zehra Canan Araci, Afra Ahmed Alsharif, Nour Samer Kabour, Ghala Amjad Dablan, Amna Almaazmi, Abdullah Al Ali, Mohammed Al Ali and Mohammad A. Alqassim

Presenting author: Zehra Canan Araci

Abstract

Motion-video footage is routine forensic evidence, yet practitioners lack a structured, reproducible basis for choosing among the many tools that convert it into three-dimensional (3D) models. This study presents an expert-validated decision framework applying the Analytic Hierarchy Process (AHP) to video-to-3D software selection under realistic forensic conditions. Four photogrammetry-based tools were evaluated across 36 trials using motion video from a mobile phone and a 360-degree camera under four conditions (indoor/outdoor, day/night). Performance combined processing and upload time with structured assessment of user experience and output quality, while four criteria (speed, accuracy, cost, user-friendliness) were weighted by ten Dubai Police forensic experts. Aggregation yielded a consistent ranking (consistency ratios within 0.10): RealityScan first, then 3DF Zephyr, Agisoft Metashape, and EyesCloud3D. Sensitivity analysis confirmed robustness except when user-friendliness dominated, favouring 3DF Zephyr. The contribution is a transferable, criteria-based evaluation method that forensic units can adapt to their priorities.