CIE53 | O02

Trustworthy AI and Information Integrity

Tuesday, 20 October | 10:45-12:15 GST

Room 4 (provisional) | Online

Chair: Firda Rahmadani

Conference schedule

Paper #35

GUARDIAN: An Operational Multi-LLM Orchestration Platform for Trustworthy AI-Assisted Governance Evaluation

Andrew Vance and Taylor Vance

Presenting author: Andrew Steven Vance

Abstract

Artificial intelligence is increasingly used to evaluate governance frameworks, cybersecurity readiness, and emerging technology policies. However, organizations increasingly rely on AI-generated assessments without visibility into how different language models interpret identical governance artifacts. This paper presents GUARDIAN (Governance Using AI, Risk Detection, Intelligence Analytics, and Norms), a consensus-based, multi-LLM orchestration platform that evaluates AI-generated governance assessments. Integrating multiple Western-trained and Arabic-first language models, GUARDIAN provides consensus analysis, identifies interpretive divergence, detects bias and prompt-poisoning, and preserves traceable evidence supporting AI-assisted decision making. Implemented using AWS, FastAPI, PostgreSQL, and Streamlit, the platform supports standalone deployment or API integration with existing government and enterprise systems. Operational testing demonstrated systematic interpretive differences between Western-trained and Arabic-first models, illustrating the value of transparent evaluation of AI-generated governance assessments across diverse cultural and regulatory environments.

Paper #88

A Framework for Coordinated Multi-Agent Reasoning with Dynamic Quality Assurance: Theory and Application to Legal Analysis

Osama Hosam, Achraf Ghorbel, Nassim Tinkicht and Nourchène Elleuch Ben Ayed

Presenting author: Osama Hosameldeen

Abstract

The intersection of artificial intelligence and complex reasoning tasks demands systems that can integrate diverse expertise while maintaining rigorous quality standards. This paper introduces a novel framework for coordinated multi-agent reasoning that addresses three fundamental challenges: maintaining analytical consistency across specialized agents, dynamically validating output quality without human intervention, and optimizing resource allocation across reasoning tasks. While demonstrated in the legal domain—specifically through a multi-agent system for patent infringement analysis—the framework's principles are domain-agnostic. The key theoretical contributions include a formal model of context-chaining that enables agents to build upon prior analyses while preserving specialized perspectives, and a multi-dimensional quality metric comprising coherence, groundedness, completeness, and structure that achieves 0.84 correlation with human expert judgments. Empirical evaluation shows that persona-based specialization yields 23% higher quality scores than single-agent baselines, context chaining contributes an additional 11% improvement, and iterative validation boosts reliability by 31%. From these results we derive three generalizable design principles for multi-agent reasoning systems: the specialization-consistency trade-off, the quality-cost frontier, and validation-driven optimization. These principles provide a blueprint for building AI systems capable of expert-level analysis across domains such as medicine, finance, and engineering.

Paper #54

Syntactic Variety, Behavioural Collapse: A Pre-registered Decision-Trace Audit of LLM-Generated Combinatorial Heuristics

Achraf Ghorbel, Nourchène Elleuch Ben Ayed, Nassim Tinkicht, Mahdi Jemmali, Majed Bouchahma and Osama Hosam

Presenting author: Majed Bouchahma

Abstract

LLM-driven heuristic design frameworks, including FunSearch, EoH, ReEvo, and their successors, often treat independently sampled code outputs as a diverse candidate pool, with diversity assessed at the codehash or token-edit level. Whether this syntactic variation translates into behavioural diversity has not been tested directly. We report a pre-registered audit of 900 heuristic implementations generated by three 7Bclass open-weight code-specialised models on two NP-hard problems, 0/1 knapsack and one-dimensional bin packing, under a five-level prompt ablation from canonical labelling to full adversarial domain anonymisation. The Output Trajectory Signature (OTS), a decision-trace equivalence framework applied to a fixed nine-instance battery, reveals severe behavioural compression: 278 syntactically distinct knapsack implementations reduce to eight OTS classes (IDR = 34.75; 95% CI [21.88, 44.00]), and 126 bin-packing implementations reduce to thirteen (IDR = 9.69; 95% CI [6.08, 9.75]). Canonical labelling produces a directional shift in textbook-baseline matching for bin packing (PAI = 0.171), although its bootstrap interval spans zero; the canonical-adversarial distributional test remains underpowered. Behavioural convergence persists under full adversarial anonymisation, suggesting a structural prior that survives surface-level domain re-skinning. Code-hash diversity substantially overstates the effective behavioural coverage of LLM-generated heuristic pools; behavioural equivalence auditing should therefore precede claims of heuristic diversity, novelty, or prompt-induced exploration.

Paper #139

IDRF: A Bounded Graph-Based Framework for Investigative Dependency Reasoning

Aleena Harold Peter, Akhil Harold Peter, Madhusmita Das and Chandramani Sahoo

Presenting author: Aleena Harold Peter

Abstract

Existing dependency analysis approaches primarily use metric-driven assessment, vulnerability detection, visualization dashboards, or prediction models. However, they often lack interpretability in explaining why certain dependencies warrant further attention. To address this limitation, we propose Investigative Dependency Reasoning Framework (IDRF), a prototype system designed to analyze python package dependencies using structural metrics and metadata collected from the python package index (PyPI) and related repositories. IDRF evaluates twelve bounded dependency environments comprising 318 packages, constructs bounded dependency graphs with limited-depth expansion, and computes investigative priority scores using heuristic metrics. The system produces human-readable investigative insight cards highlighting dependencies requiring further analysis based on interpretable scoring rules, generating 60 insight cards across the evaluated environments. Results show that IDRF can surface dependencies that warrant additional attention within bounded dependency constraints. It demonstrates how bounded graph analysis coupled with priority scoring produces interpretable dependency reasoning.

Paper #256

Modelling Misinformation Sentiment Dynamics and Attention Displacement on Social Media: Constructing Markov Chain Based Misinformation Indices within a Modified Information Disorder Spectrum

K M Najib Hayder and Mehdi Toloo

Presenting author: K M Najib Hayder

Abstract

For intervention in information disorder on social media, detection alone is insufficient because the volume and speed of circulation make full investigation of all disorder infeasible. This paper develops a conceptual prioritisation framework for information disorder on X (formerly Twitter), using Community Notes as an intervention-focused frontier sampling mechanism to narrative level instead of individual post level. A linguistic risk score measuring volatility of language to proxy for veracity, and a belief proxy score to proxy for vulnerability of users towards believing disordered information were used to define a 9-state risk-belief space, with state transition modelling using a Markov chain under partial observability. The results were used to construct two indices - Credibility Influence Index (CII) for directional drift in belief and the Attention Displacement Index (ADI) for magnitude of linguistic volatily. Monte Carlo bootstrap resampling provides uncertainty estimates. Empirical analysis on six narrative clusters supports a three-way prioritise/observe/deprioritise decision logic, offering a reproducible, uncertainty-aware conceptual triage framework, establishing the feasibility of a robust intervention prioritisation framework.