Submit

Login Login

Register Register

Home / Journals / CMC / Online First / doi:10.32604/cmc.2026.080758
Journal Menu
Special Issues
Table of Content

All issues

Online First

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Open Access

ARTICLE

Causal Counterfactual Transformers for Explainable Video-Based Action Recognition Based on CauFormer-V Framework

Hend Alshaya*
Applied College, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia
* Corresponding Author: Hend Alshaya. Email: email
(This article belongs to the Special Issue: Advancing Action Recognition: Privacy, Explainability, and Optimization)

Computers, Materials & Continua https://doi.org/10.32604/cmc.2026.080758

Received 14 February 2026; Accepted 01 June 2026; Published online 03 July 2026

Abstract

Video representation learning faces very challenging goals, including spurious temporal correlations, confounding visual features, and failure to learn real causal relationships between video events. Current transformer-based approaches learn statistical relationships rather than causal interactions, leading to weak generalization and high sensitivity to distribution changes. The current paper proposes a new Counterfactual Transformer Network, named CauFormer-V, that combines causal inference concepts with temporal representation learning for video. The framework was proposed and includes three main innovations, (1) a Causal Temporal Attention (CTA) mechanism, a mechanism that specifically models causal dependencies among video frames via do-calculus intervention, (2) a Counterfactual Video Generator (CVG) module, which is used to generate counterfactual video representations to enable causal learning, and (3) a Temporal Causal Graph (TCG) network, which is a structure that explicitly models causal dependencies across multiple temporal scales. Prolonged testing on four benchmark datasets, Something-Something V2, Kinetics-400, UCF101, and HMDB51, shows that CauFormer-V yields the highest possible results of 74.8% Top-1 accuracy on Something-Something V2, 86.7% on Kinetics-400, 97.9% on UCF101, and 78.4% on HMDB51, outperforming other leading methods by 2.1%–4.3%. Ablation studies confirm the effectiveness of each component, whereas visualization analysis indicates that CauFormer-V can extract semantically relevant causal temporal patterns. The presented framework offers a principled approach to learning powerful video representations with higher interpretability and stronger generalization.

Keywords

Causal inference; counterfactual learning; video representation learning; temporal transformer; action recognition; causal attention mechanism

  • 171

    View

  • 40

    Download

  • 0

    Like

Related articles
Copyright© 2026 Tech Science Press
© 1997-2026 TSP (Henderson, USA) unless otherwise stated

Share Link