Beyond Classical Positional Encodings: A Learnable QFT-Inspired Framework for Transformer Language Models

Sara Tehsin¹, Tallha Akram^2,*, Syed Rameez Naqvi³, Meshal Alharbi⁴, Abdulrahman Alabduljabbar²
1 Faculty of Informatics, Kaunas University of Technology, Kaunas, Lithuania
2 Dept. Information Systems, College of Computer Engineering and Sciences, Prince Sattam bin Abdulaziz University, Al-Kharj, Saudi Arabia
3 Dept. Computer Science, Tulane University, New Orleans, LA, USA
4 Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam bin Abdulaziz University, Al-Kharj, Saudi Arabia
* Corresponding Author: Tallha Akram. Email: email

Computers, Materials & Continua https://doi.org/10.32604/cmc.2026.081304

Received 27 February 2026; Accepted 14 May 2026; Published online 11 June 2026

Download PDF

Abstract

Transformers have become the dominant architecture for sequence modeling in natural language processing; however, their effectiveness critically depends on how positional information is encoded. Conventional positional encodings, while effective, may have limited structural flexibility for capturing complex global sequence relationships. Recent quantum-inspired approaches have sought to address this limitation, yet many either oversimplify quantum principles or introduce substantial computational or hardware overhead. We introduce a novel Quantum Fourier Transform (QFT)-inspired positional encoding scheme for transformers, motivated by the structured frequency representation of the QFT. Unlike prior approaches that either emulate quantum operations superficially or require complex circuit constructions, the proposed method provides a learnable hybrid encoding that preserves quantum-inspired structure while remaining aligned with hardware-efficient circuit primitives and structurally compatible with future near-term quantum implementations. Experiments on WikiText-103 indicate that the proposed encoding achieves competitive perplexity, improved robustness to input scrambling, and stable training behavior relative to alternative quantum-inspired baselines under the evaluated settings. Preliminary circuit-level simulations further suggest favorable noise resilience of the associated encoding primitives. These findings support the potential utility of incorporating quantum-inspired design principles into deep learning architectures and provide a foundation for future exploration at the interface of quantum computing and transformer-based natural language processing (NLP).

Keywords

LLMs; positional encoding; Quantum Fourier Transform; positional embeddings; hybrid quantum-classical models; near-term quantum devices; quantum transformer

Downloads
- Full-Text PDF
Citation Tools
- BibTex
- EndNote
- RIS

181

View
33

Download
0

Like

Multimodal Sentiment Analysis Based on a Cross-Modal Multihead Attention Mechanism
Lujuan Deng, Boyi Liu, Zuhe Li
Improve Chinese Aspect Sentiment Quadruplet Prediction via Instruction Learning Based on Large Generate Models
Zhaoliang Wu, Yuewei Wu, Xiaoli...
Enhancing Relational Triple Extraction in Specific Domains: Semantic Enhancement and Synergy of Large Language Models and Small Pre-Trained Language Models
Jiakai Li, Jianpeng Hu, Geng Zhang
LKPNR: Large Language Models and Knowledge Graph for Personalized News Recommendation Framework
Hao Chen, Runfeng Xie, Xiangyang...
Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models
Zheyi Chen, Liuchang Xu, Hongting...

All issues

Online First

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Beyond Classical Positional Encodings: A Learnable QFT-Inspired Framework for Transformer Language Models

Abstract

Keywords

181

33

0

Further Information

Guidelines

Follow Us

Join Us

Contact Us

WhatsApp:

Share Link