Improving Differential Equation Solving in Compact Language Models via Activation Steering and Reinforcement Learning
Anton Surkov, Vera Ignatenko*, Sergei Koltcov
Laboratory for Social and Cognitive Informatics, National Research University Higher School of Economics, Saint Petersburg, Russia
* Corresponding Author: Vera Ignatenko. Email: 
Computers, Materials & Continua https://doi.org/10.32604/cmc.2026.083643
Received 07 April 2026; Accepted 10 June 2026; Published online 03 July 2026
Abstract
Large language models have recently demonstrated promising capabilities in mathematical reasoning; however, their performance on tasks requiring strict symbolic manipulation, such as solving differential equations, remains limited, especially for compact models. In this work, we investigate whether activation steering combined with reinforcement learning can improve the quality of solutions generated by pretrained language models without modifying their weights. In particular, we focus on relatively small-scale models, which exhibit limited baseline performance on symbolic mathematical tasks, and study whether their capabilities can be enhanced through activation-level interventions. The proposed approach introduces trainable steering vectors that are injected into the internal activations of the model during generation. These vectors are learned separately for different classes of differential equations, forming a set of specialized steering experts. Training is performed using a reinforcement learning framework, where multiple candidate solutions are generated for each problem and evaluated using reward functions based on symbolic correctness, structural coincidence, or BLEU similarity. The optimization process updates only the steering parameters. Experiments are conducted on a general-purpose language model, a mathematically specialized model, and an instruction-tuned mathematical model using a synthetically generated dataset of ordinary differential equations. The results demonstrate that activation steering significantly improves the symbolic correctness and overall quality of generated solutions across most equation types. Steering applied across all transformer layers consistently outperforms steering restricted to the final layers. The largest improvements are observed for homogeneous and polynomial equations, while higher-order inhomogeneous equations remain the most challenging. Among the evaluated models, the Math-Instruct variant combined with activation steering achieves the strongest overall performance across most equation types, particularly for homogeneous, polynomial, and first-order equations. Notably, although the mathematically specialized model demonstrates stronger baseline performance than the general-purpose model, the proposed method substantially improves the latter, which in some cases achieves comparable or even higher correctness scores under steering. Overall, the results show that activation steering can significantly enhance the performance of compact language models, allowing them to partially compensate for their limited baseline capabilities on differential equation solving tasks. These findings indicate that activation steering combined with reinforcement learning serves as an effective and lightweight adaptation mechanism for improving the quality of solutions to differential equations generated by compact language models. The results highlight the potential of activation-level control as an alternative to full model fine-tuning, particularly in resource-constrained settings.
Keywords
Large language models; steering; reinforcement learning; mathematical reasoning; differential equations
Open Access
Login
Register
Submit a Paper
Propose a Special lssue
Download PDF
Downloads
Citation Tools
