DyG-Hyena: Lightweight Temporal Modeling and Efficient Information Enhancement for Continuous-Time Dynamic Graph

1  Introduction

Dynamic graph learning adds a time dimension to graph learning in order to capture historical evolution patterns and temporal order [1,2]. It plays a pivotal role in tasks such as social network analysis [3–5], user historical behavior analysis [4], and traffic prediction. To advance this field, various frameworks and unified datasets have been developed for dynamic graph learning [6–9]. These frameworks, combined with unified datasets, have achieved promising performance in dynamic link prediction [6,10–14], dynamic node classification [6,10,15,16], dynamic edge classification [11], and other related tasks.

To achieve fine-grained temporal state perception, researchers have developed continuous-time dynamic graph frameworks (CTDGs) [6,17–21], which are distinct from discrete-time dynamic graph frameworks (DTDGs) that only perform fixed snapshot-based learning. However, these CTDG frameworks face challenges regarding computational efficiency, generalizability, and information utilization. Specifically, models based on RNNs [4,20,22] are prone to gradient explosion when handling long sequences, which leads to suboptimal performance and compromised generalization. For sampling-based aggregation models [12,17,19,23,24] like GraphMixer [19], RepeatMixer [24] that rely on short-term dependencies, their neighbor sampling strategies lack generality, resulting in information loss and reduced performance. Notably, the recent FreeDyG [12], which incorporates enhanced Node Interaction Frequency (NIF) encoding, still adopts a short-term sampling-aggregation sequence learning paradigm, resulting in limited performance on datasets with long temporal dependencies. The improved models based on temporal random walk and structural counting, such as NAT [25], PINT [22], and TPNet [26], have achieved promising results. However, they exhibit limitations including random neighbor sampling and high uncertainty. Models based on Transformers [6,27,28], such as DyGFormer [6], have achieved promising results. However, they exhibit limitations including random neighbor sampling and high uncertainty. However, since the computational cost of Transformer-based architectures scales quadratically with sequence length, this results in prohibitive overhead. ScadyG [13] introduces a paradigm for large-scale dynamic graph learning. By discretizing continuous datasets, it requires specific preprocessing steps that are implementation-heavy. DyG-Mamba [10,29] represents a state-of-the-art (SOTA) model with a balance between performance and efficiency; however, it directly adopts DyGFormer’s input encoding scheme without further exploiting available information. Therefore, we posit that computational efficiency, information exploitation, and generalization ability are crucial prerequisites for continuous-time dynamic graph learning. In this work, we achieve a balance among these three aspects through two key innovations integrated within a unified framework.

We adopt an improved and simplified version of Hyena for temporal fusion and modeling of time-sorted neighborhood feature sequences. As an efficient sequence processing model, Hyena [30,31] has been widely applied in language modeling and text generation tasks. It achieves enhanced long-sequence processing capabilities with lower computational overhead through long convolutions and data-controlled gating mechanisms. In this work, we leverage Hyena to model temporal features and nonlinear transformations of dynamic graph node neighborhood sequences. This results in a significant reduction in computational overhead for the sequence modeling layer compared to Transformer-based baselines. For link prediction, we further incorporate a CVAE layer [32] before Hyena, which takes temporal encoding as conditional input to model relative temporal constraints. This design facilitates better utilization of temporal information and improves the model’s generalization ability.

We design task-specific information enhancement schemes. To fully exploit the information inherent in the data, we customize enhancement strategies for different tasks. For link prediction, we enhance graph structural information by introducing cross-order neighborhood intersection encoding—extending the information aggregation of neighbor node interactions from one-hop to two-hop neighborhoods, which effectively enhances the model’s utilization of structural information. For node classification, we supplement statistical encoding of node attributes to enrich feature representation.

We preserve the one-hop full-neighbor truncation and aggregation framework. Our investigation reveals that retaining as much one-hop neighbor information as possible enables the model to maintain strong performance, regardless of dataset granularity or the presence of unseen nodes. Therefore, we adopt DyGFormer’s one-hop full-neighbor aggregation framework to enhance the model’s generalization ability. To summarize our contributions:

•   This work introduce an improved and simplified Hyena module for temporal modeling and capturing dependencies of node neighbor sequence features, which achieves a substantial reduction in computational overhead compared to the Transformer and is well-suited for sequence modeling tasks in continuous-time dynamic graphs.

•   Our task-specific information enhancement schemes expand the pool of available information for modeling dynamic graph evolution patterns, thereby improving downstream task performance.

•   Extensive experiments demonstrate that DyG-Hyena achieves an effective balance between downstream task performance, computational efficiency, and generalization ability.

2  Related Work

2.1 Continuous Time Dynamic Graph Learning (CTDG)

Dynamic graph learning is typically categorized into discrete-time dynamic graph learning (DTDG) and continuous-time dynamic graph learning (CTDG). DTDG frameworks partition dynamic graph datasets into multiple snapshots at fixed intervals and apply static graph learning methods to each snapshot. This approach struggles to effectively capture inter-snapshot relationships and lacks fine-grained temporal awareness [14,33–37]. In contrast, CTDG frameworks [6,17,38–40], by iteratively modeling dynamic graph evolution, can effectively learn fine-grained patterns and long-term dependencies from dynamic graph datasets. The main differences among existing CTDG models lie in: (i) approaches to acquiring neighborhood information. In CTDG models, the first step is to acquire neighborhood information for each node during each interaction, including one-hop neighbor sampling [17,19,23], multi-hop neighbor sampling [21], and adaptive truncation of all one-hop neighbors tailored to the dataset [6]. (ii) strategies for aggregating neighborhood information. Beyond widely used node and edge features in graph learning, existing baselines also incorporate Neighbor Co-occurrence Encoding (NCE) features [6] and Node Interaction Frequency (NIF) features [12]. (iii) architectures for temporal context learning. Depending on how aggregated neighborhood information is processed, models are based on MLPs [19,24,26], random walk and its improvements [22,25,26], memory networks [4,20,41], Transformers [6,12], Mamba [10,29], etc.

2.2 Hyena for Temporal Fusion and Modeling of Neighborhood Feature Sequences

Hyena [30,31] is an O(Lk) sub-quadratic sequence model (where L denotes the sequence length and k denotes the kernel size) built upon H3 [42], replacing Transformer-based attention mechanisms [43] with implicit long convolutions and data-controlled gating for efficient long-sequence modeling. Geometric Hyena [44] further extends Hyena to geometric graph tasks with SE(3) equivariance, using vector-based geometric long convolutions to capture global geometric context, thereby demonstrating Hyena’s versatility across structured data domains.

Dynamic graphs feature time-sorted neighborhood sequences with local temporal relevance, long-sequence interactions, and irregular inter-event intervals. Existing methods [6,10,29] often use block-based truncation or shallow aggregation, losing temporal dependencies or sacrificing efficiency. We tailor Hyena as the temporal fusion and modeling layer for time-sorted neighborhood feature sequences in continuous-time dynamic graphs: replacing its implicit convolution with Conv1d to align with the time axis, leveraging gating to filter heterogeneous temporal features, and stacking two layers (consistent with the baseline) for cumulative long-range temporal modeling.

This customization enables efficient aggregation of time-aware neighborhood information, generating embeddings that integrate structural and temporal dependencies to support time-sensitive link prediction. Unlike general applications of Hyena, we focus on its adaptation to dynamic graph temporal characteristics, balancing long-sequence efficiency and temporal dependency capture which is a direction that is rarely explored in prior work.

Compared to lightweight sequence models for dynamic graphs such as Mamba [29,45] and RWKV [46], Hyena is more adaptable: Mamba’s SSM-based selective scanning requires complex discretization steps and is optimized for global long sequences, which aligns poorly with the local structural sequences of time-sorted neighbors that link prediction relies on; RWKV’s recurrence suffers from long-sequence gradient attenuation and weak parallelism. Hyena’s convolution—gating architecture is directly tied to the time axis, is inherently suited for local structural sequence modeling, enables efficient parallel processing of variable-length neighbor sequences, and incurs lower customization overhead, balancing temporal dependency capture, efficiency, and flexibility with O(Lk) complexity.

2.3 CVAE: Conditional Variational Autoencoder

Conditional Variational Autoencoder (CVAE) [32] is a variant of the variational autoencoder (VAE) that accepts both primary and conditional inputs, models their joint relationship, and generates new data that align with the conditional constraints while respecting the distribution of the primary input. Prior work has shown that CVAE performs well in tasks such as knowledge graph completion. In dynamic graph link prediction, it generates diverse potential link-state scenarios and effectively models conditional variables—thereby improving conditional modeling capacity, accelerating convergence, and enhancing generalization to unseen data.

Compared with simpler conditional mechanisms, such as temporal embeddings and diffusion-based encoders, CVAE is particularly effective for dynamic graph link prediction. Temporal embeddings inject conditional signals (e.g., time) into node features independently, without jointly modeling the conditional constraints and the data distribution. Diffusion-based encoders rely on local diffusion to capture conditional dependencies, which limits flexibility in modeling complex relationships and hinders generalization. By contrast, CVAE explicitly models conditional posterior distributions, integrating conditional signals with the data distribution more precisely to capture dynamic, context-aware link patterns.

3  Preliminary

3.1 Dynamic Graph Learning

A continuous-time dynamic graph is denoted as G=(V,E), where V is the node set (n=|V|) and E={(ui,vi,ti)∣i=1,…,m} (m=|E|) is the edge set with timestamps. In this work, we focus on fine-grained continuous-time dynamic graphs, where timestamps satisfy a strictly increasing order:

ti<ti+1,∀i,(1)

meaning at most one interaction occurs at the same timestamp for any node pair. Each node and edge is associated with initial features. CTDG learning iteratively aggregates neighborhood information along timestamps to obtain time-aware node embeddings Xut at target time t, which are used for downstream tasks. For clarity, we provide a schematic illustration of the temporal structure in Fig. 1. Key variables are summarized in Appendix A.1. We also define the key variables listed in Appendix A.1.

Figure 1: Schematic illustration of the temporal structure in CTDG.

3.2 Downstream Tasks

Our downstream tasks include dynamic link prediction and dynamic node classification. Dynamic link prediction aims to predict whether nodes u and v are (or will be) connected at timestamp t. Using positive–negative sampling, positive samples are observed time-stamped edges, and negative samples are formed by fixing one node (either u or v) and sampling the other from non-interacting nodes at t, we adopt three strategies (random, historical, and inductive [6]), detailed in Appendix C.3. The model distinguishes the two sample types, assigning predicted probabilities close to 1 for positive samples and close to 0 for negative samples, as illustrated in Fig. 2a. Dynamic node classification predicts node u’s category at t by learning the mapping between nodes’ time-aware embeddings and their category labels, as shown in Fig. 2b.

Figure 2: The downstream task framework of dynamic graph: (a) for dynamic link prediction and (b) for dynamic node classification.

4  Methodology

In this section, we present the design rationale behind the DyG-Hyena model, as illustrated in Fig. 3. For each time-stamped interaction (u,v,t) in the continuous-time dynamic graph, we encode the node features, edge features, and timestamps of all one-hop neighbors of both u and v. For the link prediction task, and in order to leverage the underlying graph structure, we incorporate both neighbor co-occurrence encodings and a cross-stage neighbor intersection encoding scheme. To strengthen temporal information modeling, we introduce a conditional modeling approach based on the CVAE. For the node classification task, we incorporate a statistical node feature encoding. All these encodings are subsequently fed into the Hyena-based temporal fusion and modeling layer, which operates on time-sorted neighborhood feature sequences to generate time-aware embeddings for each node at each timestamp, thereby supporting the downstream tasks.

Figure 3: The overall process of DyG-Hyena: (a) feature encoding and information enhancement, (b) Hyena Model, (c) Downstream tasks.

4.1 Dynamic Graph Feature Encoding

Consistent with [6], we obtain node representations by aggregating one-hop neighbor interaction sequences for each node. Specifically, for interacting nodes u and v in time-stamped interaction (u,v,t), we first retrieve their historical time-stamped one-hop interactions before t. Using hyperparameters, we truncate or pad these sequences to a unified one-hop neighbor sequence length L (a hyperparameter detailed in Appendix C.3) across datasets, yielding interaction sequences Sut={(u,a1,tu1),…,(u,aL,tuL)} and Svt={(v,b1,tv1),…,(v,bL,tvL)} with tu1<tu2<...<tuL<t and tv1<tv2<...<tvL<t. We then perform the following encodings on these sequences.

4.1.1 Encodings of Baseline

Similar to the baseline [6], we construct node, edge, time, and neighbor co-occurrence encodings for one-hop neighbor sequences. For node u, its node encoding is Xu,Nt={na1,…,naL}∈RL×dN (dN: node feature dimension), and edge encoding of its neighbor interactions is Xu,Et={ea1,…,eaL}∈RL×dE (dE: edge feature dimension). If node/edge features are unavailable in the dataset, we pad them with zero vectors uniformly. For brevity, we use node u as a unified example for all subsequent encodings, with the same logic applying to other nodes (e.g., v). For time encoding, we convert absolute timestamps to relative intervals Δtl=t−tul (target timestamp t, tul: timestamp of the l-th interaction in Sut, l=1,…,L), then encode Δtl via cosine function to capture temporal periodicity: Xu,Tt=cos⁡(Δtl⋅ω)∈RL×dT (dT: time feature dimension). Here, ω={αω−(i−1)/β}i=1dT, with αω and β as hyperparameters controlling time encoding frequency range, selected to make cos⁡(Δtmax⋅αω−(i−1)/β)≈0 when i→dT (Δtmax: maximum relative time interval in the dataset). ω is fixed during model training to avoid extreme values.

Consistent with the baseline [6], we also construct neighbor co-occurrence encodings for the one-hop neighbors of interacting nodes u and v. Specifically, we calculate the frequency of each neighbor in Sut and Svt (within their respective one-hop sequences) as the neighbor co-occurrence feature, denoted as Cut∈RL×2 and Cvt∈RL×2 (with the same logic for v as u, per our unified notation). These encodings are then activated and projected to the target feature dimension via:

Xu,Ct=f(Cut[:,0])+f(Cut[:,1])∈RL×dC,(2)

where f(⋅) projects 1-dimensional frequency features to the dC-dimensional space, implemented via two ReLU functions followed by a linear layer.

4.1.2 Encodings of Enhanced Information

We introduce cross-stage neighbor intersection (CNI) encoding Xu,Gt for link prediction and node feature statistical encoding Xu,Mt for node classification.

For link prediction, we argue that graph structural encodings, such as dynamic interaction information, should not be limited to one-hop interactions. Recent research on graph structures has also confirmed this conclusion. Inspired by graph ring structures, we contend that cross-stage neighbor intersections between nodes u and v can also characterize their correlation, with the correlation strength proportional to the intersection size. To enrich graph structural information, we propose CNI encoding. To reduce computational burden, we only compute intersections between two-hop and one-hop neighborhoods: specifically, the intersection of u’s two-hop neighbors with v’s one-hop neighbors, and vice versa. We adopt a sampling strategy to select two-hop neighbors for intersection counting. The concept of CNI encoding is illustrated in Fig. 4.

Figure 4: The concept of cross-stage neighbor intersection encoding.

Formally, we construct two-hop neighbor interaction sequences Sut(2) and Svt(2) (superscript (2) denotes two-hop, distinguishing from one-hop Sut and Svt). We truncate or pad these sequences to a unified two-hop neighbor sequence length L2 (hyperparameter reported in Appendix C.3) across datasets, consistent with one-hop processing. For each neighbor in one-hop Sut and Svt, we compute its occurrence frequencies in u’s two-hop Sut(2) and v’s two-hop Svt(2) respectively, deriving a 2D vector (consistent with one-hop co-occurrence feature dimension). Stacking these vectors yields cross-stage neighbor intersection (CNI) features of u and v at target timestamp t, denoted as Gut∈RL×2 and Gvt∈RL×2 (L: unified one-hop neighbor sequence length; same logic for v). For example, if Sut={a,b,a,c}, Sut(2)={b,a,c,b,b}, and Svt(2)={c,c,b,b,a}, the CNI feature for u is Gut∈RL×2=[[1,1,2,2,1,1,1,2]]T. These features are encoded as:

Xu,Gt=f(Gut[:,0])+f(Gut[:,1])∈RL×dG,(3)

where dG is CNI encoding dimension, and f(⋅) shares the same implementation as one-hop co-occurrence encoding (two ReLUs + linear layer).

Unlike the high-order neighbor node sampling based on sliding windows in Reaptmixer [24], we obtain the full set of second-order neighbors and adaptively select the first L2 second-order repeated neighbors before the dataset is truncated. This approach, based on empirical adaptation to the dataset, does not cause a significant increase in computational load, while ensuring that no high-order neighbors are missed and avoiding data loss.

For node classification, we argue that node intrinsic properties significantly impact classification performance. However, prior methods only aggregate node, edge, and time encodings, which is insufficient. We therefore compute statistical features of node encodings, specifically the number of non-zero elements, and encode them via the same method as cross-stage neighbor intersection (CNI) encoding, yielding the node feature statistical encoding Xu,Mt∈RL×dM.

4.2 CVAE and Graph Structure Enhance Hyena for Link Prediction

For link prediction, we concatenate and project the node, edge, neighbor co-occurrence, and cross-stage neighbor intersection (CNI) encodings of time-sorted one-hop neighbors, ordered by interaction time from earliest to most recent, to match the node feature dimension, yielding the fused encoding Xut that integrates structural information with preliminary temporal order cues:

Xut=[Xu,Nt∥Xu,EtXu,Ct∥Xu,Gt]∈RL×(dN+dE+dC+dG),(4)

Xut=XutWfuse+bfuse∈RL×dN(5)

Here, Wfuse and bfuse denote the weight and bias of the fusion linear projection layer; L is the unified one-hop neighbor sequence length defined in Section 4.1 (controlling the maximum temporal dependency range); and dN,dE,dC,dG are the dimensions of the node, edge, co-occurrence, and CNI encodings, respectively. The parameter L adapts to long-sequence efficiency by retaining the most recent L interactions and unifying lengths via padding, enabling efficient modeling of long-term dependencies without the quadratic complexity overhead. This fused encoding retains baseline-effective encodings [6] while incorporating graph-structure-aware CNI encoding.

Unlike the baseline’s direct fusion of time features which neglects temporal-structural conditional correlations, we employ CVAE-based conditional variational inference to model relative time constraints. The relative time encoding Xu,Tt (i.e., the time difference between each neighbor interaction and the current time t) is treated as the conditional input, and Xut serves as the main input containing both structural and temporal information. CVAE models the conditional distribution p(Xut∣Xu,Tt) via a latent variable Z∈Rlatent_dim, where latent_dim (CVAE latent dimension; values listed in Appendix C.3) controls the richness of the temporal–structural joint representation:

Z=μ(Xu,Tt,Xut)+σ(Xu,Tt,Xut)⊙ε,(6)

X^ut=CVAEDecoder(Z,Xu,Tt)(7)

Here, μ(⋅) and σ(⋅) share the same input Lin(Cat(Xu,Tt,Xut)), where Lin denotes a linear transformation and Cat is concatenation. The function σ(⋅) applies an additional Softplus activation to ensure a non-negative standard deviation. The latent dimension latent_dim balances expressive power and generalization (recommended range: latent_dim∈[dN/2,2dN]; excessively small values may cause underfitting, whereas overly large values increase overfitting risks). The noise term ε∼𝒩(0,I) enhances robustness to irregular time intervals, and the CVAEDecoder, implemented as a two-layer feed-forward network (FFN), reconstructs X^ut by binding latent representations with Xu,Tt. Both the CVAE encoder and decoder are implemented as FFN blocks.

We then employ the Hyena layer as the temporal fusion and modeling module to capture fine-grained neighborhood temporal dependencies, targeting both local temporal relevance and long-sequence efficiency. For fair comparison with the baseline, we stack two Hyena layers, feeding separate inputs for u and v to preserve their individual temporal trajectories which is a critical factor for time-sensitive link prediction. The Hyena layer integrates Conv1d, adaptive gating, and nonlinear transformations to model cumulative temporal dependencies with linear complexity. Details of its implementation are provided in Section 4.4. The final time-aware embedding of node u is given by:

Yut=Hyena(X^ut)(8)

Here, Yut and Yvt (for node v) integrate structural information, CVAE-driven conditional temporal constraints, and Hyena-modeled cumulative temporal dependencies, thereby enhancing link prediction by capturing time-sensitive link formation patterns.

4.3 Statistical Encoding Enhance Hyena for Dynamic Node Classification

For dynamic node classification, we omit the CVAE module, as reparameterization-induced “new node generation” is unnecessary for this task and may even adversely affect model performance. Instead, we incorporate relative time encoding directly into the fusion features (without misusing it as a label) and exclude both neighbor co-occurrence encoding and cross-stage neighbor intersection (CNI) encoding unlike link prediction, because these two encodings are likely to introduce noise into the fused representation of a single node. As shown in Appendix D.3, they yield no performance gain for node classification.

We further introduce a node feature statistical encoding, denoted Xu,Mt. Specifically, the fused encoding of node u is defined as:

Xut=[Xu,Nt∥Xu,EtXu,Tt∥Xu,Mt]∈RL×(dN+dE+dT+dM),(9)

Xut=XutWfuse-nodecls+bfuse-nodecls∈RL×dN(10)

Here, Wfuse-nodecls and bfuse-nodecls denote the weight and bias of the fusion projection layer used for node classification, with the subscript “fuse-nodecls” distinguishing them from the corresponding parameters in link prediction (Wfuse and bfuse).

The fused encoding is fed directly into stacked Hyena layers, identical in design to those used in link prediction, as detailed in Section 4.4, to derive time-aware node embeddings for binary classification. The final embedding of node u is given by:

Yut=Hyena(Xut)(11)

Finally, each node’s time-aware embedding Yut is passed to the downstream classification module to produce the final binary classification output (the same process applies to all other nodes).

4.4 Detailed Implementation of Hyena Temporal Fusion and Modeling Layer

As described in Sections 4.2 and 4.3, the neighbor feature sequences are input to the Hyena layer for feature fusion and temporal modeling. The specific architecture and discussion are provided below.

4.4.1 Concise Computation Process

We propose an improved Hyena layer with adaptive convolution selection (based on L) to balance performance and efficiency, retaining two stacked layers, consistent with the baseline, for fair comparison. Given input encodings (X^ut for link prediction and Xut for node classification), we perform time-aligned convolution with adaptive operator selection:

Y^ut={Conv1d(X^ut)L≤256,DSC(X^ut)L>256,(12)

Here, k denotes the convolution kernel size (candidate values listed in Appendix C.3), and DSC refers to depthwise separable Conv1d, comprising depthwise convolution for temporal dependency modeling and pointwise convolution for channel fusion. The subsequent operations are:

Y^ut=RMSNorm(Y^ut),(13)

Gate=Swish(W1Y^ut+b1),(14)

Y^ut=Y^ut⊙Gate,(15)

Yut=ReLU(Y^utW2+b2)W3+b3(16)

Here, W1,b1,W2,b2,W3,b3 are trainable parameters, and dhidden (the FFN hidden dimension) is specified in Appendix C.3. Stacking two layers extends the effective temporal dependency range to 2k (linear complexity), while separate inputs for u and v preserve their unique temporal trajectories, which is critical for time-sensitive dynamic graph tasks.

4.4.2 Adaptive Convolution Selection for Datasets with Varying L

To accommodate datasets with differing time-sorted neighborhood sequence lengths L, we explicitly provide two targeted convolution selection schemes in the improved Hyena layer. For short sequences (L≤256), standard Conv1d is adopted, as it is suitable for scenarios where local temporal dependencies are concentrated; under a fixed dN, its computational overhead is manageable, enabling integration of temporal correlations and channel interactions without significant efficiency loss. For long sequences (L>256), depthwise separable Conv1d (DSC) is selected, designed to control computational scale for long sequences (especially ultra-long sequences such as L=2048). Its split design isolates temporal dependency propagation from channel fusion, mitigating long-sequence feature drift while retaining modeling accuracy.

4.4.3 Computational Complexity Analysis

Both variants of the improved Hyena layer maintain linear complexity O(L) (scaling with sequence length L), in contrast to the baseline Transformer’s quadratic O(L2). For L≤256, the Conv1d-based Hyena outperforms the Transformer by avoiding redundant self-attention computations. For L>256, the DSC-based Hyena further improves efficiency by eliminating channel–temporal coupling redundancy, thereby avoiding the computational explosion seen in Transformers for ultra-long sequences. These results confirm that the adaptive scheme provides superior efficiency compared to the baseline across varying sequence lengths, while preserving temporal modeling accuracy.

5  Experiments

In this section, we perform experiments on DyG-Hyena to demonstrate its superior performance and analyze the role of each component in the Hyena layer.

5.1 Experimental Settings

5.1.1 Datasets and Baselines

We adopt 12 mainstream datasets for continuous-time dynamic graph (CTDG) learning, covering social and communication network scenarios. Seven datasets are reported in the main text, while the other five are placed only in the appendix. These excluded datasets contain massive simultaneous timestamps and coarse-grained temporal features, which are incompatible with the fine-grained modeling paradigm of general CTDG frameworks. All datasets are split into training, validation and test sets at a ratio of 70%/15%/15%, with detailed statistics illustrated in Appendix C.1. For fair comparison, we combine classic and recent SOTA baselines. Classical methods cover RNN-based models (JODIE, DyRep, TGN, CAWN), memory-based EdgeBank, sampling aggregation-based methods (TGAT, TCL, GraphMixer), and Transformer-based DyGFormer. Recent SOTA competitors include FreeDyG, DyG-Mamba, TPNet, RepeatMixer and ScaDyG. We conduct comparisons even if partial SOTA methods lack results on individual datasets, and the detailed baseline introductions are presented in Appendix C.2.

5.1.2 Evaluation Configurations and Metrics

Consistent with most prior continuous-time dynamic graph learning works, we adopt the unified library and experimental setup from [6] for downstream task evaluation and baseline reproduction. For link prediction, we use two evaluation settings: transductive and inductive. The transductive setting focuses on future link prediction for nodes observed during training, while the inductive setting targets future link prediction for nodes unseen during training. We use Average Precision (AP) and ROC-Area Under the Curve (ROC-AUC) as evaluation metrics. Following [6,7], we employ three negative sampling strategies: random (rnd), historical (hist), and inductive (ind), where each strategy generates distinct types of negative samples to comprehensively evaluate model performance. For node classification, we use ROC-AUC to assess classification performance.

5.1.3 Experimental Details

We perform experiments using the unified library Dyclib [6], ensuring fair comparison against baselines under a consistent evaluation framework. Consistent with prior works, DyG-Hyena is trained with the Adam optimizer, using a learning rate of 10−4, a maximum of 100 epochs, and a batch size of 200. We run five independent training trials with random seeds ranging from 0 to 4 to mitigate potential bias. An early stopping mechanism is employed for each trial with a patience of 20. Detailed implementation and evaluation configurations are provided in Appendix C.3.

5.2 Main Performance Comparison

5.2.1 Performance of Downstream Tasks

We report the Average Precision (AP) scores for transductive dynamic link prediction under the random negative sampling strategy in Table 1, and the AP scores for inductive dynamic link prediction under the same sampling strategy in Table 2, all based on 7 commonly used datasets for CTDG baselines. For baseline results, we present the outcomes of our reproduced FreeDyG [6], DyGFormer [6], and TPNet [26], as well as other previously reported baseline results; our reproduced results exhibit negligible differences from those reported in the original papers. It is worth noting that we cannot reproduce the results of ScaDyG [13], DyG-Mamba [10,29], and RepeatMixer [24], as their code is either unpublished or incomplete. Thus, we directly adopt the results from their original papers, which are obtained under the same experimental benchmarks as ours. We note that the DyG-Mamba results cited herein are from [29], given the consistency between the results reported in the two referenced works. As documented in their studies, the dataset splitting and negative sampling strategy settings are consistent with ours.

Additional results including those for historical and inductive negative sampling (both transductive and inductive settings), ROC-AUC scores for transductive/inductive dynamic link prediction, and results on an additional 5 commonly used datasets for CTDG baselines which are provided in Appendices D.1 and D.2. These 5 datasets are not reported in the main text as they lack continuous partitioning, contain numerous simultaneous interactions, and are thus unsuitable for CTDG evaluation; we include their results to align with baselines. Since EdgeBank is only applicable to the transductive setting, and DyG-Mamba, FreeDyG, and ScaDyG are only implemented on partial datasets, their full performance is not reported herein, and “N/A” is marked where results are unavailable.

We also present DyG-Hyena’s performance on dynamic node classification in Table 3, where it achieves the second-best performance on the Wikipedia and Reddit datasets.

5.2.2 Performance Analysis

Based on the above results, DyG-Hyena delivers overall superior performance on CTDG-adapted fine-grained datasets.

First, for dynamic link prediction, DyG-Hyena ranks first on five and six datasets respectively in transductive and inductive settings, and ranks second best on the remaining datasets. Meanwhile, it maintains strong performance under other negative sampling strategies and the remaining five datasets presented in the appendix, with an overall average ranking that achieves either the best or the second best. For dynamic node classification, it ranks first or second on two labeled datasets and achieves the first average ranking. This superiority stems from three key factors: (i) Adopting the Hyena-based temporal fusion and modeling layer for time-sorted neighborhood feature sequences inherently ensures strong temporal modeling capability and thus superior performance; (ii) The CVAE module effectively models the dependencies between conditional inputs (temporal encodings) and primary inputs (fused encodings), enhancing the utilization of temporal information; (iii) The proposed cross-stage neighbor intersection (CNI) encoding enriches the utilization of graph structural information.

Second, DyG-Hyena possesses robust generalization ability. In link prediction, it achieves the same performance ranking in the inductive setting (for unseen nodes during training) as in the transductive setting (for seen nodes), and exhibits stable performance across datasets with varying temporal granularities. This strong generalization can be attributed to: (i) The CVAE module generates diverse potential link states/relationships via the reparameterization trick, enabling the model to generalize better to unseen data; (ii) The one-hop full-neighbor truncation-padded aggregation framework facilitates the capture of richer neighborhood information. We quantitatively analyze DyG-Hyena’s generalization ability in Appendix D.4.

Notably, DyG-Hyena fails to achieve top-two performance on some of the five additional datasets in the appendix (e.g., USLegis, UNTrade), as detailed in Appendices D.1 and D.2. This stems from incompatibility with the proposed continuous-time dynamic graph (CTDG) framework: these datasets contain numerous simultaneous interactions (i.e., same timestamp), while the CTDG framework processes them sequentially. We include these datasets for baseline consistency and provide a detailed analysis of this limitation in Appendix A.2.

Subsequently, we elaborate on the advantages of each key module in DyG-Hyena.

5.3 Ablation Study

We verify the effectiveness of three key components in DyG-Hyena through ablation experiments: the Hyena layer, CVAE-based conditional modeling of the temporal dimension, and the proposed enhanced information encodings. Specifically, we perform ablation experiments for link prediction on the MOOC and LastFM datasets, and for dynamic node classification on the Wikipedia and Reddit datasets; the corresponding results are visualized as bar charts in Fig. 5. Additional ablation experiments with more datasets and comprehensive analyses are provided in Appendix D.3. The ablation results demonstrate that removing any component of DyG-Hyena leads to noticeable performance degradation, verifying that the Hyena layer inherently delivers strong performance and all proposed enhanced encodings contribute effectively to the model’s overall capability.

Figure 5: Illustration of the ablation experiment.

5.4 Advantages of Using Hyena

We replaced the Transformer with the Hyena layer as the temporal fusion and modeling layer for time-sorted neighborhood feature sequences, reducing the computational load of temporal modeling and feature extraction while preserving more sequence information. To fairly validate the advantages of DyG-Hyena, we conducted two key comparisons:

First, we compared DyG-Hyena (and its variants) with DyGFormer under identical settings (e.g., no concatenation technique for DyGFormer). Evaluations across multiple datasets and varying numbers of neighboring nodes (64–2048, using the Can.Parl dataset to avoid GPU memory constraints) show that DyG-Hyena achieves significantly lower GPU memory usage and shorter training time than DyGFormer, even with our enhancement modules, the resulting increase in training time is acceptable (see Appendices D.3 and D.5).

Second, we compared different candidate layers for the temporal fusion and modeling module within the DyG-Hyena framework (retaining all our information enhancement and CVAE components, distinct from the technical routes of DyG-Mamba, DyGFormer, etc.). The compared layers include Mamba [45], Transformer, and RWKV [46] (baselines introduced in Chapter 2), with results shown in Table 4. Specifically: RWKV achieves the highest computational efficiency at the cost of obvious accuracy loss. Transformer provides stable performance but brings exponentially rising computational overhead with longer sequences. Both Mamba and Hyena are lightweight and perform well in general. Hyena maintains competitive overall results and exhibits unique strengths on short-sequence datasets. Benefiting from the convolution-gating design, Hyena mitigates gradient attenuation and discretization errors, making it well-suited for dynamic link prediction with local neighbor dependence and short-range sequence modeling.

5.5 The Role of CVAE

In this section, we explored the effects of time information enhancement on link prediction. To verify the advantages of adding a CVAE layer for time modeling (DyG-Hyena (CVAE)) over the temporal embedding concatenation mechanism (DyG-Hyena (TE)), improving convergence speed and transductive AP, we compared the two models on 4 datasets. We defined convergence as test performance reaching 99% of the optimal value. Table 5 records their convergence rounds and single-epoch time. Results show DyG-Hyena (CVAE) converges faster with acceptable single-epoch time increase, and outperforms DyG-Hyena (TE) in transductive AP, benefiting from CVAE’s probabilistic framework for precise integration of temporal and structural information. Ablation experiments (Section 5.3) and generalization evaluations (Appendix D.4) further confirm the CVAE layer enhances temporal modeling, boosts transductive AP, and improves generalization on unseen data.

6  Conclusion

This paper proposes a dynamic graph learning model based on efficient convolution, namely DyG-Hyena. Our method adopts the Hyena layer as the temporal modeling backbone for sequential features to capture temporal dependencies of node/edge interaction sequences, which significantly reduces computational complexity. We further design task-specific information enhancement strategies: for dynamic link prediction, we propose CVAE-based temporal information enhancement and graph structure-aware cross-stage neighbor intersection (CNI) encoding to enrich task-relevant information; for dynamic node classification, we introduce node feature statistical encoding to enhance the modeling of node intrinsic properties. The effectiveness of DyG-Hyena is validated through extensive experiments on multiple datasets. In future work, we plan to further enhance DyG-Hyena to better handle dynamic graph datasets with varying temporal granularities and explore additional information enhancement modules to further boost its performance.

Acknowledgement: The authors would like to thank the Key R&D Special Project of Henan Province (Grant No. 231111211000) for funding support.

Funding Statement: This work was supported by the Key R&D Special Project of Henan Province (Grant No. 231111211000).

Author Contributions: The authors confirm contribution to the paper as follows: Conceptualization, Suchang Yang, Hongtao Yu and Ruiyang Huang; methodology, Suchang Yang, Hongtao Yu and Ruiyang Huang; software, Suchang Yang; validation, Suchang Yang, Hongtao Yu, Ruiyang Huang, Huansha Wang and Junzheng Li; formal analysis, Suchang Yang and Junzheng Li; investigation, Suchang Yang and Junzheng Li; resources, Hongtao Yu; data curation, Suchang Yang, Ran Li and Junzheng Li; writing—original draft preparation, Suchang Yang; writing—review and editing, Suchang Yang, Hongtao Yu, Ruiyang Huang and Ran Li; visualization, Huansha Wang; supervision, Hongtao Yu and Ruiyang Huang; project administration, Hongtao Yu; funding acquisition, Hongtao Yu. All authors reviewed and approved the final version of the manuscript.

Availability of Data and Materials: Data openly available in a public repository.

Ethics Approval: Not applicable

Conflicts of Interest: The authors declare no conflicts of interest.

Appendix A Additional Theoretical Details Discussion

Appendix A.1 Important Symbol Definitions

In this section, we define the key symbols used in this paper to facilitate better understanding, as summarized in Table A1.

Appendix A.2 Limitations

The limitations of our approach mainly lie in the following aspects: (i) We fail to achieve breakthrough performance on coarse-grained datasets, as most interactions in these datasets are concentrated at identical timestamps. This contradicts the sequential interaction learning paradigm in continuous-time dynamic graph (CTDG) learning, which inherently assumes ordered learning of interaction sequences based on temporal order, yet interactions sharing the same timestamp do not follow the model’s predefined sequential order. (ii) After introducing the cross-stage neighbor intersection (CNI) encoding, the model’s computational complexity increases noticeably, which conflicts with the core goal of reducing computational overhead. Although we adopted a neighbor sampling strategy to mitigate this issue, it may lead to the loss of certain useful information. (iii) We did not incorporate interaction-related features for node classification, yet this does not imply that interaction information is irrelevant to node-level tasks. We believe that with rational design, interaction information can contribute effectively to node-level modeling, and this will be a focus of our future research.

Appendix A.3 Intuitive Reasons for Advantages

The intuitive reasons behind DyG-Hyena’s advantages are as follows: (i) We adopt the Hyena layer for temporal context modeling, which significantly reduces the model’s computational overhead compared to Transformer-based layers (see Section 5.4). (ii) For dynamic link prediction, CVAE-based conditional modeling of relative temporal information and graph structure-aware CNI encoding enhance the input information fed into the Hyena layer from the temporal and graph structural dimensions, respectively. For dynamic node classification, while we reduce the input of inter-node interaction information, we incorporate node feature statistical encoding into the input, which strengthens the model’s ability to learn node representations.

Appendix B Theoretical Derivation

In this section, we derive key formulas for the CVAE and Hyena layers, and analyze their theoretical properties.

Appendix B.1 Enhancing Effect of CVAE

Let Xut∈RL×din denote the fused feature sequence of one-hop time-sorted neighbors of node u at timestamp t, Cut=Xu,Tt denote the relative time encoding unique to CTDG, y denote the label of dynamic edge (u,v,t), and z∈Rdz denote the latent variable characterizing temporal-structural dependencies. The evidence lower bound (ELBO) for dynamic link prediction is formulated as:

ℒELBO=Ez∼qφ(z|Xut,Cut)[log⁡pθ(y|z,Xut,Cut)]−DKL(qφ(z|Xut,Cut)∥p(z)).(A1)

The reparameterization trick conditioned on CTDG temporal constraints is written as:

z=μφ(Xut,Cut)+σφ(Xut,Cut)⊙ε,ε∼𝒩(0,I).(A2)

Let ∇φμφ and ∇φσφ denote the gradients of encoder parameters. The gradient of the expected log-likelihood satisfies:

∇φEε[log⁡pθ(y|μφ+σφ⊙ε,Xut,Cut)]=Eε[∇zlog⁡pθ(y|z,Xut,Cut)⋅(∇φμφ+ε⊙∇φσφ)].(A3)

The expectation of (∇φμφ+ε⊙∇φσφ) is uniformly bounded, yielding a Monte Carlo gradient variance of 𝒪(1/K), which reduces the iteration complexity to 𝒪(log⁡(1/ε)) and accelerates convergence on irregular CTDG interaction sequences.

For generalization, let m be the number of synthetic latent samples:

z(j)=μ+ασ⊙ε(j),ε(j)∼𝒩(0,I), j=1,…,m.(A4)

Decoding z(j) generates m temporally consistent synthetic edges that expand the support of the CTDG training distribution. The generalization gap satisfies:

GenGap∝model complexityn(1+m),(A5)

where n denotes the number of original observed edges. Increasing m tightens the gap and improves generalization to unseen nodes and timestamps.

Appendix B.2 Implementation and Formulation of Hyena

We detail Hyena’s implementation for dynamic graph learning and its differences from the standard version. Building on the CVAE layer (modeling fused features with enhanced temporal information), the Hyena layer acts as a fused feature extractor to capture intra- and inter-sequence temporal dependencies. The standard Hyena [31] matches or exceeds Transformers via alternating implicit long convolutions and gating, formulated as:

y=H⋅v,(A6)

where v is the input sequence, y is the output, and H is a data- and kernel-parameterized matrix. It projects X∈RL×din (L: unified sequence length; din: input dimension) to (U+1)⋅din, splitting into U gating sequences for implicit convolution/gating. Thus:

H=DNSN∗⋯∗D2S2∗D1S1,(A7)

where DN is the gating matrix and SN is a Toeplitz matrix from an implicit parameterized kernel.

To reduce complexity and overhead for dynamic link prediction, we retain Hyena’s convolutional gating while simplifying with Conv1d. The Hyena layer computation is:

Yut=H(Xut),(A8)

where Yut is node u’s final embedding at t, and Xut is the fused input. During batch training (X^∈RB×L×din, B: batch size), we transpose X^ to X^′∈RB×din×L (Conv1d’s batch × channel × sequence format) and apply Conv1d:

X=Conv1d(X^′).(A9)

Here, X^b,c,s′ denotes the b-th sample’s c-th channel (embedding dimension) at s-th time step; kernel K∈Rdout×din×ksize (dout: output dimension; ksize: kernel size) maps input channel i to output o via Ko,i,k. The Conv1d output Y∈RB×dout×L is:

Yb,o,s=∑i=1din∑k=1ksizeKo,i,k⋅X^b,i,s+k−⌊ksize/2⌋′+bo,(A10)

where bo is the output channel o’s bias. Hyena’s implementation references open-source repositories1.

Appendix B.3 Derivation of Convergence Rate

Appendix B.3.1 Convergence Rate of DyG-Hyena

The total loss of DyG-Hyena for CTDG learning is:

ℒ(θ)=Eqϕ(z|Xut,Cut)‖Xut−fθ(z,Cut)‖22+βDKL(qϕ(z|Xut,Cut)∥p(z))+λ‖h‖22,(A11)

where fθ denotes the Hyena-based decoder.

Under the Polyak-Łojasiewicz (PL) condition ‖∇ℒ(θ)‖22≥2μ(ℒ(θ)−ℒ∗) and Lipschitz smoothness ‖∇ℒ(θ1)−∇ℒ(θ2)‖2≤L‖θ1−θ2‖2, the convergence rate with step size η=1/L is bounded by:

ℒ(θt)−ℒ∗≤(1−μL)t(ℒ(θ0)−ℒ∗),(A12)

which indicates exponential convergence on CTDG data.

Appendix B.3.2 Acceleration Effect of CVAE on Convergence

With annealed KL divergence, the effective loss is:

ℒ~(θ)=Eqϕ‖Xut−fθ(z,Cut)‖22+λ‖h‖22.(A13)

The maximum eigenvalue of the Hessian matrix satisfies:

λmax(∇2ℒ~(θ))≤LHyena+σz2B,(A14)

where σz2 denotes the variance of latent variable z and B denotes batch size.

The expected gradient norm decays as:

E[‖∇ℒ~(θt)‖2]≤ρt‖∇ℒ~(θ0)‖2,(A15)

where ρ=1−μLHyena+σz2/B<1−μL. Thus, the CVAE module accelerates convergence for irregular CTDG sequences.

Appendix C Detailed Experimental Settings

Appendix C.1 Datesets

We used 12 widely adopted continuous-time dynamic graph datasets [6,7,10,12],which were first curated and released by EdgeBank [7] and are publicly available2:

•   Wikipedia [4] is a bipartite interaction graph containing Wikipedia page edits over one month. Nodes represent encyclopedia users and pages, links denote users’ edits to pages at specific timestamps, and the features of each link correspond to the 172-dimensional Linguistic Inquiry and Word Count (LIWC) framework [47]. This dataset additionally includes node dynamic labels indicating whether a user is prohibited from editing.

•   Reddit [4] is a bipartite graph recording a month’s worth of user posts on subreddits. Nodes are divided into users and subreddits, and links are time-stamped post submissions. Like Wikipedia, each link in Reddit contains a 172-dimensional LIWC feature, and the dataset also includes node dynamic labels.

•   MOOC [4] is a bipartite interaction network comprising online educational interactions. Nodes are students and course resources, links are records of students’ access to course resources, and each link contains a four-dimensional feature.

•   LastFM [4] is a bipartite interaction network containing records of users’ music listening behavior over one month. Nodes are users and songs, and links represent users’ listening activities.

•   Enron [48] is a 3-year dataset of ENRON employee communications. Nodes represent employees, and links denote communication interactions.

•   Social Evo. [49] is a mobile phone proximity network consisting of eight months of daily communication activities among members of an undergraduate dormitory group. Nodes represent undergraduate users, links represent communication processes, and each link has a two-dimensional feature.

•   UCI [50] is an online communication network for college students. Nodes are college students, and links represent communication processes.

•   Flights [51] is a dynamic flight network recording the evolution of air traffic over a period of time. Nodes are airports, and links represent time-stamped flights. Each link has a feature representing the cumulative number of flights between the two airports by the current timestamp that day.

•   Can. Parl. [7] is a Canadian Parliamentarian Interactive network documenting the interactions of Canadian Parliamentarians from 2006 to 2019. Each node represents a member of Parliament from a constituency, links represent two members voting together on a motion, and the weight of each link denotes the cumulative number of times one member has voted in alignment with the other to date.

•   US Legis. [7] is a social network for members of the United States Senate. Nodes represent senators, links denote their social interaction trajectories, and link weights represent the number of bills co-sponsored between members.

•   UN Trade [7] is a record of food and agricultural trade between 181 countries over 30 years. Nodes represent countries, links denote trade between countries, and their weights represent the sum of standardized agricultural import and export values between two specific countries.

•   Contact [7] documents the evolution of physical proximity among approximately 700 college students. Each node represents a student, links indicate that they are in close physical proximity to each other, and link weights quantify the degree of closeness.

We present the statistics of the above datasets in Table A2, where #N&L Feat denotes the feature dimensions of nodes or links. Consistent with our experimental setup, we adopt the statistical data reported in [6].

Appendix C.2 Baselines

The twelve baselines we selected are described as follows:

•   JODIE [4] is an RNN-based model designed for user-item bipartite interaction graphs. It updates node representations via two coupled RNNs and predicts future node representations through a projection layer.

•   DyRep [41] is built on an RNN architecture, updating node states sequentially. It also introduces a time-parameterized attention module to aggregate temporal information.

•   TGAT [17] was the first to introduce the self-attention mechanism into dynamic graph learning. It adopts a one-hop neighbor sampling and aggregation framework, taking node, edge, and time-encoded features of sampled one-hop neighbors as inputs to the self-attention layer, and iteratively computes attention-based node embeddings.

•   TGN [20, 22] is a memory-based model that maintains a memory buffer for each node to represent its current state. During message passing, each node’s memory is updated interactively via message functions, message aggregators, and a memory updater.

•   CAWN [21] is also based on RNN. First, it extracts multiple causal anonymous walks (CAWs) for each node and generates relative node identifiers. It encodes these identifiers using an RNN and aggregates the encodings to obtain the final node representations.

•   EdgeBank [7] is a memory-based heuristic method that stores observed edges in memory. If the queried node pair exists in the memory, it predicts a positive link; otherwise, it predicts a negative link. Thus, it is only applicable to transductive dynamic link prediction.

•   TCL [23] obtains sequences of one-hop neighbor interactions via a short-term sampling framework, following a breadth-first extraction logic. It then uses a Transformer to aggregate graph topological and temporal information for node representation learning, and introduces a cross-attention mechanism to model interaction relationships.

•   GraphMixer [19] adopts a short-term sampling framework to obtain one-hop neighbor information. It constructs node encoders, link encoders, and link predictors, all based on a simple MLP-Mixer. It has also demonstrated the effectiveness of parameter-fixed temporal encodings.

•   DyGFormer [6] is a Transformer-based dynamic graph learning model. It aggregates node, edge, temporal, and neighbor co-occurrence information of one-hop neighbors and concatenates them into a sequence. To capture temporal dependencies of the sequence using a Transformer while mitigating the model’s inherent computational overhead, patching technology is adopted to shorten the sequence length.

•   DyG-Mamba [10,29] is a SOTA solution that balances performance and efficiency. It replaces the sequence’s time-dependent learning layer with the SSM-based Mamba model, using time differences as control signals for the SSM. This effectively improves the model’s utilization of temporal information and achieves strong performance.

•   TPNet [26] is a temporal GNN for dynamic link prediction. It unifies relative encoding as temporal walk matrix functions (with time-decayed design to capture structural-temporal dependencies), adopts random feature propagation instead of explicit neighbor sampling, and supports efficient, generalizable transductive/inductive learning.

•   FreeDyG [12] is a recent state-of-the-art (SOTA) method. It extends neighbor co-occurrence encoding to Node Interaction Frequency (NIF) Encoding and uses a frequency-enhanced MLP-Mixer layer to aggregate temporal embeddings. It adopts a short-term dependency sampling framework to obtain neighbor sequences.

•   RepeatMixer [24] is a dynamic graph model for link prediction, solving traditional methods’ oversight of historical repeat interactions (focusing only on recent neighbors). It adopts repeat-aware neighbor sampling (defining first-order/high-order repeat nodes, selecting neighbors via sliding window), uses an MLP encoder for feature-temporal fusion, and a time-aware aggregation mechanism (Pearson Correlation Coefficient weighting).

•   ScaDyG [13] is a new paradigm for large-scale dynamic graph learning. It achieves efficient and effective dynamic graph representation learning via three core operations: time-aware topology redefinition, dynamic temporal encoding, and hypernetwork-driven message aggregation. Dataset reconstruction requires specific preprocessing steps.

Appendix C.3 Detailed Experimental Configuration

Appendix C.3.1 Model Configurations

We adopt the baseline model configurations from DyGLib [6], and our reproduced results are consistent with those reported in DyGLib. Thus, we directly use their baseline experimental results for comparison. For DyG-Hyena, we adopt the experimental model configuration as shown in Table A3.

Appendix C.3.2 Hyperparameter Settings

In Tables A4 and A5, we report the hyperparameter settings of the Hyena layer across different datasets and tasks. It is worth noting that the unified sequence length L is determined via grid search. We adopt a one-hop full-neighbor aggregation framework, where the aggregation length refers to the number of neighbors selected for aggregation. We perform grid search over the range [32, 2048] and select the optimal value for each dataset. We also report the unified two-hop neighbor sequence length L2, which is used for constructing cross-stage neighbor intersection (CNI) encoding. Since enumerating all two-hop neighbors incurs substantial computational overhead, we adopt a neighbor sampling strategy.

Appendix C.3.3 Negative Sampling Strategies in Link Prediction

We implement three negative sampling strategies for link prediction: random, historical, and inductive negative sampling, following the settings in [6]. The dataset is split into three subsets for training, validation, and testing, with each subset corresponding to a distinct time period. Random negative sampling involves randomly selecting node pairs from all possible non-interacting node pairs to form negative samples. Historical negative sampling constructs negative samples using edges that existed before the current step (training/validation/testing) but not within the current step. Inductive negative sampling selects edges that have never appeared prior to the current step to form negative samples.

Appendix C.3.4 Experimental Environment and Memory Usage

Our experiments are conducted on an Ubuntu server equipped with an Intel(R) Xeon(R) Gold 6330 CPU (2.00 GHz) and 112 physical cores. The GPU device is an NVIDIA A800-SXM4-80GB. Our code is implemented in Python 3.12, with model training conducted using PyTorch 2.6.0 on CUDA 12.4. Given the provided hyperparameters (primarily the sequence length L), the GPU memory usage for all datasets does not exceed 80 GB. We present the maximum memory usage statistics for DyG-Hyena during training in Table A6.

Appendix D Detailed Results of Experiments

Appendix D.1 Additional Results for Transductive Dynamic Link Prediction

We report AP scores for transductive dynamic link prediction under random, historical, and inductive negative sampling strategies in Table A12, and ROC-AUC scores for the same setting in Table A13.

Appendix D.2 Additional Results for Inductive Dynamic Link Prediction

We report AP scores for inductive dynamic link prediction under random, historical, and inductive negative sampling strategies in Table A14, and ROC-AUC scores for the same setting in Table A15.

Appendix D.3 Ablation Study

To evaluate the effectiveness of each component in DyG-Hyena across different tasks, we perform ablation experiments on model variants for link prediction and node classification separately. For dynamic link prediction, we design the following variants: (i) DyGFormer w/o patching: As a baseline for fair comparison, we use DyGFormer without adopting its patching technique. (ii) DyG-Hyena: Our proposed full model. (iii) DyG-Hyena w/o CVAE: In dynamic link prediction, relative time features are directly concatenated with other features, bypassing the CVAE layer. (iv) DyG-Hyena w/o CNI: The cross-stage neighbor intersection (CNI) encoding is removed from the fused input. (v) Only Hyena: Both the CVAE layer and CNI encoding are removed to independently evaluate the performance of the Hyena layer. We present ablation results for as many datasets as possible. In addition to performance metrics, we also report epoch time, convergence iterations (to 99% of the final performance), and GPU memory usage. The results are presented in Table A9.

For dynamic node classification, we design the following variants: (i) DyG-Hyena w/o SCN: Without node feature statistical encoding; (ii) DyG-Hyena w/AIF: Following a similar logic to link prediction, we add neighbor co-occurrence encoding and CNI encoding to the fused features. We conduct experiments on the Wikipedia and Reddit datasets, reporting ROC-AUC scores. These results are presented in the main text.

The ablation results indicate that the Hyena layer contributes significantly to efficiency improvement. While the introduction of the CNI and CVAE modules increases per-epoch time cost, they reduce overall convergence iterations and enhance model performance.

Appendix D.4 Evaluation of Generalization Ability

We experimentally demonstrate that DyG-Hyena possesses strong generalization ability. Specifically, we quantitatively evaluate its generalization from two complementary dimensions to avoid the limitations of a single metric. (1) We measure the AP difference between transductive and inductive settings, defined as ΔAPt,i=|APtran−APind|, which quantifies the performance degradation on unseen nodes. A smaller value indicates better generalization to out-of-distribution nodes. All experiments adopt a unified random negative sampling strategy, and the results of four representative SOTA models are presented in Table A7. (2) We evaluate the generalization across diverse temporal granularities. Datasets are divided into fine-grained (Unix timestamps: 7 datasets) and coarse-grained (year/month/day: 5 datasets), and the average AP ranking of each model is calculated for both categories. Steady performance across both categories demonstrates robust cross-granularity generalization, with results shown in Table A8.

From the results in Tables A7 and A8, DyG-Hyena achieves competitive overall performance in terms of generalization ability. Specifically: (i) The one-hop neighbor truncation-padded aggregation framework endows the model with the ability to capture rich neighborhood information, which lays the foundation for generalization to unseen nodes and datasets with different temporal granularities; (ii) The CVAE layer integrated in DyG-Hyena generates latent variables via the reparameterization trick and reconstructs feature representations, which promotes the generation of diverse potential link states. This design further enhances the model’s generalization capability when processing unseen data.

Appendix D.5 Comparison Results of Training Time and Memory

Fig. A1 shows the training time and memory usage of Hynea alone and DyGFormer w/o patching in the Can. Parl.

Figure A1: Illustration of the ablation experiment.

Appendix D.6 Additional Hardware Configuration, Benchmarks, Metrics

To enable standardized comparison with the baseline, this section reduces unfair comparisons arising from inconsistent conditions or assessment methods via experiments. Two sets of comparative experiments were conducted: (i) To mitigate unfairness caused by superior hardware in the main text, we re-evaluated the AP of transductive dynamic link prediction on four datasets (Wikipedia, Reddit, MOOC, UCI) using a machine with 56-core Intel(R) Xeon(R) Gold 6132 CPU @ 2.60 GHz and NVIDIA GeForce RTX 3090 24 G as shown in Table 1. (ii) We compared DyG-Hyena with baseline DyGFormer under more extensive datasets (tgbl-wiki [4, 52, 53], tgbl-coin(>600 K nodes, >20 million edges) [52, 53]) and stricter indicators (MRR) [52, 53] as shown in Table 2. Results show hardware conditions do not significantly improve performance, while broader datasets and stricter indicators confirm DyG-Hyena’s competitiveness.

1https://github.com/Suro-One/Hyena-Hierarchy, https://github.com/HazyResearch/safari

2https://zenodo.org/records/dynamic-graphs

References

1. Kazemi SM, Goel R, Jain K, Kobyzev I, Sethi A, Forsyth P, et al. Representation learning for dynamic graphs: A survey. J Mach Learn Res. 2020;21(70):1–73. [Google Scholar]

2. Yang C. Dynamic graph representation learning with neural networks: A survey. IEEE Access. 2024;12(70):43460–84. doi:10.1109/access.2024.3378111. [Google Scholar] [CrossRef]

3. Song W, Xiao Z, Wang Y, Charlin L, Zhang M, Tang J. Session-based social recommendation via dynamic graph attention networks. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining; 2019 Feb 11–15; Melbourne, Australia. p. 555–63. [Google Scholar]

4. Kumar S, Zhang X, Leskovec J. Predicting dynamic embedding trajectory in temporal interaction networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining; 2019 Aug 4–8; Anchorage, AK, USA. p. 1269–78. [Google Scholar]

5. Hui Y, Wang S, Chekol MW, Rudinac S, Zwetsloot IM. Non-progressive influence maximization in dynamic social networks. arXiv:2412.07402. 2024. [Google Scholar]

6. Yu L, Sun L, Du B, Lv W. Towards better dynamic graph learning: new architecture and unified library. Adv Neural Inf Process Syst. 2023;36:67686–700. [Google Scholar]

7. Poursafaei F, Huang S, Pelrine K, Rabbany R. Towards better evaluation for dynamic link prediction. Adv Neural Inf Process Syst. 2022;35:32928–41. doi:10.52202/068431-2386. [Google Scholar] [CrossRef]

8. Rozemberczki B, Scherer P, He Y, Panagopoulos G, Riedel A, Astefanoaei M, et al. Pytorch geometric temporal: spatiotemporal signal processing with neural machine learning models. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management; 2021 Nov 1–5; Online. p. 4564–73. [Google Scholar]

9. Zhou H, Zheng D, Nisa I, Ioannidis V, Song X, Karypis G. TGL: a general framework for temporal GNN training on billion-scale graphs. arXiv:2203.14883. 2022. [Google Scholar]

10. Li D, Tan S, Zhang Y, Jin M, Pan S, Okumura M, et al. DyG-mamba: continuous state space modeling on dynamic graphs. arXiv:2408.06966. 2024. [Google Scholar]

11. Zhang J, Chen J, Yang M, Feng A, Liang S, Shao S, et al. DTGB: a comprehensive benchmark for dynamic text-attributed graphs. arXiv:2406.12072. 2024. [Google Scholar]

12. Tian Y, Qi Y, Guo F. Frequency enhanced continuous-time dynamic graph model for link prediction. In: Proceedings of the Twelfth International Conference on Learning Representations; 2024 May 7–11; Vienna, Austria. [Google Scholar]

13. Wu X, Li X, Li RH, Zhao K, Wang G. ScaDyG: a new paradigm for large-scale dynamic graph learning. arXiv:2501.16002. 2025. [Google Scholar]

14. Karmim Y, Lafon M, Fournier-S’Niehotta R, Thome N. Supra-laplacian encoding for transformer on dynamic graphs. Adv Neural Inf Process Syst. 2024;37:17215–46. doi:10.52202/079017-0547. [Google Scholar] [CrossRef]

15. Chen C, Geng H, Yang N, Yang X, Yan J. EasyDGL: encode, train and interpret for continuous-time dynamic graph learning. IEEE Trans Pattern Anal Mach Intell. 2024;46(12):10845–62. doi:10.1109/tpami.2024.3443110. [Google Scholar] [PubMed] [CrossRef]

16. Sun J, Gu M, Yeh CCM, Fan Y, Chowdhary G, Zhang W. Dynamic graph node classification via time augmentation. In: Proceedings of the 2022 IEEE International Conference on Big Data (Big Data); 2022 Dec 17–20; Osaka, Japan. p. 800–5. [Google Scholar]

17. Xu D, Ruan C, Korpeoglu E, Kumar S, Achan K. Inductive representation learning on temporal graphs. arXiv:2002.07962. 2020. [Google Scholar]

18. Yu X, Liu Z, Zhang X, Fang Y. Node-time conditional prompt learning in dynamic graphs. arXiv:2405.13937. 2024. [Google Scholar]

19. Cong W, Zhang S, Kang J, Yuan B, Wu H, Zhou X, et al. Do we really need complicated model architectures for temporal networks? arXiv:2302.11636. 2023. [Google Scholar]

20. Rossi E, Chamberlain B, Frasca F, Eynard D, Monti F, Bronstein M. Temporal graph networks for deep learning on dynamic graphs. arXiv:2006.10637.2020. [Google Scholar]

21. Wang Y, Chang YY, Liu Y, Leskovec J, Li P. Inductive representation learning in temporal networks via causal anonymous walks. arXiv:2101.05974. 2021. [Google Scholar]

22. Souza A, Mesquita D, Kaski S, Garg V. Provably expressive temporal graph networks. Adv Neural Inf Process Syst. 2022;35:32257–69. doi:10.52202/068431-2337. [Google Scholar] [CrossRef]

23. Wang L, Chang X, Li S, Chu Y, Li H, Zhang W, et al. TCL: transformer-based dynamic graph modelling via contrastive learning. arXiv:2105.07944. 2021. [Google Scholar]

24. Zou T, Mao Y, Ye J, Du B. Repeat-aware neighbor sampling for dynamic graph learning. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2024 Aug 25–29; Barcelona, Spain. p. 4722–33. [Google Scholar]

25. Luo Y, Li P. Neighborhood-aware scalable temporal network representation learning. In: Proceedings of the First Learning on Graphs Conference (LoG 2022); 2022 Dec 9–12; Virtual. p. 1. [Google Scholar]

26. Lu X, Sun L, Zhu T, Lv W. Improving temporal link prediction via temporal walk matrix projection. Adv Neural Inf Process Syst. 2024;37:141153–82. doi:10.52202/079017-4482. [Google Scholar] [CrossRef]

27. Wu Y, Fang Y, Liao L. On the feasibility of simple transformer for dynamic graph modeling. In: Proceedings of the ACM Web Conference 2024; 2024 May 13–17; Singapore. p. 870–80. [Google Scholar]

28. Chen H, Jiao P, Du M, Guo X, Zhao Z, Jin D, et al. TGFormer: towards temporal graph transformer with auto-correlation mechanism. Pattern Recognit. 2026;170:112053. [Google Scholar]

29. Ding Z, Li Y, He Y, Norelli A, Wu J, Tresp V, et al. DyGMamba: efficiently modeling long-term temporal dependency on continuous-time dynamic graphs with state space models. arXiv:2408.04713. 2024. [Google Scholar]

30. Poli M, Massaroli S, Nguyen E, Fu DY, Dao T, Baccus S, et al. Hyena hierarchy: towards larger convolutional language models. In: Proceedings of the 40th International Conference on Machine Learning; 2023 Jul 23–29; Honolulu, HI, USA. p. 28043–78. [Google Scholar]

31. Fu DY, Epstein EL, Nguyen E, Thomas AW, Zhang M, Dao T, et al. Simple hardware-efficient long convolutions for sequence modeling. In: Proceedings of the 40th International Conference on Machine Learning; 2023 Jul 23–29; Honolulu, HI, USA. p. 10373–91. [Google Scholar]

32. Sohn K, Lee H, Yan X. Learning structured output representation using deep conditional generative models. Adv Neural Inf Process Syst. 2015;28:1–9. [Google Scholar]

33. Pandey AP, Varghese AJ, Patil S, Xu M. A comparative study on dynamic graph embedding based on mamba and transformers. arXiv:2412.11293. 2024. [Google Scholar]

34. You J, Du T, Leskovec J. ROLAND: graph learning framework for dynamic graphs. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2022 Aug 14–18; Washington, DC, USA. p. 2358–66. [Google Scholar]

35. Cong W, Wu Y, Tian Y, Gu M, Xia Y, Mahdavi M, et al. Dynamic graph representation learning via graph transformer networks. arXiv.2111.10447. 2021. [Google Scholar]

36. Sankar A, Wu Y, Gou L, Zhang W, Yang H. DySAT: deep neural representation learning on dynamic graphs via self-attention networks. In: Proceedings of the 13th International Conference on Web Search and Data Mining; 2020 Feb 3–7; Houston, TX, USA. p. 519–27. [Google Scholar]

37. Pareja A, Domeniconi G, Chen J, Ma T, Suzumura T, Kanezashi H, et al. EvolveGCN: evolving graph convolutional networks for dynamic graphs. Proc AAAI Conf Artif Intell. 2020;34(04):5363–70. [Google Scholar]

38. Xiong J, Zareie A, Sakellariou R. A survey of link prediction in temporal networks. arXiv:2502.21185. 2025. [Google Scholar]

39. Ma Y, Guo Z, Ren Z, Tang J, Yin D. Streaming graph neural networks. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval; 2020 Jul 25–30; Virtual. p. 719–28. [Google Scholar]

40. Wang X, Lyu D, Li M, Xia Y, Yang Q, Wang X, et al. APAN: asynchronous propagation attention network for real-time temporal graph embedding. In: Proceedings of the 2021 International Conference on Management of Data; 2021 Jun 20–25; Virtual. p. 2628–38. [Google Scholar]

41. Trivedi R, Farajtabar M, Biswal P, Zha H. DyRep: learning representations over dynamic graphs. In: Proceedings of the International Conference on Learning Representations; 2019 May 6–9; New Orleans, LA, USA. [Google Scholar]

42. Fu DY, Dao T, Saab KK, Thomas AW, Rudra A, Ré C. Hungry hungry hippos: towards language modeling with state space models. arXiv:2212.14052. 2022. [Google Scholar]

43. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Adv Neural Inf Process Syst. 2017;30:6000–10. doi:10.65215/pc26a033. [Google Scholar] [CrossRef]

44. Moskalev A, Prakash M, Xu J, Cui T, Liao R, Mansi T. Geometric hyena networks for large-scale equivariant learning. arXiv:2505.22560. 2025. [Google Scholar]

45. Gu A, Dao T. Linear-time sequence modeling with selective state spaces. In: Proceedings of the First Conference on Language Modeling; 2024 Oct 7–9; Philadelphia, PA, USA. [Google Scholar]

46. Peng B, Alcaide E, Anthony Q, Albalak A, Arcadinho S, Biderman S, et al. RWKV: reinventing rnns for the transformer era. arXiv:2305.13048. 2023. [Google Scholar]

47. Pennebaker JW, Francis ME, Booth RJ. Linguistic inquiry and word count: lIWC 2001. Mahway Lawrence Erlbaum Assoc. 2001;71(2001):206–29. doi:10.4018/978-1-60960-741-8.ch012. [Google Scholar] [CrossRef]

48. Shetty J, Adibi J. The Enron email dataset database schema and brief statistical report. Inf Sci Inst Tech Rep Univ South Calif. 2004;4(1):120–8. [Google Scholar]

49. Madan A, Cebrian M, Moturu S, Farrahi K. Sensing the “health state” of a community. IEEE Pervasive Comput. 2011;11(4):36–45. doi:10.1109/mprv.2011.79. [Google Scholar] [CrossRef]

50. Panzarasa P, Opsahl T, Carley KM. Patterns and dynamics of users’ behavior and interaction: network analysis of an online community. J Am Soc Inf Sci Technol. 2009;60(5):911–32. [Google Scholar]

51. Strohmeier M, Olive X, Lübbe J, Schäfer M, Lenders V. Crowdsourced air traffic data from the OpenSky Network 2019–20. Earth Syst Sci Data. 2021;13(2):357–66. doi:10.5194/essd-13-357-2021. [Google Scholar] [CrossRef]

52. Huang S, Poursafaei F, Danovitch J, Fey M, Hu W, Rossi E, et al. Temporal graph benchmark for machine learning on temporal graphs. In: Proceedings of the 37th International Conference on Neural Information Processing Systems; 2023 Dec 10–16; New Orleans, LA, USA. [Google Scholar]

53. Gastinger J, Huang S, Galkin M, Loghmani E, Parviz A, Poursafaei F, et al. TGB 2.0: a benchmark for learning on temporal knowledge graphs and heterogeneous graphs. In: Proceedings of the 38th International Conference on Neural Information Processing Systems; 2024 Dec 10–15; Vancouver, BC, Canada. [Google Scholar]