dymad.modules.sequential

Classes

SequentialBase(seq_len, *[, last_only, net, ...])

Interface module that handles time-delayed input sequences.

SimpleRNN(seq_len, *[, last_only, net, ...])

A simple recurrent neural network module

StepwiseModel(seq_len, *[, last_only, net, ...])

Naive application of a network to each step of a sequence.

VanillaRNN(seq_len, *[, last_only, net, ...])

Vanilla RNN from pytorch.
class dymad.modules.sequential.SequentialBase(seq_len, *, last_only=True, net=None, input_dim=-1, hidden_dim=-1, output_dim=-1, n_layers=2, activation=<class 'torch.nn.modules.activation.ReLU'>, weight_init=<function xavier_uniform_>, bias_init=<function zeros_>, gain=1.0, dtype=None, device=None, **kwargs)

Bases: Module

Interface module that handles time-delayed input sequences.

The module assumes that the input is given in shape (…, seq_len * input_dim), where seq_len is the length of the time-delay/sequence, and input_dim is the dimension of each step’s input features. The module reshapes the input to (…, seq_len, input_dim) and passes it to an internal sequential model (e.g., RNN), and return the output either at the last step (…, output_dim) or the full sequence flattened (…, seq_len * output_dim).

It considers two types of architectures

  • Internally construct RNN-like models, that applies to the input in the standard way. This is usually good for defining dynamics.

  • Externally provided models, that process step by step and not necessarily recurrently. This is usually good for defining encoders/decoders. Examples are MLP and GNN.

  • In both cases, the models expect input of shape (-1, seq_len, input_dim) and return output of shape (-1, seq_len, output_dim).

  • In either case, a subclass must implement _run_seq() method that defines how to run the model.

The module can also operate in two modes

  • last_only=True: returns only the output at the last step (…, output_dim). Usually for dynamics.

  • last_only=False: returns the outputs at all steps, flattened (…, seq_len * output_dim). Usually for encoders/decoders.

Parameters:

  • seq_len (int) – Length of the input sequences.

  • last_only (Optional[bool]) – Whether to return only the last step output, default is True.

  • net (Optional[nn.Module]) – Optional externally provided model. If None, an internal RNN-like model is constructed.

  • input_dim (Optional[int]) – Dimension of the input features of all steps (for RNN-like).

  • hidden_dim (Optional[int]) – Width of the hidden layers (for RNN-like).

  • output_dim (Optional[int]) – Dimension of the output features of all steps (for RNN-like).

  • n_layers (Optional[int]) – Number of layers (for RNN-like).

  • activation (Union[str, nn.Module, Callable[[], nn.Module]]) – Activation function (for RNN-like).

  • weight_init (Union[str, Callable[[torch.Tensor, float], None]]) – Weight initialization method (for RNN-like).

  • bias_init (Callable[[torch.Tensor], None]) – Bias initialization method (for RNN-like).

  • gain (Optional[float]) – Gain factor for weight initialization (for RNN-like).

  • dtype – Data type for the module (for RNN-like).

  • device – Device for the module (for RNN-like).

  • **kwargs – Additional keyword arguments passed to the internal model constructor.

forward(x, u=None)

The network does concatenation internally, because x and u are both time-delayed and concatenated, and applying the sequential model requires to stack x and u of the same steps and then concatenate.

Parameters:

  • x (Tensor) – Stacked input tensor of shape (…, seq_len * x_dim)

  • u (Tensor | None) – (Optional) Stacked control tensor of shape (…, seq_len * u_dim); needed when serving as encoder with inputs.

Returns:

Stacked output tensor of shape (…, output_dim),

where the last slot is the output of the sequential model at the last step if last_only is True, otherwise all outputs are concatenated.

Return type:

output

class dymad.modules.sequential.SimpleRNN(seq_len, *, last_only=True, net=None, input_dim=-1, hidden_dim=-1, output_dim=-1, n_layers=2, activation=<class 'torch.nn.modules.activation.ReLU'>, weight_init=<function xavier_uniform_>, bias_init=<function zeros_>, gain=1.0, dtype=None, device=None, **kwargs)

Bases: SequentialBase

A simple recurrent neural network module

One layer, unidirectional, but supports arbitrary activations with a linear readout.

class dymad.modules.sequential.StepwiseModel(seq_len, *, last_only=True, net=None, input_dim=-1, hidden_dim=-1, output_dim=-1, n_layers=2, activation=<class 'torch.nn.modules.activation.ReLU'>, weight_init=<function xavier_uniform_>, bias_init=<function zeros_>, gain=1.0, dtype=None, device=None, **kwargs)

Bases: SequentialBase

Naive application of a network to each step of a sequence.

class dymad.modules.sequential.VanillaRNN(seq_len, *, last_only=True, net=None, input_dim=-1, hidden_dim=-1, output_dim=-1, n_layers=2, activation=<class 'torch.nn.modules.activation.ReLU'>, weight_init=<function xavier_uniform_>, bias_init=<function zeros_>, gain=1.0, dtype=None, device=None, **kwargs)

Bases: SequentialBase

Vanilla RNN from pytorch.