RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer

Submodules

RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.attn module

class RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.attn.MultiHeadAttention(*args: Any, **kwargs: Any)

Bases: Module

The MultiHeadAttention class is used to perform multi-head attention in the transformer model. It splits the input vectors into different heads or projections, calculates the attention scores, and concatenates the results before applying a linear transformation to obtain the final output.

d_model: number of units in the model (dimensionality of the feature vectors)

build_model()

forward(queries, keys, values, mask)

class RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.attn.ScaledDotProductAttention(*args: Any, **kwargs: Any)

Bases: Module

Scaled Dot-Product Attention

forward(queries, keys, values, mask=None)

RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.decoder module

class RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.decoder.Decoder(*args: Any, **kwargs: Any)

Bases: Module

forward(inputs, encoder_out, mask_attn, mask_enc_dec)

inputs: [batch_size, tgt_len, input_size] encoder_out: [batch_size, src_len, d_model]

class RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.decoder.DecoderLayer(*args: Any, **kwargs: Any)

Bases: Module

build_model()

forward(decoder_in, encoder_out, mask=None, mask_enc_dec=None)

RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.embed module

class RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.embed.PositionalEmbedding(*args: Any, **kwargs: Any)

Bases: Module

forward(x)

class RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.embed.PositionalEncoding(*args: Any, **kwargs: Any)

Bases: Module

Positional Encoding class for Transformers.

This module implements positional encoding for a Transformer model. Positional encoding helps maintain the temporal information of the data. The chosen implementation follows the vanilla method as originally defined in the “Attention Is All You Need” paper.

Parameters:

• d_model (int) – The dimensionality of the model, representing the number of features or channels in the input and output embeddings.

• dropout (float, optional) – The dropout probability applied to the positional encodings, by default 0.1.

• max_len (int, optional) – The maximum sequence length for which positional encodings are calculated, by default 5000.

pe

The positional encodings matrix.

Type:

torch.Tensor

forward(x, pos=0)

Forward pass through the Positional Encoding layer.

Examples

Create a PositionalEncoding instance:

>>> positional_encoder = PositionalEncoding(d_model=512, dropout=0.1, max_len=1000)
>>> input_data = torch.rand(32, 1000, 512)
>>> output_data = positional_encoder(input_data)

Notes

PositionalEncoding is a crucial component in Transformer models to capture the order of tokens in input sequences.

References

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., … & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 30-31).

forward(x, pos=0)

Apply positional encoding to the input data.

Parameters:

• x (torch.Tensor) – The input tensor to which positional encoding is applied.

• pos (int, optional) – The starting position for adding positional encodings, by default 0.

Returns:

The input tensor with positional encoding added and dropout applied.

Return type:

torch.Tensor

RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.encoder module

class RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.encoder.Encoder(*args: Any, **kwargs: Any)

Bases: Module

forward(inputs, mask)

inputs: [batch_size, seq_len, input_size] mask: [batch_size, 1, 1, seq_len]

class RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.encoder.EncoderLayer(*args: Any, **kwargs: Any)

Bases: Module

build_model()

forward(encoder_in, mask=None)

RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.model module

class RADAR.time_series.algorithms.modelsTransformersTS.vanillaTransformer.model.Transformer(*args: Any, **kwargs: Any)

Bases: Module

Transformer para detección de anomalías en series temporales.

Parameters:

• size_enc_in – dimensión de entrada del encoder

• size_dec_in – dimensión de entrada/salida del decoder

• ulayers_feedfwd – número de unidades en las capas feedforward

• d_qk – dimensión de claves y consultas (Q/K)

• d_v – dimensión de los valores (V)

• d_model – dimensión interna del modelo

• n_layers – número de capas en encoder y decoder

• n_heads – número de cabezas de atención

• dropout_rate – tasa de dropout

• embedding_scale – si escalar embeddings por sqrt(d_model)

• attns_outs – si retornar atenciones como salida

forward(enc_inputs, dec_inputs)

enc_inputs: [B, L_enc, D_enc] dec_inputs: [B, L_dec, D_dec]

gen_mask(src, tgt)

Genera máscaras para atención en series temporales.

src: [B, L_src, D] tgt: [B, L_tgt, D]

Module contents