pytorchrl.agent.actors.feature_extractors package

Submodules

pytorchrl.agent.actors.feature_extractors.cnn module

class pytorchrl.agent.actors.feature_extractors.cnn.CNN(input_space, rgb_norm=True, activation=<class 'torch.nn.modules.activation.ReLU'>, final_activation=True, dropout=0.0, layer_norm=False, strides=[4, 2, 1], filters=[32, 64, 64], kernel_sizes=[8, 4, 3], output_sizes=[256, 448])[source]

Bases: torch.nn.modules.module.Module

Convolutional Neural Network.

Parameters

input_space (gym.Space) – Environment observation space.
rgb_norm (bool) – Whether or not to divide input by 255.
activation (func) – Non-linear activation function.
final_activation (bool) – Whether or not to apply activation function after last layer.
layer_norm (bool) – Use layer normalization.
dropout (float) – Dropout probability.
strides (list) – Convolutional layers strides.
filters (list) – Convolutional layers number of filters.
kernel_sizes (list) – Convolutional layers kernel sizes.
output_sizes (list) – output hidden layers sizes.

forward(inputs)[source]

Forward pass Neural Network

Parameters: inputs (torch.tensor) – Input data.
Returns: out – Output feature map.
Return type: torch.tensor

training: bool

pytorchrl.agent.actors.feature_extractors.dictnet module

class pytorchrl.agent.actors.feature_extractors.dictnet.DictNet(input_space, shared_hidden_sizes=[256, 256], output_size=256, activation=<class 'torch.nn.modules.activation.ReLU'>)[source]

Bases: torch.nn.modules.module.Module

Neural network architecture designed to deal with dict observation spaces. It extracts individual features independently with either a MLP or a CNN, and then combines the information with a final sequence of fully connected layers.

Parameters

input_space (gym.Space) – Environment observation space.
shared_hidden_sizes (list) – Hidden layers sizes used to combine multimodel features.
activation (func) – Non-linear activation function.

forward(inputs)[source]

Forward pass Neural Network

Parameters: inputs (torch.tensor) – Input data.
Returns: out – Output feature map.
Return type: torch.tensor

training: bool

pytorchrl.agent.actors.feature_extractors.ensemble_layer module

class pytorchrl.agent.actors.feature_extractors.ensemble_layer.EnsembleFC(in_features: int, out_features: int, ensemble_size: int, weight_decay: float = 0.0, bias: bool = True)[source]

Bases: torch.nn.modules.module.Module

extra_repr() → str[source]

Set the extra representation of the module

To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.

forward(input: torch.Tensor) → torch.Tensor[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

reset_parameters() → None[source]

training: bool

pytorchrl.agent.actors.feature_extractors.fixup_cnn module

class pytorchrl.agent.actors.feature_extractors.fixup_cnn.FixupCNN(input_space, output_size=256, rgb_norm=True, activation=<class 'torch.nn.modules.activation.ReLU'>, final_activation=True)[source]

Bases: torch.nn.modules.module.Module

A larger version of the IMPALA CNN with Fixup init. See Fixup: https://arxiv.org/abs/1901.09321. :param input_space: Environment observation space. :type input_space: gym.Space :param rgb_norm: Whether or not to divide input by 255. :type rgb_norm: bool

forward(inputs)[source]

Forward pass Neural Network :param inputs: Input data. :type inputs: torch.tensor

Returns: out – Output feature map.
Return type: torch.tensor

training: bool

class pytorchrl.agent.actors.feature_extractors.fixup_cnn.FixupResidualModule(depth, num_residual)[source]

Bases: torch.nn.modules.module.Module

FixupCNN resudial block. :param depth: Number of input feature maps. :type depth: int :param num_residual: Number of residual feature blocks in the architecture :type num_residual: int

forward(x)[source]

Forward pass residual block :param inputs: Input feature map. :type inputs: torch.tensor

Returns: out – Output feature map.
Return type: torch.tensor

training: bool

pytorchrl.agent.actors.feature_extractors.mlp module

class pytorchrl.agent.actors.feature_extractors.mlp.MLP(input_space, hidden_sizes=[256, 256], output_size=256, activation=<class 'torch.nn.modules.activation.ReLU'>, layer_norm=False, dropout=0.0, final_activation=True)[source]

Bases: torch.nn.modules.module.Module

Multilayer Perceptron network

Parameters

input_space (gym.Space) – Environment observation space.
hidden_sizes (list) – Hidden layers sizes.
output_size (int) – Size of output feature map.
activation (func) – Non-linear activation function.
layer_norm (bool) – Use layer normalization.
dropout (float) – Dropout probability.
final_activation (bool) – Whether or not to apply activation function after last layer.

forward(inputs)[source]

Forward pass Neural Network

Parameters: inputs (torch.tensor) – Input data.
Returns: out – Output feature map.
Return type: torch.tensor

training: bool

pytorchrl.agent.actors.feature_extractors.utils module

pytorchrl.agent.actors.feature_extractors.utils.get_gain(activation)[source]

Module contents

pytorchrl.agent.actors.feature_extractors.default_feature_extractor(input_space)[source]

Returns the default net to use as a feature extractor given input_space.

Parameters: input_space (gym.Space) – Environment observation space.

pytorchrl.agent.actors.feature_extractors.get_feature_extractor(name)[source]: Returns model class from name.