pytorchrl package

Subpackages

• pytorchrl.agent package
  • Subpackages
    • pytorchrl.agent.actors package
      • Subpackages
      • Submodules
      • pytorchrl.agent.actors.base module
      • pytorchrl.agent.actors.model_based_planner_actor module
      • pytorchrl.agent.actors.off_policy_actor module
      • pytorchrl.agent.actors.on_policy_actor module
      • pytorchrl.agent.actors.utils module
      • Module contents
    • pytorchrl.agent.algorithms package
      • Subpackages
      • Submodules
      • pytorchrl.agent.algorithms.base module
      • pytorchrl.agent.algorithms.utils module
      • Module contents
    • pytorchrl.agent.env package
      • Subpackages
      • Submodules
      • pytorchrl.agent.env.env_wrappers module
      • pytorchrl.agent.env.make_env module
      • pytorchrl.agent.env.vec_env module
      • pytorchrl.agent.env.vector_wrappers module
      • Module contents
    • pytorchrl.agent.storages package
      • Subpackages
      • Submodules
      • pytorchrl.agent.storages.base module
      • Module contents
  • Module contents
• pytorchrl.envs package
  • Subpackages
    • pytorchrl.envs.animal_olympics package
      • Submodules
      • pytorchrl.envs.animal_olympics.animal_olympics_env_factory module
      • pytorchrl.envs.animal_olympics.utils module
      • pytorchrl.envs.animal_olympics.wrappers module
      • Module contents
    • pytorchrl.envs.atari package
      • Submodules
      • pytorchrl.envs.atari.atari_env_factory module
      • pytorchrl.envs.atari.utils module
      • pytorchrl.envs.atari.wrappers module
      • Module contents
    • pytorchrl.envs.causal_world package
      • Submodules
      • pytorchrl.envs.causal_world.causal_world_env_factory module
      • Module contents
    • pytorchrl.envs.habitat package
      • Submodules
      • pytorchrl.envs.habitat.habitat_env_factory module
      • pytorchrl.envs.habitat.utils module
      • pytorchrl.envs.habitat.wrappers module
      • Module contents
    • pytorchrl.envs.mujoco package
      • Submodules
      • pytorchrl.envs.mujoco.mujoco_env_factory module
      • Module contents
    • pytorchrl.envs.obstacle_tower package
      • Submodules
      • pytorchrl.envs.obstacle_tower.obstacle_tower_env_factory module
      • pytorchrl.envs.obstacle_tower.utils module
      • pytorchrl.envs.obstacle_tower.wrappers module
      • Module contents
    • pytorchrl.envs.pybullet package
      • Submodules
      • pytorchrl.envs.pybullet.pybullet_env_factory module
      • pytorchrl.envs.pybullet.sparse_reacher module
      • Module contents
  • Submodules
  • pytorchrl.envs.common module
  • Module contents
• pytorchrl.scheme package
  • Subpackages
    • pytorchrl.scheme.base package
      • Submodules
      • pytorchrl.scheme.base.utils module
      • pytorchrl.scheme.base.worker module
      • pytorchrl.scheme.base.worker_set module
      • Module contents
    • pytorchrl.scheme.collection package
      • Submodules
      • pytorchrl.scheme.collection.c_worker module
      • pytorchrl.scheme.collection.c_worker_set module
      • Module contents
    • pytorchrl.scheme.gradients package
      • Submodules
      • pytorchrl.scheme.gradients.g_worker module
      • pytorchrl.scheme.gradients.g_worker_set module
      • Module contents
    • pytorchrl.scheme.updates package
      • Submodules
      • pytorchrl.scheme.updates.u_worker module
      • Module contents
  • Submodules
  • pytorchrl.scheme.scheme module
  • pytorchrl.scheme.utils module
  • Module contents

Submodules

pytorchrl.learner module

class pytorchrl.learner.Learner(scheme, target_steps, log_dir=None)

Bases: object

Task learner class. Class to manage the training process. It pushes forward the training process by calling the update workers and tracks progress.

Parameters

• scheme (Scheme) – Training scheme class instance, handling coordination of workers.

• target_steps (int) – Number of environment steps to reach to complete training.

• log_dir (str) – Target directory for model checkpoints and logs.

done()

Return True if training has finished (target_steps reached).

Returns

flag – True if training has reached the target number of steps.

Return type

bool

get_metrics(add_algo_metrics=True, add_episodes_metrics=False, add_scheme_metrics=False, add_time_metrics=False)

Returns current value of tracked metrics.

print_info(add_algo_info=True, add_episodes_info=True, add_scheme_info=False, add_time_info=False)

Print relevant information about the training process

save_model(fname='model.state_dict')

Save currently learned actor_critic version.

Returns

save_name – Path to saved file.

Return type

str

step()

Takes a logical synchronous optimization step.

stop()

Stop all threads.

update_algorithm_parameter(parameter_name, new_parameter_value)

If parameter_name is an attribute of the algorithm used for training, change its value to new_parameter_value value.

Parameters

• parameter_name (str) – Worker.algo attribute name

• new_parameter_value (int or float) – New value for parameter_name.

pytorchrl.utils module

class pytorchrl.utils.LoadFromFile(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

class pytorchrl.utils.RunningMeanStd(epsilon=0.0001, shape=(), device=device(type='cpu'))

Bases: object

Class to keep track on the running mean and variance of tensors batches.

update(x)

update_from_moments(batch_mean, batch_var, batch_count)

pytorchrl.utils.cleanup_log_dir(log_dir)

Create log directory and remove old files.

Parameters

log_dir (str) – Path to log directory.

pytorchrl.utils.clip_grad_norm_(parameters, norm_type: float = 2.0)

This is the official clip_grad_norm implemented in pytorch but the max_norm part has been removed. https://github.com/pytorch/pytorch/blob/52f2db752d2b29267da356a06ca91e10cd732dbc/torch/nn/utils/clip_grad.py#L9

pytorchrl.utils.save_argparse(args, filename, exclude=None)

Module contents

class pytorchrl.DataTransition(Observation, RecurrentHiddenStates, Done, Action, Reward, NextObservation, NextRecurrentHiddenStates, NextDone, EnvironmentInformation)

Bases: tuple

property Action

Alias for field number 3

property Done

Alias for field number 2

property EnvironmentInformation

Alias for field number 8

property NextDone

Alias for field number 7

property NextObservation

Alias for field number 5

property NextRecurrentHiddenStates

Alias for field number 6

property Observation

Alias for field number 0

property RecurrentHiddenStates

Alias for field number 1

property Reward

Alias for field number 4