narla.neurons

Network

class narla.neurons.Network[source]

Bases: torch.nn.modules.module.Module

abstract forward(X)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Return type: Tensor

training: bool

Neuron

class narla.neurons.Neuron(observation_size, number_of_actions, learning_rate)[source]

Bases: object

Neuron base class

Parameters

observation_size (int) – Size of the observation which the Neuron will receive
number_of_actions (int) – Number of actions available to the Neuron

abstract act(observation)[source]

Take an action based on the observation

Parameters: observation (Tensor) – Observation from the Neuron’s local environment
Return type: Tensor

property history: narla.history.history.History

Access the History of the Neuron

Return type: History

abstract learn(*reward_types)[source]

Update the Neuron’s internal Network

Parameters: reward_types (RewardTypes) – RewardTypes to be used for learning

record(**kwargs)[source]

Record information in the Neuron’s internal History. This stored data will be used for learning

Parameters: kwargs – Keyword arguments

NeuronSettings

class narla.neurons.NeuronSettings(learning_rate=0.0001, neuron_type=NeuronTypes.POLICY_GRADIENT)[source]

Bases: narla.settings.base_settings.BaseSettings

create_neuron(observation_size, number_of_actions)[source]

Return type: Neuron

learning_rate: float = 0.0001: Learning rate for Neurons

neuron_type: narla.neurons.neuron_types.NeuronTypes = 'policy_gradient': Type of Neuron that will be used

NeuronTypes

class narla.neurons.NeuronTypes(value)[source]

Bases: enum.Enum

An enumeration.

ACTOR_CRITIC = 'actor_critic'

DEEP_Q = 'deep_q'

POLICY_GRADIENT = 'policy_gradient'

to_neuron_type()[source]

Convert Neuron Enum to class Type

Return type: Type[Neuron]

actor_critic

Network

class narla.neurons.actor_critic.Network(input_size, output_size, embedding_size=128)[source]

Bases: narla.neurons.network.Network

forward(X)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Return type: Tuple[Tensor, Tensor]

training: bool

Neuron

class narla.neurons.actor_critic.Neuron(observation_size, number_of_actions, learning_rate=0.0001)[source]

Bases: narla.neurons.neuron.Neuron

The ActorCritic Neuron is based on this PyTorch example

Parameters

observation_size (int) – Size of the observation which the Neuron will receive
number_of_actions (int) – Number of actions available to the Neuron
learning_rate (float) – Learning rate for the Neuron’s Network

act(observation)[source]

Take an action based on the observation

Parameters: observation (Tensor) – Observation from the Neuron’s local environment
Return type: Tensor

get_returns(*reward_types)[source]

Return type: Tensor

learn(*reward_types)[source]

Update the Neuron’s internal Network

Parameters: reward_types (RewardTypes) – RewardTypes to be used for learning

deep_q

Network

class narla.neurons.deep_q.Network(input_size, output_size, embedding_size=128)[source]

Bases: narla.neurons.network.Network

clone()[source]

Return type: Network

forward(X)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

Neuron

class narla.neurons.deep_q.Neuron(observation_size, number_of_actions, learning_rate=0.0001)[source]

Bases: narla.neurons.neuron.Neuron

The DeepQ Neuron is based on this PyTorch example

Parameters

observation_size (int) – Size of the observation which the Neuron will receive
number_of_actions (int) – Number of actions available to the Neuron
learning_rate (float) – Learning rate for the Neuron’s Network

act(observation)[source]

Take an action based on the observation

Parameters: observation (Tensor) – Observation from the Neuron’s local environment
Return type: Tensor

learn(*reward_types)[source]

Update the Neuron’s internal Network

Parameters: reward_types (RewardTypes) – RewardTypes to be used for learning

sample_history(*reward_types)[source]

Return type: Tuple[Tensor, …]

update_target_network()[source]

policy_gradient

Network

class narla.neurons.policy_gradient.Network(input_size, output_size, embedding_size=32)[source]

Bases: narla.neurons.network.Network

clone()[source]

Return type: Network

forward(X)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

Neuron

class narla.neurons.policy_gradient.Neuron(observation_size, number_of_actions, learning_rate=0.01)[source]

Bases: narla.neurons.neuron.Neuron

PolicyGradient Neuron

Parameters

observation_size (int) – Size of the observation which the Neuron will receive
number_of_actions (int) – Number of actions available to the Neuron
learning_rate (float) – Learning rate for the Neuron’s Network

act(observation)[source]

Take an action based on the observation

Parameters: observation (Tensor) – Observation from the Neuron’s local environment
Return type: Tensor

get_returns(*reward_types)[source]

Return type: Tensor

learn(*reward_types)[source]

Update the Neuron’s internal Network

Parameters: reward_types (RewardTypes) – RewardTypes to be used for learning