AIToolbox
A library that offers tools for AI problem solving.
AIToolbox::MDP::TreeBackupL Class Reference

This class implements off-policy control via Tree Backup(lambda). More...

#include <AIToolbox/MDP/Algorithms/TreeBackupL.hpp>

Inheritance diagram for AIToolbox::MDP::TreeBackupL:
AIToolbox::MDP::OffPolicyControl< TreeBackupL > AIToolbox::MDP::OffPolicyBase

Public Types

using Parent = OffPolicyControl< TreeBackupL >
 
- Public Types inherited from AIToolbox::MDP::OffPolicyControl< TreeBackupL >
using Parent = OffPolicyBase
 
- Public Types inherited from AIToolbox::MDP::OffPolicyBase
using Trace = std::tuple< size_t, size_t, double >
 
using Traces = std::vector< Trace >
 

Public Member Functions

 TreeBackupL (const size_t s, const size_t a, const double discount=1.0, const double alpha=0.1, const double lambda=0.9, const double tolerance=0.001, const double epsilon=0.1)
 Basic constructor. More...
 
void setLambda (double l)
 This function sets the new lambda parameter. More...
 
double getLambda () const
 This function returns the currently set lambda parameter. More...
 
- Public Member Functions inherited from AIToolbox::MDP::OffPolicyControl< TreeBackupL >
 OffPolicyControl (size_t s, size_t a, double discount=1.0, double alpha=0.1, double tolerance=0.001, double epsilon=0.1)
 Basic constructor. More...
 
void stepUpdateQ (const size_t s, const size_t a, const size_t s1, const double rew)
 This function updates the internal QFunction using the discount set during construction. More...
 
void setEpsilon (double e)
 This function sets the epsilon parameter. More...
 
double getEpsilon () const
 This function will return the currently set epsilon parameter. More...
 
- Public Member Functions inherited from AIToolbox::MDP::OffPolicyBase
 OffPolicyBase (size_t s, size_t a, double discount=1.0, double alpha=0.1, double tolerance=0.001)
 Basic construtor. More...
 
void setLearningRate (double a)
 This function sets the learning rate parameter. More...
 
double getLearningRate () const
 This function will return the current set learning rate parameter. More...
 
void setDiscount (double d)
 This function sets the new discount parameter. More...
 
double getDiscount () const
 This function returns the currently set discount parameter. More...
 
void setTolerance (double t)
 This function sets the trace cutoff parameter. More...
 
double getTolerance () const
 This function returns the currently set trace cutoff parameter. More...
 
void clearTraces ()
 This function clears the already set traces. More...
 
const TracesgetTraces () const
 This function returns the currently set traces. More...
 
void setTraces (const Traces &t)
 This function sets the currently set traces. More...
 
size_t getS () const
 This function returns the number of states on which QLearning is working. More...
 
size_t getA () const
 This function returns the number of actions on which QLearning is working. More...
 
const QFunctiongetQFunction () const
 This function returns a reference to the internal QFunction. More...
 
void setQFunction (const QFunction &qfun)
 This function allows to directly set the internal QFunction. More...
 

Additional Inherited Members

- Protected Member Functions inherited from AIToolbox::MDP::OffPolicyBase
void updateTraces (size_t s, size_t a, double error, double traceDiscount)
 This function updates the traces using the input data. More...
 
- Protected Attributes inherited from AIToolbox::MDP::OffPolicyControl< TreeBackupL >
double epsilon_
 
- Protected Attributes inherited from AIToolbox::MDP::OffPolicyBase
size_t S
 
size_t A
 
double discount_
 
double alpha_
 
double tolerance_
 
QFunction q_
 
Traces traces_
 

Detailed Description

This class implements off-policy control via Tree Backup(lambda).

See also
TreeBackupLEvaluation

Member Typedef Documentation

◆ Parent

Constructor & Destructor Documentation

◆ TreeBackupL()

AIToolbox::MDP::TreeBackupL::TreeBackupL ( const size_t  s,
const size_t  a,
const double  discount = 1.0,
const double  alpha = 0.1,
const double  lambda = 0.9,
const double  tolerance = 0.001,
const double  epsilon = 0.1 
)
inline

Basic constructor.

Parameters
sThe size of the state space.
aThe size of the action space.
discountDiscount for the problem.
alphaLearning rate parameter.
lambdaLambda trace parameter.
toleranceTrace cutoff parameter.
epsilonThe epsilon of the implied target greedy epsilon policy.

Member Function Documentation

◆ getLambda()

double AIToolbox::MDP::TreeBackupL::getLambda ( ) const
inline

This function returns the currently set lambda parameter.

◆ setLambda()

void AIToolbox::MDP::TreeBackupL::setLambda ( double  l)
inline

This function sets the new lambda parameter.

The lambda parameter must be >= 0.0 and <= 1.0, otherwise the function will throw an std::invalid_argument.

Parameters
lThe new lambda parameter.

The documentation for this class was generated from the following file: