AIToolbox::POMDP::AMDP Class Reference

This class implements the Augmented MDP algorithm. More...

#include <AIToolbox/POMDP/Algorithms/AMDP.hpp>

Public Types
using	Discretizer = std::function< size_t(const Belief &)>

Public Member Functions
	AMDP (size_t nBeliefs, size_t entropyBuckets)
	Basic constructor. More...
void	setBeliefSize (size_t nBeliefs)
	This function sets a new number of sampled beliefs. More...
void	setEntropyBuckets (size_t buckets)
	This function sets the new number of buckets in which to discretize the entropy. More...
size_t	getBeliefSize () const
	This function returns the currently set number of sampled beliefs. More...
size_t	getEntropyBuckets () const
	This function returns the currently set number of entropy buckets. More...
template<IsModel M>
std::tuple< MDP::Model, Discretizer >	discretizeDense (const M &model)
	This function constructs an approximate dense MDP of the provided POMDP model. More...
template<IsModel M>
std::tuple< MDP::SparseModel, Discretizer >	discretizeSparse (const M &model)
	This function constructs an approximate sparse MDP of the provided POMDP model. More...

Detailed Description

This class implements the Augmented MDP algorithm.

This algorithm transforms a POMDP into an approximately equivalent MDP. This is done by extending the original POMDP statespace with a discretized entropy component, which approximates a sufficient statistic for the belief. In essence, AMDP builds states which contain intrinsically information about the uncertainty of the agent.

In order to compute a new transition and reward function, AMDP needs to sample possible transitions at random, since each belief can potentially update to any other belief. We sample beliefs using the BeliefGenerator class which creates both random beliefs and beliefs generated using the original POMDP model, in order to try to obtain beliefs distributed in a way that better resembles the original problem.

Once this is done, it is simply a matter of taking each belief, computing every possible new belief given an action and observation, and sum up all possibilities.

This class also bundles together with the resulting MDP a function to convert an original POMDP belief into an equivalent AMDP state; this is done so that a policy can be applied, observation gathered and beliefs updated while continuing to use the approximated model.

Member Typedef Documentation

Discretizer

using AIToolbox::POMDP::AMDP::Discretizer = std::function<size_t(const Belief&)>

Constructor & Destructor Documentation

AMDP()

AIToolbox::POMDP::AMDP::AMDP	(	size_t	nBeliefs,
		size_t	entropyBuckets
	)

Basic constructor.

Parameters

nBeliefs	The number of beliefs to sample from when building the MDP model.
entropyBuckets	The number of buckets into which discretize entropy.

Member Function Documentation

discretizeDense()

template<IsModel M>

std::tuple< MDP::Model, AMDP::Discretizer > AIToolbox::POMDP::AMDP::discretizeDense

(

const M &

model

)

This function constructs an approximate dense MDP of the provided POMDP model.

Template Parameters

M	The type of the POMDP model.

Parameters

model

The POMDP model to be approximated.

Returns

A tuple containing a dense MDP model which approximates the POMDP argument, and a function that converts a POMDP belief into a state of the MDP model.

discretizeSparse()

template<IsModel M>

std::tuple< MDP::SparseModel, AMDP::Discretizer > AIToolbox::POMDP::AMDP::discretizeSparse

(

const M &

model

)

This function constructs an approximate sparse MDP of the provided POMDP model.

Template Parameters

M	The type of the POMDP model.

Parameters

model

The POMDP model to be approximated.

Returns

A tuple containing a sparse MDP model which approximates the POMDP argument, and a function that converts a POMDP belief into a state of the MDP model.

getBeliefSize()

size_t AIToolbox::POMDP::AMDP::getBeliefSize

(

)

const

This function returns the currently set number of sampled beliefs.

Returns

The number of sampled beliefs.

getEntropyBuckets()

size_t AIToolbox::POMDP::AMDP::getEntropyBuckets

(

)

const

This function returns the currently set number of entropy buckets.

Returns

The number of entropy buckets.

setBeliefSize()

void AIToolbox::POMDP::AMDP::setBeliefSize

(

size_t

nBeliefs

)

This function sets a new number of sampled beliefs.

Parameters

nBeliefs

The new number of sampled beliefs.

setEntropyBuckets()

void AIToolbox::POMDP::AMDP::setEntropyBuckets

(

size_t

buckets

)

This function sets the new number of buckets in which to discretize the entropy.

Parameters

buckets

The new number of buckets.

---

The documentation for this class was generated from the following file:

• include/AIToolbox/POMDP/Algorithms/AMDP.hpp