Namespaces
	TigerProblemUtils

Classes
struct	ActionNode

class	AMDP
	This class implements the Augmented MDP algorithm. More...

class	BeliefGenerator
	This class generates reachable beliefs from a given Model. More...

class	BeliefNode
	This is a belief node of the rPOMCP tree. More...

struct	BeliefParticle

class	BlindStrategies
	This class implements the blind strategies lower bound. More...

class	FastInformedBound
	This class implements the Fast Informed Bound algorithm. More...

class	GapMin
	This class implements the GapMin algorithm. More...

class	HeadBeliefNode
	This class is the root node of the rPOMCP graph. More...

class	IncrementalPruning
	This class implements the Incremental Pruning algorithm. More...

struct	is_witness_lp
	This check the interface for a WitnessLP. More...

class	LinearSupport
	This class represents the LinearSupport algorithm. More...

class	Model
	This class represents a Partially Observable Markov Decision Process. More...

class	PBVI
	This class implements the Point Based Value Iteration algorithm. More...

class	PERSEUS
	This class implements the PERSEUS algorithm. More...

class	Policy
	This class represents a POMDP Policy. More...

class	POMCP
	This class represents the POMCP online planner using UCB1. More...

class	Projecter
	This class offers projecting facilities for Models. More...

class	QMDP
	This class implements the QMDP algorithm. More...

class	rPOMCP
	This class represents the rPOMCP online planner. More...

class	RTBSS
	This class represents the RTBSS online planner. More...

class	SARSOP
	This class implements the SARSOP algorithm. More...

class	SparseModel
	This class represents a Partially Observable Markov Decision Process. More...

struct	VEntry

class	Witness
	This class implements the Witness algorithm. More...

Typedefs
template<bool UseEntropy>
using	ActionNodes = std::vector< ActionNode< UseEntropy > >

template<bool UseEntropy>
using	TrackBelief = std::unordered_map< size_t, BeliefParticle< UseEntropy >, std::hash< size_t >, std::equal_to< size_t > >

template<bool UseEntropy>
using	BeliefNodes = std::unordered_map< size_t, BeliefNode< UseEntropy > >

using	SampleBelief = std::vector< std::pair< size_t, unsigned > >

using	Belief = ProbabilityVector
	This represents a belief, which is a probability distribution over states. More...

Functions
Model< MDP::Model >	makeChengD35 ()
	This function returns a POMDP model of the chengD35 problem. More...

Model< MDP::Model >	makeEJS4 ()
	This function returns a POMDP model of the ejs4 problem. More...

AIToolbox::POMDP::Model< AIToolbox::MDP::Model >	makeTigerProblem ()
	This function sets up the tiger problem in a Model. More...

Model< MDP::Model >	parseCassandra (std::istream &input)
	This function parses a POMDP from a Cassandra formatted stream. More...

template<IsModelEigen M>
std::ostream &	operator<< (std::ostream &os, const M &model)
	This function outputs a POMDP model to a stream. More...

template<MDP::IsModel M>
std::istream &	operator>> (std::istream &is, Model< M > &m)
	This function parses a Model from a stream. More...

template<MDP::IsModel M>
std::istream &	operator>> (std::istream &is, SparseModel< M > &m)
	This function implements input from stream for the POMDP::SparseModel class. More...

std::ostream &	operator<< (std::ostream &os, const Policy &p)
	This function outputs a Policy to a stream. More...

std::istream &	operator>> (std::istream &is, Policy &p)
	This function reads a policy from a file. More...

std::strong_ordering	operator<=> (const VEntry &lhs, const VEntry &rhs)
	This function lexicographically sorts VEntries. More...

bool	operator== (const VEntry &lhs, const VEntry &rhs)

VEntry	makeVEntry (size_t S, size_t a, size_t O)
	This function creates and zeroes a VEntry. More...

size_t	hash_value (const VEntry &v)
	This function enables hashing of VEntries with boost::hash. More...

const MDP::Values &	unwrap (const VEntry &ve)
	This function is used as iterator projection to obtain the Values of a VEntry. More...

ValueFunction	makeValueFunction (size_t S)
	This function creates a default ValueFunction. More...

double	weakBoundDistance (const VList &oldV, const VList &newV)
	This function returns a weak measure of distance between two VLists. More...

template<IsModel M>
auto	makeSOSA (const M &m)
	This function creates the SOSA matrix for the input POMDP. More...

template<IsModel M>
void	updateBeliefUnnormalized (const M &model, const Belief &b, const size_t a, const size_t o, Belief *bRet)
	Creates a new belief reflecting changes after an action and observation for a particular Model. More...

template<IsModel M>
Belief	updateBeliefUnnormalized (const M &model, const Belief &b, const size_t a, const size_t o)
	Creates a new belief reflecting changes after an action and observation for a particular Model. More...

template<IsModel M>
void	updateBelief (const M &model, const Belief &b, const size_t a, const size_t o, Belief *bRet)
	Creates a new belief reflecting changes after an action and observation for a particular Model. More...

template<IsModel M>
Belief	updateBelief (const M &model, const Belief &b, const size_t a, const size_t o)
	Creates a new belief reflecting changes after an action and observation for a particular Model. More...

template<IsModel M>
void	updateBeliefPartial (const M &model, const Belief &b, const size_t a, Belief *bRet)
	This function partially updates a belief. More...

template<IsModel M>
Belief	updateBeliefPartial (const M &model, const Belief &b, const size_t a)
	This function partially updates a belief. More...

template<IsModel M>
void	updateBeliefPartialUnnormalized (const M &model, const Belief &b, const size_t a, const size_t o, Belief *bRet)
	This function terminates the unnormalized update of a partially updated belief. More...

template<IsModel M>
Belief	updateBeliefPartialUnnormalized (const M &model, const Belief &b, const size_t a, const size_t o)
	This function terminates the unnormalized update of a partially updated belief. More...

template<IsModel M>
void	updateBeliefPartialNormalized (const M &model, const Belief &b, const size_t a, const size_t o, Belief *bRet)
	This function terminates the normalized update of a partially updated belief. More...

template<IsModel M>
Belief	updateBeliefPartialNormalized (const M &model, const Belief &b, const size_t a, const size_t o)
	This function terminates the normalized update of a partially updated belief. More...

template<IsModel M>
double	beliefExpectedReward (const M &model, const Belief &b, const size_t a)
	This function computes an immediate reward based on a belief rather than a state. More...

template<typename ActionRow >
void	crossSumBestAtBelief (const Belief &b, const ActionRow &row, VEntry outp, double value=nullptr)
	This function computes the best VEntry for the input belief from the input VLists. More...

template<typename ActionRow >
VEntry	crossSumBestAtBelief (const Belief &b, const ActionRow &row, const size_t a, double *value=nullptr)
	This function allocates and computes the best VEntry for the input belief from the input VLists. More...

template<typename Projections >
VEntry	crossSumBestAtBelief (const Belief &b, const Projections &projs, double *value=nullptr)
	This function computes the best VEntry for the input belief across all actions. More...

template<IsModel M>
std::tuple< size_t, double >	bestConservativeAction (const M &pomdp, MDP::QFunction immediateRewards, const Belief &initialBelief, const VList &lbVList, MDP::Values *alpha=nullptr)
	This function obtains the best action with respect to the input VList. More...

template<bool useLP = true, IsModel M>
std::tuple< size_t, double >	bestPromisingAction (const M &pomdp, const MDP::QFunction &immediateRewards, const Belief &belief, const MDP::QFunction &ubQ, const UpperBoundValueFunction &ubV, Vector *vals=nullptr)
	This function obtains the best action with respect to the input QFunction and UbV. More...

Variables
template<typename LP >
constexpr bool	is_witness_lp_v = is_witness_lp<LP>::value

template<typename M >
concept	IsGenerativeModel
	This concept represents the required interface for a generative POMDP. More...

template<typename M >
concept	IsModel
	This concept represents the required interface for a POMDP Model. More...

template<typename M >
concept	IsModelEigen
	This concept represents the required interface that allows POMDP algorithms to leverage Eigen. More...

Typedef Documentation

◆ ActionNodes

template<bool UseEntropy>

using AIToolbox::POMDP::ActionNodes = typedef std::vector<ActionNode<UseEntropy> >

◆ Belief

using AIToolbox::POMDP::Belief = typedef ProbabilityVector

This represents a belief, which is a probability distribution over states.

◆ BeliefNodes

template<bool UseEntropy>

using AIToolbox::POMDP::BeliefNodes = typedef std::unordered_map<size_t, BeliefNode<UseEntropy> >

◆ SampleBelief

using AIToolbox::POMDP::SampleBelief = typedef std::vector<std::pair<size_t, unsigned> >

◆ TrackBelief

template<bool UseEntropy>

using AIToolbox::POMDP::TrackBelief = typedef std::unordered_map< size_t, BeliefParticle<UseEntropy>, std::hash<size_t>, std::equal_to<size_t> >

◆ UpperBoundValueFunction

using AIToolbox::POMDP::UpperBoundValueFunction = typedef std::pair<std::vector<Belief>, std::vector<double> >

◆ ValueFunction

using AIToolbox::POMDP::ValueFunction = typedef std::vector<VList>

◆ VList

using AIToolbox::POMDP::VList = typedef std::vector<VEntry>

◆ VObs

using AIToolbox::POMDP::VObs = typedef std::vector<size_t>

Function Documentation

◆ beliefExpectedReward()

template<IsModel M>

double AIToolbox::POMDP::beliefExpectedReward	(	const M &	model,
		const Belief &	b,
		const size_t	a
	)

This function computes an immediate reward based on a belief rather than a state.

Parameters

model	The POMDP model to use.
b	The belief to use.
a	The action performed from the belief.

Returns: The immediate reward.

◆ bestConservativeAction()

template<IsModel M>

std::tuple<size_t, double> AIToolbox::POMDP::bestConservativeAction	(	const M &	pomdp,
		MDP::QFunction	immediateRewards,
		const Belief &	initialBelief,
		const VList &	lbVList,
		MDP::Values *	alpha = `nullptr`
	)

This function obtains the best action with respect to the input VList.

This function pretty much does what the Projecter class does. The difference is that while the Projecter expands one step in the future every single entry in the input VList, this only does it to the input Belief.

This allows to both avoid a lot of work if we wouldn't need to reuse the Projecter results a lot, and simplifies the crossSum step.

Parameters

pomdp	The model to use.
immediateRewards	The immediate rewards of the model.
initialBelief	The belief where the best action needs to be found.
lbVList	The alphavectors to use.
alpha	Optionally, the output alphavector for the best action. Does not need preallocation.

Returns: The best action in the input belief with respect to the input VList.

◆ bestPromisingAction()

template<bool useLP = true, IsModel M>

std::tuple<size_t, double> AIToolbox::POMDP::bestPromisingAction	(	const M &	pomdp,
		const MDP::QFunction &	immediateRewards,
		const Belief &	belief,
		const MDP::QFunction &	ubQ,
		const UpperBoundValueFunction &	ubV,
		Vector *	vals = `nullptr`
	)

This function obtains the best action with respect to the input QFunction and UbV.

This function simply computes the upper bound for all beliefs that can be reached from the input belief. For each action, their values are summed (after multiplying each by the probability of it happening), and the best action extracted.

Template Parameters

useLP Whether we want to use LP interpolation, rather than sawtooth. Defaults to true.

Parameters

pomdp	The model to look the action for.
immediateRewards	The immediate rewards of the model.
belief	The belief to find the best action in.
ubQ	The current QFunction for this model.
ubV	The current list of belief/values for this model.
vals	Optionally, an output vector containing the per-action upper-bound values. Does not need preallocation, and passing it does not result in more work.

Returns: The best action-value pair.

◆ crossSumBestAtBelief() [1/3]

template<typename ActionRow >

VEntry AIToolbox::POMDP::crossSumBestAtBelief	(	const Belief &	b,
		const ActionRow &	row,
		const size_t	a,
		double *	value = `nullptr`
	)

This function allocates and computes the best VEntry for the input belief from the input VLists.

This function allocates the VEntry to return, rather than expecting one already allocated.

See also: crossSumBestAtBelief(const Belief &, const ActionRow &, VEntry *, double *);

Template Parameters

ActionRow The type of the list of VLists.

Parameters

b	The belief to compute the VEntry for.
row	The list of VLists, one per observation.
a	The action this Ventry stands for.
value	A pointer to double, which gets set to the value of the given belief with the generated VEntry.

Returns: The best VEntry for the input belief.

◆ crossSumBestAtBelief() [2/3]

template<typename ActionRow >

void AIToolbox::POMDP::crossSumBestAtBelief	(	const Belief &	b,
		const ActionRow &	row,
		VEntry *	outp,
		double *	value = `nullptr`
	)

This function computes the best VEntry for the input belief from the input VLists.

This function computes the best alphavector for the input belief. It assumes as input a list of VLists, one per observation. You can produce them with the Projecter class, for example.

For each observation it will select the match with the best value, and use it to compute the output VEntry.

The action is not used to perform computations here, but it is fed directly into the returned VEntry.

Template Parameters

ActionRow The type of the list of VLists.

Parameters

b	The belief to compute the VEntry for.
row	The list of VLists, one per observation.
outp	A pointer to a pre-allocated VEntry where to set the output.
value	A pointer to double, which gets set to the value of the given belief with the generated VEntry.

◆ crossSumBestAtBelief() [3/3]

template<typename Projections >

VEntry AIToolbox::POMDP::crossSumBestAtBelief	(	const Belief &	b,
		const Projections &	projs,
		double *	value = `nullptr`
	)

This function computes the best VEntry for the input belief across all actions.

This function needs the projections of the previous timestep's VLists in order to work. It will then compute the best VEntry for the input belief across all actions.

Template Parameters

Projections The type of the 2D array of VLists containing all the projections.

Parameters

b	The belief to compute the VEntry for.
projs	The projections of the old VLists.
value	A pointer to double, which gets set to the value of the given belief with the generated VEntry.

Returns: The best VEntry for the input belief.

◆ hash_value()

size_t AIToolbox::POMDP::hash_value ( const VEntry & v )

inline

This function enables hashing of VEntries with boost::hash.

◆ makeChengD35()

Model<MDP::Model> AIToolbox::POMDP::makeChengD35 ( )

This function returns a POMDP model of the chengD35 problem.

◆ makeEJS4()

Model<MDP::Model> AIToolbox::POMDP::makeEJS4 ( )

This function returns a POMDP model of the ejs4 problem.

◆ makeSOSA()

template<IsModel M>

auto AIToolbox::POMDP::makeSOSA ( const M & m )

This function creates the SOSA matrix for the input POMDP.

The SOSA matrix is a way to represent the observation and transition functions in a single function, at the same time.

Each cell in this four-dimensional matrix contains the probability of getting to state s' while obtaining observation o when starting with state s and action a.

This matrix is less space-efficient than storing both matrices separately, but it can save you some time if you need its values multiple times in a loop (for example in the FastInformedBound algorithm).

Parameters

m	The input POMDP to extract the SOSA matrix from.

Returns: The SOSA matrix for the input pomdp.

◆ makeTigerProblem()

AIToolbox::POMDP::Model<AIToolbox::MDP::Model> AIToolbox::POMDP::makeTigerProblem ( )

inline

This function sets up the tiger problem in a Model.

This function builds the AAAI-94 Tiger problem, with a 0.95 discount factor. The problem can be stated as follows:

The agent stands in front of two doors. He can open either of them. Behind one door, there is an agent-eater tiger, and in the other a small treasure. The agent does not know to what each door leads to, but instead of just opening the door, he can listen. When he listens, it will hear the tiger from either the left or right door. Its hearing is imperfect though, and 15% of the time it will hear the tiger behind the wrong door.

Once the agent opens a door, it will either get a great penalty due to being eaten by the tiger, or get the reward. After that the game will automatically reset to an unknown state: the agent needs to start guessing again where the new tiger and treasure are.

The states here are the positions of the tiger and treasure: since there are two doors, there are two states.

There are three actions, corresponding to the listen action and open door actions.

There are two possible observations, which are always random but for the listen action: in that case, we will obtain the correct information about the true state 85% of the time.

The solutions of this problem have been computed using Tony Cassandra's pomdp-solve program (www.pomdp.org).

Returns: The Model representing the problem.

◆ makeValueFunction()

ValueFunction AIToolbox::POMDP::makeValueFunction ( size_t S )

This function creates a default ValueFunction.

The default ValueFunction contains a single VList with inside a single VEntry: do action 0, with all value zeroes.

The VList is a necessary byproduct that is needed when computing the whole ValueFunction recursively via dynamic programming.

In the end, to act, it's not needed, but it's probably more hassle to remove the entry, and so we leave it there. So in general we always assume it's there.

Another peculiarity of the default VEntry is that it's the only place where the observation id vector is empty, since nobody is ever supposed to go looking in there.

Parameters

S	The number of possible states.

Returns: A new ValueFunction.

◆ makeVEntry()

VEntry AIToolbox::POMDP::makeVEntry	(	size_t	S,
		size_t	a,
		size_t	O
	)

inline

This function creates and zeroes a VEntry.

We use this instead of making a constructor to keep VEntry as an aggregate, which makes some stuff easier.

Parameters

S	The size of the state space.
a	The action associated with this VEntry.
O	The size of the observation space.

Returns: The newly created VEntry.

◆ operator<<() [1/2]

template<IsModelEigen M>

std::ostream& AIToolbox::POMDP::operator<<	(	std::ostream &	os,
		const M &	model
	)

This function outputs a POMDP model to a stream.

Template Parameters

M	The type of the model.

Parameters

os	The output stream.
model	The model to output.

Returns: The resulting output stream.

◆ operator<<() [2/2]

std::ostream& AIToolbox::POMDP::operator<<	(	std::ostream &	os,
		const Policy &	p
	)

This function outputs a Policy to a stream.

Parameters

os	The stream where the policy is printed.
p	The policy that is begin printed.

Returns: The original stream.

◆ operator<=>()

std::strong_ordering AIToolbox::POMDP::operator<=>	(	const VEntry &	lhs,
		const VEntry &	rhs
	)

This function lexicographically sorts VEntries.

This is useful during testing in order to sort and compare a solution with the correct result.

Parameters

lhs	The left hand side of the comparison.
rhs	The right hand side of the comparison.

Returns: True if lhs is less than rhs lexicographically, false otherwise.

◆ operator==()

bool AIToolbox::POMDP::operator==	(	const VEntry &	lhs,
		const VEntry &	rhs
	)

◆ operator>>() [1/3]

template<MDP::IsModel M>

std::istream & AIToolbox::POMDP::operator>>	(	std::istream &	is,
		Model< M > &	m
	)

This function parses a Model from a stream.

This function does not modify the input model if the parsing fails.

Template Parameters

M	The underlying MDP model. Needs to have operator>> implemented.

Parameters

is	The input stream.
m	The model to write into.

Returns: The input stream.

◆ operator>>() [2/3]

std::istream& AIToolbox::POMDP::operator>>	(	std::istream &	is,
		Policy &	p
	)

This function reads a policy from a file.

This function reads files that have been outputted through operator<<(std::ostream&, const Policy&). If not enough values can be extracted from the stream, the function stops and the input policy is not modified. In addition, it checks whether the probability values are within 0 and 1.

Parameters

is	The stream were the policy is being read from.
p	The policy that is being assigned.

Returns: The input stream.

◆ operator>>() [3/3]

template<MDP::IsModel M>

std::istream & AIToolbox::POMDP::operator>>	(	std::istream &	is,
		SparseModel< M > &	m
	)

This function implements input from stream for the POMDP::SparseModel class.

Note that as all other input function, it does not actually change the input model if the reading fails.

Template Parameters

M	The underlying MDP model. Needs to have operator<< implemented.

Parameters

is	The input stream.
m	The model to write into.

Returns: The input stream.

◆ parseCassandra()

Model<MDP::Model> AIToolbox::POMDP::parseCassandra ( std::istream & input )

This function parses a POMDP from a Cassandra formatted stream.

This function may throw std::runtime_errors depending on whether the input is correctly formed or not.

Parameters

input The input stream.

Returns: The parsed model.

◆ unwrap()

const MDP::Values& AIToolbox::POMDP::unwrap ( const VEntry & ve )

inline

This function is used as iterator projection to obtain the Values of a VEntry.

◆ updateBelief() [1/2]

template<IsModel M>

Belief AIToolbox::POMDP::updateBelief	(	const M &	model,
		const Belief &	b,
		const size_t	a,
		const size_t	o
	)

Creates a new belief reflecting changes after an action and observation for a particular Model.

This function needs to create a new belief since modifying a belief in place is not possible. This is because each cell update for the new belief requires all values from the previous belief.

NOTE: This function assumes that the update and the normalization are possible, i.e. that from the input belief and action it is possible to receive the input observation.

If that cannot be guaranteed, use the updateBeliefUnnormalized() function and do the normalization yourself (and check for it).

Template Parameters

M	The type of the POMDP Model.

Parameters

model	The model used to update the belief.
b	The old belief.
a	The action taken during the transition.
o	The observation registered.

◆ updateBelief() [2/2]

template<IsModel M>

void AIToolbox::POMDP::updateBelief	(	const M &	model,
		const Belief &	b,
		const size_t	a,
		const size_t	o,
		Belief *	bRet
	)

Creates a new belief reflecting changes after an action and observation for a particular Model.

This function writes directly into the provided Belief pointer. It assumes that the pointer points to a correctly sized Belief. It does a basic nullptr check.

NOTE: This function assumes that the update and the normalization are possible, i.e. that from the input belief and action it is possible to receive the input observation.

If that cannot be guaranteed, use the updateBeliefUnnormalized() function and do the normalization yourself (and check for it).

Template Parameters

M	The type of the POMDP Model.

Parameters

model	The model used to update the belief.
b	The old belief.
a	The action taken during the transition.
o	The observation registered.
bRet	The output belief.

◆ updateBeliefPartial() [1/2]

template<IsModel M>

Belief AIToolbox::POMDP::updateBeliefPartial	(	const M &	model,
		const Belief &	b,
		const size_t	a
	)

This function partially updates a belief.

This function is useful in case one needs to update a belief for all possible observations. In such a case, it is possible to avoid repeating the same operations by creating an intermediate belief, that only depends on the action and not on the observation.

From this intermediate result it will be then possible to obtain the end belief by supplying the same action and the desired observation.

Template Parameters

M	The type of the POMDP Model.

Parameters

model	The model used to update the belief.
b	The old belief.
a	The action taken during the transition.

◆ updateBeliefPartial() [2/2]

template<IsModel M>

void AIToolbox::POMDP::updateBeliefPartial	(	const M &	model,
		const Belief &	b,
		const size_t	a,
		Belief *	bRet
	)

This function partially updates a belief.

This function is useful in case one needs to update a belief for all possible observations. In such a case, it is possible to avoid repeating the same operations by creating an intermediate belief, that only depends on the action and not on the observation.

From this intermediate result it will be then possible to obtain the end belief by supplying the same action and the desired observation.

Template Parameters

M	The type of the POMDP Model.

Parameters

model	The model used to update the belief.
b	The old belief.
a	The action taken during the transition.
bRet	The output belief.

◆ updateBeliefPartialNormalized() [1/2]

template<IsModel M>

Belief AIToolbox::POMDP::updateBeliefPartialNormalized	(	const M &	model,
		const Belief &	b,
		const size_t	a,
		const size_t	o
	)

This function terminates the normalized update of a partially updated belief.

This function is useful in case one needs to update a belief for all possible observations. In such a case, it is possible to avoid repeating the same operations by creating an intermediate belief, that only depends on the action and not on the observation.

See also: updateBeliefPartial

Note that the input action here must be the same one that produced the intermediate result.

NOTE: This function assumes that the update and the normalization are possible, i.e. that from the input belief and action it is possible to receive the input observation.

If that cannot be guaranteed, use the updateBeliefUnnormalized() function and do the normalization yourself (and check for it).

Template Parameters

M	The type of the POMDP Model.

Parameters

model	The model used to update the belief.
b	The intermediate belief.
a	The action taken during the transition.
o	The observation registered.

◆ updateBeliefPartialNormalized() [2/2]

template<IsModel M>

void AIToolbox::POMDP::updateBeliefPartialNormalized	(	const M &	model,
		const Belief &	b,
		const size_t	a,
		const size_t	o,
		Belief *	bRet
	)

This function terminates the normalized update of a partially updated belief.

This function is useful in case one needs to update a belief for all possible observations. In such a case, it is possible to avoid repeating the same operations by creating an intermediate belief, that only depends on the action and not on the observation.

See also: updateBeliefPartial

Note that the input action here must be the same one that produced the intermediate result.

NOTE: This function assumes that the update and the normalization are possible, i.e. that from the input belief and action it is possible to receive the input observation.

If that cannot be guaranteed, use the updateBeliefUnnormalized() function and do the normalization yourself (and check for it).

Template Parameters

M	The type of the POMDP Model.

Parameters

model	The model used to update the belief.
b	The intermediate belief.
a	The action taken during the transition.
o	The observation registered.
bRet	The output belief.

◆ updateBeliefPartialUnnormalized() [1/2]

template<IsModel M>

Belief AIToolbox::POMDP::updateBeliefPartialUnnormalized	(	const M &	model,
		const Belief &	b,
		const size_t	a,
		const size_t	o
	)

This function terminates the unnormalized update of a partially updated belief.

This function is useful in case one needs to update a belief for all possible observations. In such a case, it is possible to avoid repeating the same operations by creating an intermediate belief, that only depends on the action and not on the observation.

See also: updateBeliefPartial

Note that the input action here must be the same one that produced the intermediate result.

Template Parameters

M	The type of the POMDP Model.

Parameters

model	The model used to update the belief.
b	The intermediate belief.
a	The action taken during the transition.
o	The observation registered.

◆ updateBeliefPartialUnnormalized() [2/2]

template<IsModel M>

void AIToolbox::POMDP::updateBeliefPartialUnnormalized	(	const M &	model,
		const Belief &	b,
		const size_t	a,
		const size_t	o,
		Belief *	bRet
	)

This function terminates the unnormalized update of a partially updated belief.

This function is useful in case one needs to update a belief for all possible observations. In such a case, it is possible to avoid repeating the same operations by creating an intermediate belief, that only depends on the action and not on the observation.

See also: updateBeliefPartial

Note that the input action here must be the same one that produced the intermediate result.

Template Parameters

M	The type of the POMDP Model.

Parameters

model	The model used to update the belief.
b	The intermediate belief.
a	The action taken during the transition.
o	The observation registered.
bRet	The output belief.

◆ updateBeliefUnnormalized() [1/2]

template<IsModel M>

Belief AIToolbox::POMDP::updateBeliefUnnormalized	(	const M &	model,
		const Belief &	b,
		const size_t	a,
		const size_t	o
	)

Creates a new belief reflecting changes after an action and observation for a particular Model.

This function needs to create a new belief since modifying a belief in place is not possible. This is because each cell update for the new belief requires all values from the previous belief.

This function will not normalize the output, nor is guaranteed to return a non-completely-zero vector.

Template Parameters

M	The type of the POMDP Model.

Parameters

model	The model used to update the belief.
b	The old belief.
a	The action taken during the transition.
o	The observation registered.

◆ updateBeliefUnnormalized() [2/2]

template<IsModel M>

void AIToolbox::POMDP::updateBeliefUnnormalized	(	const M &	model,
		const Belief &	b,
		const size_t	a,
		const size_t	o,
		Belief *	bRet
	)

Creates a new belief reflecting changes after an action and observation for a particular Model.

This function writes directly into the provided Belief pointer. It assumes that the pointer points to a correctly sized Belief. It does a basic nullptr check.

This function will not normalize the output, nor is guaranteed to return a non-completely-zero vector.

Template Parameters

M	The type of the POMDP Model.

Parameters

model	The model used to update the belief.
b	The old belief.
a	The action taken during the transition.
o	The observation registered.
bRet	The output belief.

◆ weakBoundDistance()

double AIToolbox::POMDP::weakBoundDistance	(	const VList &	oldV,
		const VList &	newV
	)

This function returns a weak measure of distance between two VLists.

The logic of the weak bound is the following: the variation between the old VList and the new one is equal to the maximum distance between a ValueFunction in the old VList with its closest match in the new VList. So the farthest from closest.

We define distance between two ValueFunctions as the maximum between their element-wise difference.

Parameters

oldV	The fist VList to compare.
newV	The second VList to compare.

Returns: The weak bound distance between the two arguments.

Variable Documentation

◆ is_witness_lp_v

template<typename LP >

constexpr bool AIToolbox::POMDP::is_witness_lp_v = is_witness_lp<LP>::value

inlineconstexpr

◆ IsGenerativeModel

template<typename M >

concept AIToolbox::POMDP::IsGenerativeModel

Initial value:

= MDP::IsGenerativeModel<M> && requires (M m) {
        { m.sampleSOR(m.getS(), m.getA()) } -> std::convertible_to<std::tuple<size_t, size_t, double>>;
    }

This concept represents the required interface for a generative POMDP.

This concept tests for the interface of a generative POMDP model. The interface must be implemented and be public in the parameter class. The interface is the following:

std::tuple<size_t, size_t, double> sampleSOR(size_t s, size_t a) const : Returns a sampled state-observation-reward tuple from (s,a)

In addition the POMDP needs to respect the interface for the MDP generative model.

See also: MDP::IsGenerativeModel

Note that, at least for now, we can avoid asking this generative model for the total number of observation possible, because they are not required as parameters for the functions, but just returned. This may change in future though, depending on algorithms' requirements.

◆ IsModel

template<typename M >

concept AIToolbox::POMDP::IsModel

Initial value:

= MDP::IsModel<M> && IsGenerativeModel<M> && HasIntegralObservationSpace<M> && requires (M m, size_t s, size_t a) {
        { m.getObservationProbability(s, a, s) } -> std::convertible_to<double>;
    }

This concept represents the required interface for a POMDP Model.

This struct tests for the interface of a POMDP model.

The interface must be implemented and be public in the parameter class. The interface is the following:

size_t getO() const : Returns the number of observations of the Model.
double getObservationProbability(size_t s1, size_t a, size_t o) const : Returns the probability for observation o after action a and final state s1.

In addition the POMDP needs to respect the interface for the POMDP generative model and the MDP model.

See also: IsGenerativeModel; MDP::IsModel; HasIntegralObservationSpace

◆ IsModelEigen

template<typename M >

concept AIToolbox::POMDP::IsModelEigen

Initial value:

= MDP::IsModelEigen<M> && IsModel<M> && requires (M m, size_t a) {
        m.getObservationFunction(a);
        requires IsDerivedFromEigen<std::remove_cvref_t<decltype((m.getObservationFunction(a)))>>;
    }

This concept represents the required interface that allows POMDP algorithms to leverage Eigen.

This concept tests for the interface of a POMDP model which uses Eigen matrices internally.

The interface must be implemented and be public in the parameter class. The interface is the following:

O getObservationFunction(size_t a) const : Returns the observation function for a given action as a matrix S'xO, where O is some Eigen matrix type.

In addition the POMDP needs to respect the interface for the POMDP model and the Eigen MDP model.

See also: POMDP::IsModel; MDP::IsModelEigen

Namespaces

Classes

Typedefs

Functions

Variables

Typedef Documentation

◆ ActionNodes

◆ Belief

◆ BeliefNodes

◆ SampleBelief

◆ TrackBelief

◆ UpperBoundValueFunction

◆ ValueFunction

◆ VList

◆ VObs

Function Documentation

◆ beliefExpectedReward()

◆ bestConservativeAction()

◆ bestPromisingAction()

◆ crossSumBestAtBelief() [1/3]

◆ crossSumBestAtBelief() [2/3]

◆ crossSumBestAtBelief() [3/3]

◆ hash_value()

◆ makeChengD35()

◆ makeEJS4()

◆ makeSOSA()

◆ makeTigerProblem()

◆ makeValueFunction()

◆ makeVEntry()

◆ operator<<() [1/2]

◆ operator<<() [2/2]

◆ operator<=>()

◆ operator==()

◆ operator>>() [1/3]

◆ operator>>() [2/3]

◆ operator>>() [3/3]

◆ parseCassandra()

◆ unwrap()

◆ updateBelief() [1/2]

◆ updateBelief() [2/2]

◆ updateBeliefPartial() [1/2]

◆ updateBeliefPartial() [2/2]

◆ updateBeliefPartialNormalized() [1/2]

◆ updateBeliefPartialNormalized() [2/2]

◆ updateBeliefPartialUnnormalized() [1/2]

◆ updateBeliefPartialUnnormalized() [2/2]

◆ updateBeliefUnnormalized() [1/2]

◆ updateBeliefUnnormalized() [2/2]

◆ weakBoundDistance()

Variable Documentation

◆ is_witness_lp_v

◆ IsGenerativeModel

◆ IsModel

◆ IsModelEigen