Value function computed starting from the goal

Policy pi of choosing the best action a given a value function V

Value function

Policy

POMDP is like doing MDP in a belief space