You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm trying to implement a maintenance planning algorithm using POMCP. In this context, the decision maker is mainly interested in knowing when to perform a certain action given current and historical sensor observations. In this context, there also exists a concept of terminal states. When such terminal state is reached, any further actions are irrelevant, e.g., whenever the component fails or maintenance is initiated. The concept of terminal states is also mentioned in the original POMCP paper from Silver, D., and Veness, J. (2010). In particular, their Simulate and Rollout functions take it into account.
Because of a previous issue I opened (#73), I took a closer look at the _rollout function. It seems the current stopping condition only takes the max tree-depth into account. Is this observation correct or am I missing something?
The text was updated successfully, but these errors were encountered:
POMCP implemented here doesn't have a designated concept for a terminal state. But you can achieve the same effect (in terms of value estimation, asymptotically) by defining a terminal state, such that it transitions to itself and has 0 reward. See this comment on a similar topic: #8 (comment)
I'm trying to implement a maintenance planning algorithm using POMCP. In this context, the decision maker is mainly interested in knowing when to perform a certain action given current and historical sensor observations. In this context, there also exists a concept of terminal states. When such terminal state is reached, any further actions are irrelevant, e.g., whenever the component fails or maintenance is initiated. The concept of terminal states is also mentioned in the original POMCP paper from Silver, D., and Veness, J. (2010). In particular, their
Simulate
andRollout
functions take it into account.Because of a previous issue I opened (#73), I took a closer look at the
_rollout
function. It seems the current stopping condition only takes the max tree-depth into account. Is this observation correct or am I missing something?The text was updated successfully, but these errors were encountered: