Characterizing and Modifying User Experience of Computing Environments Based on Behavior Logs

There is described a method and system in an interactive computing environment modified with user experience values based on behavior logs. An experience valuation system determines an experience value and an estimated experience value. The experience value is based on a current state of interaction data from a user session, based on a history of past events, and an estimation function defined by parameters to model the user experience values. The estimated experience value is determined based on, in addition to the current state and the estimation function, next states associated with the current state, and a reward function. The parameters of the estimation function are updated based on a comparison of the expected experience value and the estimated experience value. For another aspect, the method and system may further include a state prediction system to determine probabilities of transitioning that may be applied to determine the estimated experience value.