Utility

• The resulting states of an action can be given utility values.
• The simplest version of this is with money.
• So, with the armed bandits, you think of slot machines.
• They return a certain amount of money if you get a certain result.
• 2 pounds is the utility of getting three cherries.
• Of course, utility is often hard to judge.
• What is the utility of a burger and fries?
• You can form relationships between two things (e.g. burger < burger and fries), but even that isn't constant.
• If you do something and get the utility of the result, you can predict the utility of an action by trying it over and over again.
• If the values can change (probability and utility) there is another exploration exploitation trade off.
• If you don't know the values, there is an exploration exploitation trade off.