Assignment 7 information

[12/9] There is a new version of a7code.com out, version 1.0.4. This differs from version 1.0.3 only in that the return value of the play-match is now a list where the first element is the net win/loss of the player and the second element is the total amount bet over all the hands. (The total amount bet is not necessarily the number of hands because of the double-down action.)
[12/3] The web testers are up.
[12/3] One new and one more undocumented procedure (although you wrote something similar back for Assignment 2)
- (get-reward-vector action) --- returns a list containing the avegaged observed reward for all of the "action" states starting with state number 0. This is in a7header.scm
- (corresponding-max vals objs) --- returns the element of the list objs that corresponds to the maximum value of the list of numbers vals. If there are multiple values in vals that have the same maximum value, then one of the corresponding elements from objs will be picked randomly. This is in a7code.com
Support code:
- a7code.com --- compiled support code, version 1.0.4 out 12/9
- a7header.scm --- source code and source code headers, version 1.0.2 out 12/3
- a7example.scm --- new version posted 11/27
- assign7.scm --- stubs file
[12/1] One new and one undocumented procedure:
- (soft-hand? h) --- given a hand h, it returns #t if there is an ace in the hand that is counted as 11. This procedure was in the support code from the start but was undocumented.
- (get-action-state-visits state-num action) --- returns the number of times that you have taken action from hit state state-num. This has been added to the a7header.scm file in version 1.0.1.
  To use the text's exploration function, you need this procedure instead of get-visits-element which is documented in the assignment handouts. The get-visits-element returns the number times you have visited a state, regardless of what action was taken to get you there.
  Another way to think of it is that get-visits-element records the number of times you transitioned into the "action" state state-num, while get-action-state-visits records the number of times you transitioned out of a "hit" state under the given action.
See the "handouts" page (off the main page) for the assignment 7 handouts.
If you choose to do value iteration or policy iteration, then you will need to create extra storage for the new utility values. You can look through the a7header.scm file to see the details, but here are the essentials. The utilities are stored in a Scheme vector. You can use the make-vector procedure to create a vector (see the reference manual for details), or you can do all your computations using lists. When you want to update the utility values, you can do the following with a vector:
```
  (set! HIT-UTILITIES new-utilities-vector)
```
or with a list, do the following:
```
  (set! HIT-UTILITIES (list->vector new-utilities-list))
```

Support code information

There are a few features of the support code that were promised in the two handouts (and one or two features that were not described in either handout). Here are the details:

The support code now (with version 1.0) does lots of error checking, so it should provide reasonable (and understandable) error messages if, for example, your code causes an error when called from the compiled support code.
Note that much of the implementation of the learning tables now appears in the a7header.scm file as source code.
There is a variable check-table-args (default value #t) that checks the argument values when accessing the utility, transition, and reward tables. If you turn this off, your code will run a little faster, but you should leave it on while you are debugging your transform-state procedure.
There are a number of variables that control what is printed when a hand or match is played.
- print-narration, default value is #t
  Prints a narration of each hand. If this is set to #f, nothing will be printed to the screen (except possibly the progress of the match).
- print-match-progress, default value is 10
  If print-narration is #f, this will print out a one-line message every print-match-progress hands. With the default value of 10, it will print out a message every ten hands. If you set it to any value that is not a positive integer, it will not print any messages.
- print-player-play, default value is #t
  If print-narration is #t, this will print what happens when your player plays its hand.
- print-dealer-play, default value is #t
  If print-narration is #t, this will print what happens when the dealer plays its hand.
- print-bet-settlement, default value is #t
  If print-narration is #t, this will print a summary of the hand, including whether the player won or lost.
The variable enable-table-updates (default value is #t) controls whether updates are made to the transition and reward tables. You might want to turn this off if, for example, you have finished learning your player and you want to test it. Otherwise, the transition probabilities and average rewards would be updated as it plays.
There is a procedure that will save your transform-state procedure and the learned utility, transition, and reward tables to a file:
```
  (save-learning fname)
```
The filename should be a double-quote delimited string, and you should use the extension ".scm"