Data Processing

exception AlphaZero.processing.go.game_converter.SizeMismatchError
exception AlphaZero.processing.go.game_converter.NoResultError
exception AlphaZero.processing.go.game_converter.SearchProbsMismatchError
class AlphaZero.processing.go.game_converter.GameConverter(features)

Convert SGF files to network input feature files.

convert_game(file_name, bd_size)

Read the given SGF file into an iterable of (input,output) pairs for neural network training

Each input is a GameState converted into one-hot neural net features Each output is an action as an (x,y) pair (passes are skipped)

If this game’s size does not match bd_size, a SizeMismatchError is raised

Parameters:
  • file_name – file name
  • bd_size – board size
Returns:

neural network input, move and result
Return type:

tuple
sgfs_to_hdf5(sgf_files, hdf5_file, bd_size=19, ignore_errors=True, verbose=False)

Convert all files in the iterable sgf_files into an hdf5 group to be stored in hdf5_file.

The resulting file has the following properties:

states : dataset with shape (n_data, n_features, board width, board height)

actions : dataset with shape (n_data, 2) (actions are stored as x,y tuples of where the move was played)

results : dataset with shape (n_data, 1), +1 if current player wins, -1 otherwise

file_offsets : group mapping from filenames to tuples of (index, length)

For example, to find what positions in the dataset come from ‘test.sgf’:

index, length = file_offsets[‘test.sgf’]

test_states = states[index:index+length]

test_actions = actions[index:index+length]

Parameters:
  • sgf_files – an iterable of relative or absolute paths to SGF files
  • hdf5_file – the name of the HDF5 where features will be saved
  • bd_size – side length of board of games that are loaded
  • ignore_errors – if True, issues a Warning when there is an unknown
  • rather than halting. Note that sgf.ParseException and (exception) –
  • exceptions are always skipped (go.IllegalMove) –
  • verbose – display setting
Returns:

None
selfplay_to_hdf5(sgf_pkl_files, hdf5_file, bd_size=19, ignore_errors=True, verbose=False)

Convert all files in the iterable sgf_files into an hdf5 group to be stored in hdf5_file.

The resulting file has the following properties:

states : dataset with shape (n_data, n_features, board width, board height)

actions : dataset with shape (n_data, 2) (actions are stored as x,y tuples of where the move was played)

results : dataset with shape (n_data, 1), +1 if current player wins, -1 otherwise

file_offsets : group mapping from filenames to tuples of (index, length)

For example, to find what positions in the dataset come from ‘test.sgf’:

index, length = file_offsets[‘test.sgf’]

test_states = states[index:index+length]

test_actions = actions[index:index+length]

Parameters:
  • sgf_pkl_files – an iterable of relative or absolute paths to SGF and PKL files
  • hdf5_file – the name of the HDF5 where features will be saved
  • bd_size – side length of board of games that are loaded
  • ignore_errors – if True, issues a Warning when there is an unknown
  • rather than halting. Note that sgf.ParseException and (exception) –
  • exceptions are always skipped (go.IllegalMove) –
  • verbose – display setting
Returns:

None
AlphaZero.processing.go.game_converter.run_game_converter(cmd_line_args=None)

Run conversions.

Parameters:cmd_line_args – command-line args may be passed in as a list
Returns:None
class AlphaZero.processing.state_converter.StateTensorConverter(config, feature_list=None)

a class to convert from AlphaGo GameState objects to tensors of one-hot features for NN inputs

get_board_history(state)

A feature encoding WHITE and BLACK on separate planes of recent history_length states

Parameters:state – the game state
Returns:numpy.ndarray
state_to_tensor(state)

Convert a GameState to a Theano-compatible tensor :param state: the game state

Returns:numpy.ndarray
class AlphaZero.processing.state_converter.TensorActionConverter(config)

a class to convert output tensors from NN to action tuples

tensor_to_action(tensor)
Parameters:tensor – a 1D prob tensor with length flat_move_output
Returns:a list of (action, prob)
Return type:list
class AlphaZero.processing.state_converter.ReverseTransformer(config)
lr_reflection(action_prob)

Flips the coordinate of action probability vector like np.fliplr Modification is made in place. Note that PASS_MOVE should be placed at the end of this vector. Condition check is disabled for efficiency.

Parameters:action_prob – action probabilities
Returns:None
reverse_nprot90(action_prob, transform_id)
Reverse the coordinate transform of np.rot90 performed in go.Gamestate.transform
Rotate the coordinates by Pi/4 * id clockwise
Parameters:
  • action_prob – action probability vector
  • transform_id – argument passed to np.rot90
Returns:

None
reverse_transform(action_prob, transform_id)
Reverse the coordinates for go.GameState.transform
The function make modifications in place
Parameters:
  • action_prob – list of (action, prob)
  • transform_id – number used to perform the transform, range: [0, 7]
Returns:

None