mirror of
https://github.com/opelly27/Stockfish.git
synced 2026-05-20 10:57:43 +00:00
New NNUE architecture and net
Introduces a new NNUE network architecture and associated network parameters The summary of the changes: * Position for each perspective mirrored such that the king is on e..h files. Cuts the feature transformer size in half, while preserving enough knowledge to be good. See https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY/edit#heading=h.b40q4rb1w7on. * The number of neurons after the feature transformer increased two-fold, to 1024x2. This is possibly mostly due to the now very optimized feature transformer update code. * The number of neurons after the second layer is reduced from 16 to 8, to reduce the speed impact. This, perhaps surprisingly, doesn't harm the strength much. See https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY/edit#heading=h.6qkocr97fezq The AffineTransform code did not work out-of-the box with the smaller number of neurons after the second layer, so some temporary changes have been made to add a special case for InputDimensions == 8. Also additional 0 padding is added to the output for some archs that cannot process inputs by <=8 (SSE2, NEON). VNNI uses an implementation that can keep all outputs in the registers while reducing the number of loads by 3 for each 16 inputs, thanks to the reduced number of output neurons. However GCC is particularily bad at optimization here (and perhaps why the current way the affine transform is done even passed sprt) (see https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY/edit# for details) and more work will be done on this in the following days. I expect the current VNNI implementation to be improved and extended to other architectures. The network was trained with a slightly modified version of the pytorch trainer (https://github.com/glinscott/nnue-pytorch); the changes are in https://github.com/glinscott/nnue-pytorch/pull/143 The training utilized 2 datasets. dataset A - https://drive.google.com/file/d/1VlhnHL8f-20AXhGkILujnNXHwy9T-MQw/view?usp=sharing dataset B - as described in https://github.com/official-stockfish/Stockfish/commit/ba01f4b95448bcb324755f4dd2a632a57c6e67bc The training process was as following: train on dataset A for 350 epochs, take the best net in terms of elo at 20k nodes per move (it's fine to take anything from later stages of training). convert the .ckpt to .pt --resume-from-model from the .pt file, train on dataset B for <600 epochs, take the best net. Lambda=0.8, applied before the loss function. The first training command: python3 train.py \ ../nnue-pytorch-training/data/large_gensfen_multipvdiff_100_d9.binpack \ ../nnue-pytorch-training/data/large_gensfen_multipvdiff_100_d9.binpack \ --gpus "$3," \ --threads 1 \ --num-workers 1 \ --batch-size 16384 \ --progress_bar_refresh_rate 20 \ --smart-fen-skipping \ --random-fen-skipping 3 \ --features=HalfKAv2_hm^ \ --lambda=1.0 \ --max_epochs=600 \ --default_root_dir ../nnue-pytorch-training/experiment_$1/run_$2 The second training command: python3 serialize.py \ --features=HalfKAv2_hm^ \ ../nnue-pytorch-training/experiment_131/run_6/default/version_0/checkpoints/epoch-499.ckpt \ ../nnue-pytorch-training/experiment_$1/base/base.pt python3 train.py \ ../nnue-pytorch-training/data/michael_commit_b94a65.binpack \ ../nnue-pytorch-training/data/michael_commit_b94a65.binpack \ --gpus "$3," \ --threads 1 \ --num-workers 1 \ --batch-size 16384 \ --progress_bar_refresh_rate 20 \ --smart-fen-skipping \ --random-fen-skipping 3 \ --features=HalfKAv2_hm^ \ --lambda=0.8 \ --max_epochs=600 \ --resume-from-model ../nnue-pytorch-training/experiment_$1/base/base.pt \ --default_root_dir ../nnue-pytorch-training/experiment_$1/run_$2 STC: https://tests.stockfishchess.org/tests/view/611120b32a8a49ac5be798c4 LLR: 2.97 (-2.94,2.94) <-0.50,2.50> Total: 22480 W: 2434 L: 2251 D: 17795 Ptnml(0-2): 101, 1736, 7410, 1865, 128 LTC: https://tests.stockfishchess.org/tests/view/611152b32a8a49ac5be798ea LLR: 2.93 (-2.94,2.94) <0.50,3.50> Total: 9776 W: 442 L: 333 D: 9001 Ptnml(0-2): 5, 295, 4180, 402, 6 closes https://github.com/official-stockfish/Stockfish/pull/3646 bench: 5189338
This commit is contained in:
Tomasz Sobczyk
committed by
Joost VandeVondele
Joost VandeVondele
parent
dabaf2220f
commit
d61d38586e
@@ -0,0 +1,122 @@
|
||||
/*
|
||||
Stockfish, a UCI chess playing engine derived from Glaurung 2.1
|
||||
Copyright (C) 2004-2021 The Stockfish developers (see AUTHORS file)
|
||||
|
||||
Stockfish is free software: you can redistribute it and/or modify
|
||||
it under the terms of the GNU General Public License as published by
|
||||
the Free Software Foundation, either version 3 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
Stockfish is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
GNU General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU General Public License
|
||||
along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
//Definition of input features HalfKP of NNUE evaluation function
|
||||
|
||||
#ifndef NNUE_FEATURES_HALF_KA_V2_HM_H_INCLUDED
|
||||
#define NNUE_FEATURES_HALF_KA_V2_HM_H_INCLUDED
|
||||
|
||||
#include "../nnue_common.h"
|
||||
|
||||
#include "../../evaluate.h"
|
||||
#include "../../misc.h"
|
||||
|
||||
namespace Stockfish {
|
||||
struct StateInfo;
|
||||
}
|
||||
|
||||
namespace Stockfish::Eval::NNUE::Features {
|
||||
|
||||
// Feature HalfKAv2_hm: Combination of the position of own king
|
||||
// and the position of pieces. Position mirrored such that king always on e..h files.
|
||||
class HalfKAv2_hm {
|
||||
|
||||
// unique number for each piece type on each square
|
||||
enum {
|
||||
PS_NONE = 0,
|
||||
PS_W_PAWN = 0,
|
||||
PS_B_PAWN = 1 * SQUARE_NB,
|
||||
PS_W_KNIGHT = 2 * SQUARE_NB,
|
||||
PS_B_KNIGHT = 3 * SQUARE_NB,
|
||||
PS_W_BISHOP = 4 * SQUARE_NB,
|
||||
PS_B_BISHOP = 5 * SQUARE_NB,
|
||||
PS_W_ROOK = 6 * SQUARE_NB,
|
||||
PS_B_ROOK = 7 * SQUARE_NB,
|
||||
PS_W_QUEEN = 8 * SQUARE_NB,
|
||||
PS_B_QUEEN = 9 * SQUARE_NB,
|
||||
PS_KING = 10 * SQUARE_NB,
|
||||
PS_NB = 11 * SQUARE_NB
|
||||
};
|
||||
|
||||
static constexpr IndexType PieceSquareIndex[COLOR_NB][PIECE_NB] = {
|
||||
// convention: W - us, B - them
|
||||
// viewed from other side, W and B are reversed
|
||||
{ PS_NONE, PS_W_PAWN, PS_W_KNIGHT, PS_W_BISHOP, PS_W_ROOK, PS_W_QUEEN, PS_KING, PS_NONE,
|
||||
PS_NONE, PS_B_PAWN, PS_B_KNIGHT, PS_B_BISHOP, PS_B_ROOK, PS_B_QUEEN, PS_KING, PS_NONE },
|
||||
{ PS_NONE, PS_B_PAWN, PS_B_KNIGHT, PS_B_BISHOP, PS_B_ROOK, PS_B_QUEEN, PS_KING, PS_NONE,
|
||||
PS_NONE, PS_W_PAWN, PS_W_KNIGHT, PS_W_BISHOP, PS_W_ROOK, PS_W_QUEEN, PS_KING, PS_NONE }
|
||||
};
|
||||
|
||||
// Orient a square according to perspective (rotates by 180 for black)
|
||||
static Square orient(Color perspective, Square s, Square ksq);
|
||||
|
||||
// Index of a feature for a given king position and another piece on some square
|
||||
static IndexType make_index(Color perspective, Square s, Piece pc, Square ksq);
|
||||
|
||||
public:
|
||||
// Feature name
|
||||
static constexpr const char* Name = "HalfKAv2_hm(Friend)";
|
||||
|
||||
// Hash value embedded in the evaluation file
|
||||
static constexpr std::uint32_t HashValue = 0x7f234cb8u;
|
||||
|
||||
// Number of feature dimensions
|
||||
static constexpr IndexType Dimensions =
|
||||
static_cast<IndexType>(SQUARE_NB) * static_cast<IndexType>(PS_NB) / 2;
|
||||
|
||||
static constexpr int KingBuckets[64] = {
|
||||
-1, -1, -1, -1, 31, 30, 29, 28,
|
||||
-1, -1, -1, -1, 27, 26, 25, 24,
|
||||
-1, -1, -1, -1, 23, 22, 21, 20,
|
||||
-1, -1, -1, -1, 19, 18, 17, 16,
|
||||
-1, -1, -1, -1, 15, 14, 13, 12,
|
||||
-1, -1, -1, -1, 11, 10, 9, 8,
|
||||
-1, -1, -1, -1, 7, 6, 5, 4,
|
||||
-1, -1, -1, -1, 3, 2, 1, 0
|
||||
};
|
||||
|
||||
// Maximum number of simultaneously active features.
|
||||
static constexpr IndexType MaxActiveDimensions = 32;
|
||||
|
||||
// Get a list of indices for active features
|
||||
static void append_active_indices(
|
||||
const Position& pos,
|
||||
Color perspective,
|
||||
ValueListInserter<IndexType> active);
|
||||
|
||||
// Get a list of indices for recently changed features
|
||||
static void append_changed_indices(
|
||||
Square ksq,
|
||||
StateInfo* st,
|
||||
Color perspective,
|
||||
ValueListInserter<IndexType> removed,
|
||||
ValueListInserter<IndexType> added);
|
||||
|
||||
// Returns the cost of updating one perspective, the most costly one.
|
||||
// Assumes no refresh needed.
|
||||
static int update_cost(StateInfo* st);
|
||||
static int refresh_cost(const Position& pos);
|
||||
|
||||
// Returns whether the change stored in this StateInfo means that
|
||||
// a full accumulator refresh is required.
|
||||
static bool requires_refresh(StateInfo* st, Color perspective);
|
||||
};
|
||||
|
||||
} // namespace Stockfish::Eval::NNUE::Features
|
||||
|
||||
#endif // #ifndef NNUE_FEATURES_HALF_KA_V2_HM_H_INCLUDED
|
||||
Reference in New Issue
Block a user