Commit Graph

5880 Commits

Author SHA1 Message Date
nodchip ae045e2cd8 Merge pull request #258 from kennyfrc/stockfish-nnue-2020-08-30-macos
mac-compatible makefile with instructions for stockfish-nnue-2020-08-30
2020-12-09 08:39:36 +09:00
Kenn Costales 055f907315 Merge branch 'master' into stockfish-nnue-2020-08-30-macos 2020-12-08 22:49:11 +08:00
kennyfrc bb26ce5aa1 mac specific makefile with compilation instructions 2020-12-08 22:14:18 +08:00
Tomasz Sobczyk 3a1bd1185f Add binpack coarse shuffle tool. 2020-12-06 19:08:52 +09:00
Vizvezdenec 8630d03dd4 Add comments to uncommented parts of code
https://github.com/official-stockfish/Stockfish/pull/3250

No functional change
2020-12-05 16:58:42 +01:00
Vizvezdenec be7a03a957 Introduce static history
The idea of this patch can be described as following: we update static
history stats based on comparison of the static evaluations of the
position before and after the move. If the move increases static evaluation
it's assigned positive bonus, if it decreases static evaluation
it's assigned negative bonus. These stats are used in movepicker
to sort quiet moves.

passed STC
https://tests.stockfishchess.org/tests/view/5fca4c0842a050a89f02cd66
LLR: 3.00 (-2.94,2.94) {-0.25,1.25}
Total: 78152 W: 7409 L: 7171 D: 63572
Ptnml(0-2): 303, 5695, 26873, 5871, 334

passed LTC
https://tests.stockfishchess.org/tests/view/5fca6be442a050a89f02cd75
LLR: 2.94 (-2.94,2.94) {0.25,1.25}
Total: 40240 W: 1602 L: 1441 D: 37197
Ptnml(0-2): 19, 1306, 17305, 1475, 15

closes https://github.com/official-stockfish/Stockfish/pull/3253

bench 3845156
2020-12-05 16:48:33 +01:00
Tomasz Sobczyk 28d6d7cb03 Avoid computing gradient for validation loss. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk fafb9557a8 Get train loss from update_parameters. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 4eb0e77a2a Store references instead of copying the results of intermediate autograd computations. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 6cd0b03098 Add some comments regarding the current state of autograd loss computation. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 99cb869db3 Reintroduce use_wdl. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk cf6bc7ecaf Cleanup around get_loss 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 256c4b55ec Properly apply gradient norm clipping after it's scaled in the update_parameters. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk de675e3503 Reintroduce optional scaling of the teacher signal. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 01ae7b1e2c Simplify passing constants that may vary between calls. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk cbd973fdaa Detect constant expressions in autograd and return 0 grad early. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk e975889132 Move cross_entropy calculation to a separate function. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 891abf5511 Make the autograd loss expression chain thread_local. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 8adf00ae6e Identify a single evalation chain by ID in autograd to prevent cache reuse for subsequent evaluations of the same expression tree. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk cb812c742c Add [[nodiscard]] attributes to autograd functions. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 26f19e1429 Make automatic differentiation node types constexpr. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk aec6017195 When forming an autograd expression only copy parts that are rvalue references, store references to lvalues. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk a5c20bee5b Apply gradient clipping. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk d103867558 Add memoization to the autograd expression evaluator. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk aa55692b97 Cross entropy loss. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 539bd2d1c8 Replace the old loss/grad calculation completely. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk b71d1e8620 Pass the new loss function to update_parameters 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 5a58eb803a Loss func with autograd 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 541fb8177a More utility in autograd. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 6ce0245787 Basic autograd 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 1322a9a5fd Prevent false sharing of num_calls counter in the shared input trainer. Fix current_operation not being local to the executing thread. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 2aa7f5290e Fix comparison of integers with different signedness. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk a97b65eaef Fix compilation error with USE_BLAS 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 622e0b14c2 Remove superfluous example shuffling. Shuffling now only happens on reading. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 34510dd08a Remove used examples asyncronously. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 0bee8fef64 Don't unnecessarily copy the batch part. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk e954b14196 Prefetch weights for feature transformer backprop to shared cache. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 8009973381 Special case for alpha=1 in saxpy, slight performance increase. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 49b2dcb1f3 Preallocate memory for unique_features. Keep the training_features temporary buffer as a thread_local so we reuse the storage. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 1c8495b54b Remove handwritten saxpy because compilers optimize the second look anyway. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 15c528ca7b Prepare feature transformer learner. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk a3c78691a2 Prepare input slice trainer. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 401fc0fbab Prepare clipped relu trainer. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 774b023641 Add chunked for each with workers. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk cc11375f6d Skeleton for new evaluate learner 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 0d4b803b08 Prepare trainer affine transform. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 4ea8572b6d Add single threaded sgemm. 2020-11-30 08:54:53 +09:00
SFisGOD 7364006757 Update default net to nn-62ef826d1a6d.nnue
Include scaling change as suggested by Dietrich Kappe,
the one who trained net for Komodo.  According to him,
some nets may require different scaling in order to utilize its full strength.

STC:
LLR: 2.93 (-2.94,2.94) {-0.25,1.25}
Total: 99856 W: 9669 L: 9401 D: 80786
Ptnml(0-2): 374, 7468, 34037, 7614, 435
https://tests.stockfishchess.org/tests/view/5fc2697642a050a89f02c8ec

LTC:
LLR: 2.96 (-2.94,2.94) {0.25,1.25}
Total: 29840 W: 1220 L: 1081 D: 27539
Ptnml(0-2): 10, 969, 12827, 1100, 14
https://tests.stockfishchess.org/tests/view/5fc2ea5142a050a89f02c957

Bench: 3561701
2020-11-29 16:54:06 +01:00
Unai Corzo 2442ba2b0e Reductions simplification
Simplify increase reduction for captures/promotions if late move and at low depth.

STC https://tests.stockfishchess.org/tests/view/5fbff65067cbf42301d6b3ae
LLR: 2.97 (-2.94,2.94) {-1.25,0.25}
Total: 49088 W: 4607 L: 4555 D: 39926
Ptnml(0-2): 177, 3615, 16932, 3619, 201

LTC https://tests.stockfishchess.org/tests/view/5fc0902967cbf42301d6b3fc
LLR: 2.99 (-2.94,2.94) {-0.75,0.25}
Total: 160944 W: 6153 L: 6193 D: 148598
Ptnml(0-2): 90, 5525, 69294, 5461, 102

closes https://github.com/official-stockfish/Stockfish/pull/3248

bench: 3834568
2020-11-29 16:52:51 +01:00
syzygy1 045728a7da Remove piece lists
This patch removes the incrementally updated piece lists from the Position object.

This has been tried before but always failed. My reasons for trying again are:

* 32-bit systems (including phones) are now much less important than they were some years ago (and are absent from fishtest);
* NNUE may have made SF less finely tuned to the order in which moves were generated.

STC:
LLR: 2.94 (-2.94,2.94) {-1.25,0.25}
Total: 55272 W: 5260 L: 5216 D: 44796
Ptnml(0-2): 208, 4147, 18898, 4159, 224
https://tests.stockfishchess.org/tests/view/5fc2986a42a050a89f02c926

LTC:
LLR: 2.96 (-2.94,2.94) {-0.75,0.25}
Total: 16600 W: 673 L: 608 D: 15319
Ptnml(0-2): 14, 533, 7138, 604, 11
https://tests.stockfishchess.org/tests/view/5fc2f98342a050a89f02c95c

closes https://github.com/official-stockfish/Stockfish/pull/3247

Bench: 3940967
2020-11-29 16:51:01 +01:00