3a1bd1185f
Add binpack coarse shuffle tool.
2020-12-06 19:08:52 +09:00
8630d03dd4
Add comments to uncommented parts of code
...
https://github.com/official-stockfish/Stockfish/pull/3250
No functional change
2020-12-05 16:58:42 +01:00
be7a03a957
Introduce static history
...
The idea of this patch can be described as following: we update static
history stats based on comparison of the static evaluations of the
position before and after the move. If the move increases static evaluation
it's assigned positive bonus, if it decreases static evaluation
it's assigned negative bonus. These stats are used in movepicker
to sort quiet moves.
passed STC
https://tests.stockfishchess.org/tests/view/5fca4c0842a050a89f02cd66
LLR: 3.00 (-2.94,2.94) {-0.25,1.25}
Total: 78152 W: 7409 L: 7171 D: 63572
Ptnml(0-2): 303, 5695, 26873, 5871, 334
passed LTC
https://tests.stockfishchess.org/tests/view/5fca6be442a050a89f02cd75
LLR: 2.94 (-2.94,2.94) {0.25,1.25}
Total: 40240 W: 1602 L: 1441 D: 37197
Ptnml(0-2): 19, 1306, 17305, 1475, 15
closes https://github.com/official-stockfish/Stockfish/pull/3253
bench 3845156
2020-12-05 16:48:33 +01:00
28d6d7cb03
Avoid computing gradient for validation loss.
2020-12-02 08:56:20 +09:00
fafb9557a8
Get train loss from update_parameters.
2020-12-02 08:56:20 +09:00
4eb0e77a2a
Store references instead of copying the results of intermediate autograd computations.
2020-12-02 08:56:20 +09:00
6cd0b03098
Add some comments regarding the current state of autograd loss computation.
2020-12-02 08:56:20 +09:00
99cb869db3
Reintroduce use_wdl.
2020-12-02 08:56:20 +09:00
cf6bc7ecaf
Cleanup around get_loss
2020-12-02 08:56:20 +09:00
256c4b55ec
Properly apply gradient norm clipping after it's scaled in the update_parameters.
2020-12-02 08:56:20 +09:00
de675e3503
Reintroduce optional scaling of the teacher signal.
2020-12-02 08:56:20 +09:00
01ae7b1e2c
Simplify passing constants that may vary between calls.
2020-12-02 08:56:20 +09:00
cbd973fdaa
Detect constant expressions in autograd and return 0 grad early.
2020-12-02 08:56:20 +09:00
e975889132
Move cross_entropy calculation to a separate function.
2020-12-02 08:56:20 +09:00
891abf5511
Make the autograd loss expression chain thread_local.
2020-12-02 08:56:20 +09:00
8adf00ae6e
Identify a single evalation chain by ID in autograd to prevent cache reuse for subsequent evaluations of the same expression tree.
2020-12-02 08:56:20 +09:00
cb812c742c
Add [[nodiscard]] attributes to autograd functions.
2020-12-02 08:56:20 +09:00
26f19e1429
Make automatic differentiation node types constexpr.
2020-12-02 08:56:20 +09:00
aec6017195
When forming an autograd expression only copy parts that are rvalue references, store references to lvalues.
2020-12-02 08:56:20 +09:00
a5c20bee5b
Apply gradient clipping.
2020-12-02 08:56:20 +09:00
d103867558
Add memoization to the autograd expression evaluator.
2020-12-02 08:56:20 +09:00
aa55692b97
Cross entropy loss.
2020-12-02 08:56:20 +09:00
539bd2d1c8
Replace the old loss/grad calculation completely.
2020-12-02 08:56:20 +09:00
b71d1e8620
Pass the new loss function to update_parameters
2020-12-02 08:56:20 +09:00
5a58eb803a
Loss func with autograd
2020-12-02 08:56:20 +09:00
541fb8177a
More utility in autograd.
2020-12-02 08:56:20 +09:00
6ce0245787
Basic autograd
2020-12-02 08:56:20 +09:00
1322a9a5fd
Prevent false sharing of num_calls counter in the shared input trainer. Fix current_operation not being local to the executing thread.
2020-11-30 08:54:53 +09:00
2aa7f5290e
Fix comparison of integers with different signedness.
2020-11-30 08:54:53 +09:00
a97b65eaef
Fix compilation error with USE_BLAS
2020-11-30 08:54:53 +09:00
622e0b14c2
Remove superfluous example shuffling. Shuffling now only happens on reading.
2020-11-30 08:54:53 +09:00
34510dd08a
Remove used examples asyncronously.
2020-11-30 08:54:53 +09:00
0bee8fef64
Don't unnecessarily copy the batch part.
2020-11-30 08:54:53 +09:00
e954b14196
Prefetch weights for feature transformer backprop to shared cache.
2020-11-30 08:54:53 +09:00
8009973381
Special case for alpha=1 in saxpy, slight performance increase.
2020-11-30 08:54:53 +09:00
49b2dcb1f3
Preallocate memory for unique_features. Keep the training_features temporary buffer as a thread_local so we reuse the storage.
2020-11-30 08:54:53 +09:00
1c8495b54b
Remove handwritten saxpy because compilers optimize the second look anyway.
2020-11-30 08:54:53 +09:00
15c528ca7b
Prepare feature transformer learner.
2020-11-30 08:54:53 +09:00
a3c78691a2
Prepare input slice trainer.
2020-11-30 08:54:53 +09:00
401fc0fbab
Prepare clipped relu trainer.
2020-11-30 08:54:53 +09:00
774b023641
Add chunked for each with workers.
2020-11-30 08:54:53 +09:00
cc11375f6d
Skeleton for new evaluate learner
2020-11-30 08:54:53 +09:00
0d4b803b08
Prepare trainer affine transform.
2020-11-30 08:54:53 +09:00
4ea8572b6d
Add single threaded sgemm.
2020-11-30 08:54:53 +09:00
7364006757
Update default net to nn-62ef826d1a6d.nnue
...
Include scaling change as suggested by Dietrich Kappe,
the one who trained net for Komodo. According to him,
some nets may require different scaling in order to utilize its full strength.
STC:
LLR: 2.93 (-2.94,2.94) {-0.25,1.25}
Total: 99856 W: 9669 L: 9401 D: 80786
Ptnml(0-2): 374, 7468, 34037, 7614, 435
https://tests.stockfishchess.org/tests/view/5fc2697642a050a89f02c8ec
LTC:
LLR: 2.96 (-2.94,2.94) {0.25,1.25}
Total: 29840 W: 1220 L: 1081 D: 27539
Ptnml(0-2): 10, 969, 12827, 1100, 14
https://tests.stockfishchess.org/tests/view/5fc2ea5142a050a89f02c957
Bench: 3561701
2020-11-29 16:54:06 +01:00
2442ba2b0e
Reductions simplification
...
Simplify increase reduction for captures/promotions if late move and at low depth.
STC https://tests.stockfishchess.org/tests/view/5fbff65067cbf42301d6b3ae
LLR: 2.97 (-2.94,2.94) {-1.25,0.25}
Total: 49088 W: 4607 L: 4555 D: 39926
Ptnml(0-2): 177, 3615, 16932, 3619, 201
LTC https://tests.stockfishchess.org/tests/view/5fc0902967cbf42301d6b3fc
LLR: 2.99 (-2.94,2.94) {-0.75,0.25}
Total: 160944 W: 6153 L: 6193 D: 148598
Ptnml(0-2): 90, 5525, 69294, 5461, 102
closes https://github.com/official-stockfish/Stockfish/pull/3248
bench: 3834568
2020-11-29 16:52:51 +01:00
045728a7da
Remove piece lists
...
This patch removes the incrementally updated piece lists from the Position object.
This has been tried before but always failed. My reasons for trying again are:
* 32-bit systems (including phones) are now much less important than they were some years ago (and are absent from fishtest);
* NNUE may have made SF less finely tuned to the order in which moves were generated.
STC:
LLR: 2.94 (-2.94,2.94) {-1.25,0.25}
Total: 55272 W: 5260 L: 5216 D: 44796
Ptnml(0-2): 208, 4147, 18898, 4159, 224
https://tests.stockfishchess.org/tests/view/5fc2986a42a050a89f02c926
LTC:
LLR: 2.96 (-2.94,2.94) {-0.75,0.25}
Total: 16600 W: 673 L: 608 D: 15319
Ptnml(0-2): 14, 533, 7138, 604, 11
https://tests.stockfishchess.org/tests/view/5fc2f98342a050a89f02c95c
closes https://github.com/official-stockfish/Stockfish/pull/3247
Bench: 3940967
2020-11-29 16:51:01 +01:00
2bc4ae172a
Update README.md
...
fix a few typos
closes https://github.com/official-stockfish/Stockfish/pull/3245
No functional change
2020-11-29 16:47:00 +01:00
6c429c4d65
Search simplification
...
STC https://tests.stockfishchess.org/tests/view/5fc2083942a050a89f02c8bb
LLR: 2.93 (-2.94,2.94) {-1.25,0.25}
Total: 23200 W: 2251 L: 2160 D: 18789
Ptnml(0-2): 86, 1726, 7895, 1797, 96
LTC https://tests.stockfishchess.org/tests/view/5fc22d7b42a050a89f02c8d0
LLR: 2.92 (-2.94,2.94) {-0.75,0.25}
Total: 15832 W: 653 L: 590 D: 14589
Ptnml(0-2): 7, 521, 6795, 588, 5
closes https://github.com/official-stockfish/Stockfish/pull/3244/files
bench: 3827317
Simplify search.
2020-11-29 16:42:40 +01:00
66da1e802c
Remove bonus for killers.
...
Passed non-regression STC:
LLR: 2.94 (-2.94,2.94) {-1.25,0.25}
Total: 14712 W: 1416 L: 1315 D: 11981
Ptnml(0-2): 59, 1029, 5082, 1124, 62
https://tests.stockfishchess.org/tests/view/5fbfa31f67cbf42301d6b36e
Passed non-regression LTC:
LLR: 2.95 (-2.94,2.94) {-0.75,0.25}
Total: 27536 W: 1099 L: 1044 D: 25393
Ptnml(0-2): 11, 929, 11838, 974, 16
https://tests.stockfishchess.org/tests/view/5fbfac9167cbf42301d6b371
closes https://github.com/official-stockfish/Stockfish/pull/3241
Bench: 3887644
2020-11-29 16:40:06 +01:00
Tomasz Sobczyk