Commit Graph

5969 Commits

Author SHA1 Message Date
Tomasz Sobczyk cf6bc7ecaf Cleanup around get_loss 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 256c4b55ec Properly apply gradient norm clipping after it's scaled in the update_parameters. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk de675e3503 Reintroduce optional scaling of the teacher signal. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 01ae7b1e2c Simplify passing constants that may vary between calls. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk cbd973fdaa Detect constant expressions in autograd and return 0 grad early. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk e975889132 Move cross_entropy calculation to a separate function. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 891abf5511 Make the autograd loss expression chain thread_local. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 8adf00ae6e Identify a single evalation chain by ID in autograd to prevent cache reuse for subsequent evaluations of the same expression tree. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk cb812c742c Add [[nodiscard]] attributes to autograd functions. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 26f19e1429 Make automatic differentiation node types constexpr. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk aec6017195 When forming an autograd expression only copy parts that are rvalue references, store references to lvalues. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk a5c20bee5b Apply gradient clipping. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk d103867558 Add memoization to the autograd expression evaluator. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk aa55692b97 Cross entropy loss. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 539bd2d1c8 Replace the old loss/grad calculation completely. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk b71d1e8620 Pass the new loss function to update_parameters 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 5a58eb803a Loss func with autograd 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 541fb8177a More utility in autograd. 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 6ce0245787 Basic autograd 2020-12-02 08:56:20 +09:00
Tomasz Sobczyk 1322a9a5fd Prevent false sharing of num_calls counter in the shared input trainer. Fix current_operation not being local to the executing thread. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 2aa7f5290e Fix comparison of integers with different signedness. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk a97b65eaef Fix compilation error with USE_BLAS 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 622e0b14c2 Remove superfluous example shuffling. Shuffling now only happens on reading. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 34510dd08a Remove used examples asyncronously. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 0bee8fef64 Don't unnecessarily copy the batch part. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk e954b14196 Prefetch weights for feature transformer backprop to shared cache. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 8009973381 Special case for alpha=1 in saxpy, slight performance increase. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 49b2dcb1f3 Preallocate memory for unique_features. Keep the training_features temporary buffer as a thread_local so we reuse the storage. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 1c8495b54b Remove handwritten saxpy because compilers optimize the second look anyway. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 15c528ca7b Prepare feature transformer learner. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk a3c78691a2 Prepare input slice trainer. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 401fc0fbab Prepare clipped relu trainer. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 774b023641 Add chunked for each with workers. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk cc11375f6d Skeleton for new evaluate learner 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 0d4b803b08 Prepare trainer affine transform. 2020-11-30 08:54:53 +09:00
Tomasz Sobczyk 4ea8572b6d Add single threaded sgemm. 2020-11-30 08:54:53 +09:00
SFisGOD 7364006757 Update default net to nn-62ef826d1a6d.nnue
Include scaling change as suggested by Dietrich Kappe,
the one who trained net for Komodo.  According to him,
some nets may require different scaling in order to utilize its full strength.

STC:
LLR: 2.93 (-2.94,2.94) {-0.25,1.25}
Total: 99856 W: 9669 L: 9401 D: 80786
Ptnml(0-2): 374, 7468, 34037, 7614, 435
https://tests.stockfishchess.org/tests/view/5fc2697642a050a89f02c8ec

LTC:
LLR: 2.96 (-2.94,2.94) {0.25,1.25}
Total: 29840 W: 1220 L: 1081 D: 27539
Ptnml(0-2): 10, 969, 12827, 1100, 14
https://tests.stockfishchess.org/tests/view/5fc2ea5142a050a89f02c957

Bench: 3561701
2020-11-29 16:54:06 +01:00
Unai Corzo 2442ba2b0e Reductions simplification
Simplify increase reduction for captures/promotions if late move and at low depth.

STC https://tests.stockfishchess.org/tests/view/5fbff65067cbf42301d6b3ae
LLR: 2.97 (-2.94,2.94) {-1.25,0.25}
Total: 49088 W: 4607 L: 4555 D: 39926
Ptnml(0-2): 177, 3615, 16932, 3619, 201

LTC https://tests.stockfishchess.org/tests/view/5fc0902967cbf42301d6b3fc
LLR: 2.99 (-2.94,2.94) {-0.75,0.25}
Total: 160944 W: 6153 L: 6193 D: 148598
Ptnml(0-2): 90, 5525, 69294, 5461, 102

closes https://github.com/official-stockfish/Stockfish/pull/3248

bench: 3834568
2020-11-29 16:52:51 +01:00
syzygy1 045728a7da Remove piece lists
This patch removes the incrementally updated piece lists from the Position object.

This has been tried before but always failed. My reasons for trying again are:

* 32-bit systems (including phones) are now much less important than they were some years ago (and are absent from fishtest);
* NNUE may have made SF less finely tuned to the order in which moves were generated.

STC:
LLR: 2.94 (-2.94,2.94) {-1.25,0.25}
Total: 55272 W: 5260 L: 5216 D: 44796
Ptnml(0-2): 208, 4147, 18898, 4159, 224
https://tests.stockfishchess.org/tests/view/5fc2986a42a050a89f02c926

LTC:
LLR: 2.96 (-2.94,2.94) {-0.75,0.25}
Total: 16600 W: 673 L: 608 D: 15319
Ptnml(0-2): 14, 533, 7138, 604, 11
https://tests.stockfishchess.org/tests/view/5fc2f98342a050a89f02c95c

closes https://github.com/official-stockfish/Stockfish/pull/3247

Bench: 3940967
2020-11-29 16:51:01 +01:00
Unai Corzo 2bc4ae172a Update README.md
fix a few typos

closes https://github.com/official-stockfish/Stockfish/pull/3245

No functional change
2020-11-29 16:47:00 +01:00
Unai Corzo 6c429c4d65 Search simplification
STC https://tests.stockfishchess.org/tests/view/5fc2083942a050a89f02c8bb
LLR: 2.93 (-2.94,2.94) {-1.25,0.25}
Total: 23200 W: 2251 L: 2160 D: 18789
Ptnml(0-2): 86, 1726, 7895, 1797, 96

LTC https://tests.stockfishchess.org/tests/view/5fc22d7b42a050a89f02c8d0
LLR: 2.92 (-2.94,2.94) {-0.75,0.25}
Total: 15832 W: 653 L: 590 D: 14589
Ptnml(0-2): 7, 521, 6795, 588, 5

closes https://github.com/official-stockfish/Stockfish/pull/3244/files

bench: 3827317

Simplify search.
2020-11-29 16:42:40 +01:00
lonfom169 66da1e802c Remove bonus for killers.
Passed non-regression STC:
LLR: 2.94 (-2.94,2.94) {-1.25,0.25}
Total: 14712 W: 1416 L: 1315 D: 11981
Ptnml(0-2): 59, 1029, 5082, 1124, 62
https://tests.stockfishchess.org/tests/view/5fbfa31f67cbf42301d6b36e

Passed non-regression LTC:
LLR: 2.95 (-2.94,2.94) {-0.75,0.25}
Total: 27536 W: 1099 L: 1044 D: 25393
Ptnml(0-2): 11, 929, 11838, 974, 16
https://tests.stockfishchess.org/tests/view/5fbfac9167cbf42301d6b371

closes https://github.com/official-stockfish/Stockfish/pull/3241

Bench: 3887644
2020-11-29 16:40:06 +01:00
Lolligerhans d6d6972a66 Refine rook penalty on closed files
+-----------------+
| . . . . . . . . | All files are closed. Some files are
| . . . . . o o . | more valuable for rooks, because
| . . . . o . . o | they might open in the future.
| . . . o x . . x |
| o . o x . x x . |
| x o x . . . . . | x  our pawns
| . x . . . . . . | o  their pawns
| . . . . . . . . | ^  rooks are scored higher on these files
+-----------------+
            ^ ^

Files containing none of our own pawns are open or half-open (otherwise
they are closed). Rooks on (half-)open files recieve a bonus for the
future potential to act along all ranks.

This commit refines the (relative) penalty of rooks on closed files.
Files that contain one of our blocked pawns are considered less likely
to open in the future; rooks on these files are now penalized stronger.

This bonus does not generally correlate with mobility. If the condition
is sufficiently refined in the future, it may be beneficial to adjust or
override mobility scores in some cases.

LTC
LLR: 2.94 (-2.94,2.94) {0.25,1.25}
Total: 494384 W: 71565 L: 70231 D: 352588
Ptnml(0-2): 3907, 48050, 142118, 49036, 4081
https://tests.stockfishchess.org/tests/view/5fb9312e67cbf42301d6afb9

LTC (non-regression w/ book noob_3moves.epd)
LLR: 2.95 (-2.94,2.94) {-0.75,0.25}
Total: 208520 W: 27044 L: 26937 D: 154539
Ptnml(0-2): 1557, 19850, 61391, 19853, 1609
https://tests.stockfishchess.org/tests/view/5fc01ced67cbf42301d6b3df

STC
LLR: 2.94 (-2.94,2.94) {-0.25,1.25}
Total: 98392 W: 20269 L: 19868 D: 58255
Ptnml(0-2): 1804, 11297, 22589, 11706, 1800
https://tests.stockfishchess.org/tests/view/5fb7f88a67cbf42301d6af10

closes https://github.com/official-stockfish/Stockfish/pull/3242

Bench: 3682630
2020-11-29 16:38:03 +01:00
mstembera 9b7983a452 Cleaned up MakeIndex()
The index order in kpp_board_index[][] is reversed to be more optimal for the access pattern

STC https://tests.stockfishchess.org/tests/view/5fbd74f967cbf42301d6b24f
LLR: 2.93 (-2.94,2.94) {-1.25,0.25}
Total: 27504 W: 2686 L: 2607 D: 22211
Ptnml(0-2): 84, 2001, 9526, 2034, 107

closes https://github.com/official-stockfish/Stockfish/pull/3233

No functional change
2020-11-29 16:36:49 +01:00
nodchip ef4fdb40f9 Merge pull request #254 from noobpwnftw/merge
Merge
2020-11-28 09:03:38 +09:00
noobpwnftw 0b2ae6cb64 Merge remote-tracking branch 'remotes/official/master' into merge 2020-11-28 06:47:04 +08:00
Tomasz Sobczyk 92b14a5ba2 Add docs for transform. 2020-11-27 09:16:22 +09:00
Tomasz Sobczyk 89294e2e4f Add transform command. Add transform nudged_static subcommand. 2020-11-27 09:16:22 +09:00
Vizvezdenec 190dd26b9f use classical for certain endgames.
STC https://tests.stockfishchess.org/tests/view/5fbc64c067cbf42301d6b1d6
LLR: 2.97 (-2.94,2.94) {-0.25,1.25}
Total: 53360 W: 5223 L: 5024 D: 43113
Ptnml(0-2): 184, 3877, 18390, 4014, 215

LTC https://tests.stockfishchess.org/tests/view/5fbc97f267cbf42301d6b1ee
LLR: 2.96 (-2.94,2.94) {0.25,1.25}
Total: 126472 W: 5111 L: 4766 D: 116595
Ptnml(0-2): 50, 4032, 54749, 4333, 72

closes https://github.com/official-stockfish/Stockfish/pull/3240

bench: 3820648
2020-11-26 08:20:06 +01:00
MaximMolchanov 7615e3485e Calculate sum from first elements
in affine transform for AVX512/AVX2/SSSE3

The idea is to initialize sum with the first element instead of zero.
Reduce one add_epi32 and one set_zero SIMD instructions for each output dimension.

sum = 0; for i = 1 to n sum += a[i] ->
sum = a[1]; for i = 2 to n sum += a[i]

STC:
LLR: 2.95 (-2.94,2.94) {-0.25,1.25}
Total: 69048 W: 7024 L: 6799 D: 55225
Ptnml(0-2): 260, 5175, 23458, 5342, 289
https://tests.stockfishchess.org/tests/view/5faf2cf467cbf42301d6aa06

closes https://github.com/official-stockfish/Stockfish/pull/3227

No functional change.
2020-11-25 21:10:13 +01:00