filecoin lotus benchmarks v26 GeForce GTX 2080 Ti

1,コンパイルbench

# git show
commit d11e375f828526eab2a59fd6fd3db88e3c5cb9d3 (HEAD -> interopnet, tag: interop.5.13, origin/interopnet)
Merge: 7b8ee402 1d8c8b9a
Author: Travis Person 
Date:   Wed May 13 05:44:28 2020 +0000

    Merge remote-tracking branch 'origin/fix/check-gas-limits' into interopnet

# export ALL_PROXY=socks5://127.0.0.1:1080
# git config --global http.proxy 'socks5://127.0.0.1:1080'
# export GO111MODULE=on
# export GOPROXY=https://goproxy.cn

env RUSTFLAGS="-C target-cpu=native -g" FFI_BUILD_FROM_SOURCE=1 make clean deps bench

go build -o bench ./cmd/lotus-bench
go run github.com/GeertJohan/go.rice/rice append --exec bench -i ./build

# ./bench --version
2020-05-14T15:01:36.340+0800	INFO	lotus-bench	lotus-bench/main.go:74	Starting lotus-bench
lotus-bench version 0.3.0

2、基準テスト

# export RUST_LOG=Trace && export FIL_PROOFS_MAXIMIZE_CACHING=1

# ./bench sealing --storage-dir=/storage/benchtmp --sector-size=32GiB

# nohup ./bench sealing --storage-dir=/raid5/benchtmp --sector-size=32GiB >> /raid5/bench.log 2>&1 &

2020-05-14T23:16:04.644 INFO filecoin_proofs::api::seal > seal_commit_phase1:end
2020-05-14T23:16:04.711 INFO filcrypto::proofs::api > seal_commit_phase1: finish
2020-05-14T23:16:04.805+0800	INFO	lotus-bench	lotus-bench/main.go:499	Generating PoRep for sector (2)
2020-05-14T23:16:04.825 INFO filcrypto::proofs::api > seal_commit_phase2: start
2020-05-14T23:16:04.900 INFO filecoin_proofs::api::seal > seal_commit_phase2:start
2020-05-14T23:16:04.901 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
2020-05-14T23:16:04.901 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
2020-05-14T23:16:04.902 INFO filecoin_proofs::caches > trying parameters memory cache for: STACKED[34359738368]
2020-05-14T23:16:04.902 INFO filecoin_proofs::caches > no params in memory cache for STACKED[34359738368]
2020-05-14T23:16:04.904 INFO storage_proofs_core::parameter_cache > parameter set identifier for cache: layered_drgporep::PublicParams{ graph: stacked_graph::StackedGraph{expansion_degree: 8 base_graph: drgraph::BucketGraph{size: 1073741824; degree: 6; hasher: poseidon_hasher} }, challenges: LayerChallenges { layers: 11, max_count: 18 }, tree: merkletree-poseidon_hasher-8-8-0 }
2020-05-14T23:16:04.904 INFO storage_proofs_core::parameter_cache > ensuring that all ancestor directories for: "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.params" exist
2020-05-14T23:16:04.904 INFO storage_proofs_core::parameter_cache > checking cache_path: "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.params" for parameters
2020-05-14T23:16:07.290 INFO storage_proofs_core::parameter_cache > read parameters from cache "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.params" 
2020-05-14T23:16:07.290 INFO filecoin_proofs::api::seal > got groth params (34359738368) while sealing
2020-05-14T23:16:07.290 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
2020-05-14T23:16:07.290 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
2020-05-14T23:16:07.290 INFO filecoin_proofs::api::seal > snark_proof:start
2020-05-14T23:16:07.298 INFO bellperson::groth16::prover > Bellperson 0.7.0 is being used!
2020-05-14T23:20:18.490 INFO bellperson::gpu::locks > GPU is available for FFT!
2020-05-14T23:20:18.490 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2020-05-14T23:20:18.496 DEBUG bellperson::gpu::locks > GPU lock acquired!
2020-05-14T23:20:18.732 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
2020-05-14T23:20:18.732 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 2080 Ti
2020-05-14T23:20:18.732 INFO bellperson::domain > GPU FFT kernel instantiated!
2020-05-14T23:23:12.352 DEBUG bellperson::gpu::locks > GPU lock released!
2020-05-14T23:24:13.677 INFO bellperson::gpu::locks > GPU is available for Multiexp!
2020-05-14T23:24:13.677 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2020-05-14T23:24:13.840 DEBUG bellperson::gpu::locks > GPU lock acquired!
2020-05-14T23:24:13.933 INFO bellperson::gpu::multiexp > Multiexp: 1 working device(s) selected. (CPU utilization: 0)
2020-05-14T23:24:13.933 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 2080 Ti (Chunk-size: 7488999)
2020-05-14T23:24:13.933 INFO bellperson::multiexp > GPU Multiexp kernel instantiated!
2020-05-14T23:50:07.906 DEBUG bellperson::gpu::locks > GPU lock released!
2020-05-14T23:50:09.849 INFO filecoin_proofs::api::seal > snark_proof:finish
2020-05-14T23:50:09.849 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
2020-05-14T23:50:09.849 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
2020-05-14T23:50:09.849 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
2020-05-14T23:50:09.849 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
2020-05-14T23:50:09.849 INFO filecoin_proofs::caches > trying parameters memory cache for: STACKED[34359738368]-verifying-key
2020-05-14T23:50:09.849 INFO filecoin_proofs::caches > no params in memory cache for STACKED[34359738368]-verifying-key
2020-05-14T23:50:09.850 INFO storage_proofs_core::parameter_cache > parameter set identifier for cache: layered_drgporep::PublicParams{ graph: stacked_graph::StackedGraph{expansion_degree: 8 base_graph: drgraph::BucketGraph{size: 1073741824; degree: 6; hasher: poseidon_hasher} }, challenges: LayerChallenges { layers: 11, max_count: 18 }, tree: merkletree-poseidon_hasher-8-8-0 }
2020-05-14T23:50:09.850 INFO storage_proofs_core::parameter_cache > ensuring that all ancestor directories for: "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.vk" exist
2020-05-14T23:50:09.850 INFO storage_proofs_core::parameter_cache > checking cache_path: "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.vk" for verifying key
2020-05-14T23:50:09.946 INFO storage_proofs_core::parameter_cache > read verifying key from cache "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.vk" 
2020-05-14T23:50:09.946 INFO filecoin_proofs::api::seal > got verifying key (34359738368) while verifying seal
2020-05-14T23:50:09.971 INFO bellperson::gpu::locks > GPU is available for Multiexp!
2020-05-14T23:50:09.971 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2020-05-14T23:50:10.023 DEBUG bellperson::gpu::locks > GPU lock acquired!
2020-05-14T23:50:10.123 INFO bellperson::gpu::multiexp > Multiexp: 1 working device(s) selected. (CPU utilization: 0)
2020-05-14T23:50:10.124 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 2080 Ti (Chunk-size: 7488999)
2020-05-14T23:50:10.124 INFO bellperson::multiexp > GPU Multiexp kernel instantiated!
2020-05-14T23:50:10.219 DEBUG bellperson::gpu::locks > GPU lock released!
2020-05-14T23:50:10.219 INFO filecoin_proofs::api::seal > seal_commit_phase2:end
2020-05-14T23:50:10.219 INFO filcrypto::proofs::api > seal_commit_phase2: finish
2020-05-14T23:50:10.219 INFO filcrypto::proofs::api > verify_seal: start
2020-05-14T23:50:10.220 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
2020-05-14T23:50:10.220 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
2020-05-14T23:50:10.220 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
2020-05-14T23:50:10.220 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
2020-05-14T23:50:10.220 INFO filecoin_proofs::caches > trying parameters memory cache for: STACKED[34359738368]-verifying-key
2020-05-14T23:50:10.220 INFO filecoin_proofs::caches > found params in memory cache for STACKED[34359738368]-verifying-key
2020-05-14T23:50:10.220 INFO filecoin_proofs::api::seal > got verifying key (34359738368) while verifying seal
2020-05-14T23:50:10.237 INFO bellperson::gpu::locks > GPU is available for Multiexp!
2020-05-14T23:50:10.237 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
2020-05-14T23:50:10.414 DEBUG bellperson::gpu::locks > GPU lock acquired!
2020-05-14T23:50:10.522 INFO bellperson::gpu::multiexp > Multiexp: 1 working device(s) selected. (CPU utilization: 0)
2020-05-14T23:50:10.522 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 2080 Ti (Chunk-size: 7488999)
2020-05-14T23:50:10.522 INFO bellperson::multiexp > GPU Multiexp kernel instantiated!
2020-05-14T23:50:10.612 DEBUG bellperson::gpu::locks > GPU lock released!
2020-05-14T23:50:10.612 INFO filcrypto::proofs::api > verify_seal: finish
2020-05-14T23:50:22.791+0800	WARN	lotus-bench	lotus-bench/main.go:90	failed to run seals:
    main.glob..func3
        /root/interopnet/lotus/cmd/lotus-bench/main.go:233
  - porep proof for sector 1 was invalid:
    main.runSeals
        /root/interopnet/lotus/cmd/lotus-bench/main.go:550

3,GeForce GTX 2080 Tiテスト

# lscpu
AMD Ryzen 9 3950X 16-Core Processor

# lspci | grep -i nvidia
2d:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1)
2d:00.1 Audio device: NVIDIA Corporation GP102 HDMI Audio Controller (rev a1)

# free -h
                                              /      
 ：       125Gi       729Mi       104Gi       0.0Ki        20Gi       123Gi
 ：       105Gi       302Mi       105Gi

# git show
commit 54b3eafd6d541a68199965bbf13ec2e339c625f5 (HEAD -> master, origin/master, origin/HEAD)
Merge: 5c73602d f6589407
Author: Whyrusleeping 
Date:   Fri May 15 13:05:27 2020 -0700

    Merge pull request #1758 from filecoin-project/fix/lotus-bench-secp

    register secp signatures for lotus bench usage

env RUSTFLAGS="-C target-cpu=native -g" FFI_BUILD_FROM_SOURCE=1 make clean deps bench

# export RUST_LOG=Trace && export FIL_PROOFS_MAXIMIZE_CACHING=1

nohup ./bench sealing --storage-dir=/raid5/benchtmp --sector-size=32GiB >> /raid5/bench.log 2>&1 &

results (v26) (34359738368)
seal: addPiece: 6m26.577337189s (84.8 MiB/s)
seal: preCommit phase 1: 4h17m35.603586994s (2.12 MiB/s)
seal: preCommit phase 2: 1h43m21.981600421s (5.28 MiB/s)
seal: commit phase 1: 1m4.253378137s (510 MiB/s)
seal: commit phase 2: 1h2m37.451786187s (8.72 MiB/s)
seal: verify: 37.58607ms
unseal: 4h17m51.34613723s  (2.12 MiB/s)

generate candidates: 7.194132ms (4.34 TiB/s)
compute winnnig post proof (cold): 14.527132336s
compute winnnig post proof (hot): 6.148464516s
verify winnnig post proof (cold): 98.07689ms
verify winnnig post proof (hot): 14.109101ms

compute window post proof (cold): 15m57.884786147s
compute window post proof (hot): 16m16.748839396s
verify window post proof (cold): 5.504320785s
verify window post proof (hot): 44.248935ms