BladeBit - Fast Chia (XCH) RAM-only k32-only Plotter

BladeBit Chia Plotter

Release Builds

A fast RAM-only, k32-only, Chia plotter.

Requirements

416 GiB of RAM are required to run it, plus a few more megabytes for stack space and small allocations.

64-bit is supported only, for obvious reasons.

Prerequisites

Only Linux & Windows are supported.

Linux

Install the following packages:

# CentOS or Amazon Linux
sudo yum group install -y "Development Tools"
sudo yum install -y cmake gmp-devel numactl-devel

# Ubuntu or Debian-based
sudo apt install -y build-essential cmake libgmp-dev libnuma-dev

Windows

Must have Visual Studio 2019 or its build tools installed.

Building

# Clone the repo & its submodules
git clone https://github.com/Chia-Network/bladebit.git && cd bladebit

# Create a build directory for cmake and cd into it
mkdir build
cd build

# Generate config files & build
cmake ..
cmake --build . --target bladebit --config Release

The resulting binary will be found under the build/ directory. On Windows it will be under build/Release/.

Usage

Run bladebit with the -h for complete usage and command line options:

# Linux
build/bladebit -h

# Windows
build/Release/bladebit.exe -h

License

Licensed under the Apache 2.0 license. See LICENSE.

Other Details

Disk I/O

Writes to disk only occur to the final plot file, and it is done sequentially, un-buffered, with direct I/O. This means that writes will be block-aligned. If you've gotten faster writes elsewhere in your drive than you will get with this, it is likely that it is using buffered writes, therefore it "finishes" before it actually finishes writing to disk. The kernel will handle the I/O in the background from cache (you can confirm this with tools such as iotop). The final writes here ought to pretty much saturate your sequential writes. Writes begin happening in the background at Phase 3 and will continue to do so, depending on the disk I/O throughput, through the next plot, if it did not finish beforehand. At some point in Phase 1 of the next plot, it might stall if it still has not finished writing to disk and a buffer it requires is still being written to disk. On the system I tested, there was no interruption when using an NVMe drive.

Pool Plots

Pool plots are fully supported and tested against the chia-blockchain implementation. The community has also verified that pool plots are working properly and winning proofs with them.

NUMA systems

Memory is bound on interleaved mode for NUMA systems which currently gives the best performance on systems with several nodes. This is the default behavior on NUMA systems, it can be disabled with with the -m or --no-numa switch.

Huge TLBs

This is not supported yet. Some folks have reported some gains when using huge page sizes. Although this was something I wanted to test, I focused first instead on things that did not necessarily depended on system config. But I'd like to add support for it in the future (trivial from the development point of view, I have just not configured the test system with huge page sizes).

Other Observations

This implementation is highly memory-bound so optimizing your system towards fast memory access is essential. CPUs with large caches will benefit as well.

Copyright 2021 Harold Brenes, Chia Network Inc

Owner
Chia Network
The green cryptocurrency with Chialisp
Chia Network
Comments
  • Crashed! on second plot at Table 3

    Crashed! on second plot at Table 3

    OS: Clear Linux 37640 CPU: Intel 12700T RAM: 48GB DDR4 SSD: 1024GB Kioxia M.2 PCIe Gen 4

    Command run ./bladebit -f xxxxxx-c xxxxxx -t 19 -n 12 diskplot -t1 /home/admin/temp -b 64 --cache 30G /run/media/admin/hdd/chia/farm

    crash.log './bladebit(_Z12CrashHandleri+0xaa)[0x55b8df83718a] /usr/lib64/glibc-hwcaps/x86-64-v3/libc.so.6(+0x3ac50)[0x55b8df21bc50] ./bladebit(_ZN15DiskBufferQueue15CmdWriteBucketsERKNS_7CommandEm+0xe4)[0x55b8df844684] ./bladebit(_ZN15DiskBufferQueue11CommandMainEv+0x48f)[0x55b8df845d3f] ./bladebit(+0x112e3f)[0x55b8df845e3f] ./bladebit(ZN6Thread17ThreadStarterUnixEPS+0x6d)[0x55b8df8381bd] /usr/lib64/glibc-hwcaps/x86-64-v3/libc.so.6(+0x90a6d)[0x55b8df271a6d] /usr/lib64/glibc-hwcaps/x86-64-v3/libc.so.6(__clone+0x40)[0x55b8df2fd800]`

  • Could NOT find NUMA (missing: NUMA_LIBRARIES) on Clear linux

    Could NOT find NUMA (missing: NUMA_LIBRARIES) on Clear linux

    After running cmake .. It fails to complete with the error CMake Error at /usr/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:230 (message): Could NOT find NUMA (missing: NUMA_INCLUDE_DIR NUMA_LIBRARY) Call Stack (most recent call first): /usr/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE) cmake_modules/FindNUMA.cmake:19 (find_package_handle_standard_args) My OS is Clear Linux 37640.

  • Impossible to plot at RAM after update 1.6.1 - Ubuntu

    Impossible to plot at RAM after update 1.6.1 - Ubuntu

    I update the Chia to version 1.6.1 and after that when I try to plot by terminal(Ubuntu) shows this message: You’re trying to run bladebit version 1 but currently version 2.0.0 is installed.

    Command lines:

    chia plotters bladebit -r 47 -n 150 -f ffffffff -c cccccccc -d /media/ed/271C236D39BA565C You’re trying to run bladebit version 1 but currently version 2.0.0 is installed

    chia plotters bladebit -r 47 -n 150 -f ffffffff -c cccccccc ramplot -d /media/ed/271C236D39BA565C usage: chia plotters [-h] {chiapos,madmax,bladebit,bladebit2,install,version} … chia plotters: error: unrecognized arguments: ramplot

    chia plotters bladebit -r 47 -n 150 -f ffffffff -c cccccccc -ramplot -d /media/ed/271C236D39BA565C usage: chia plotters bladebit [-h] [-r THREADS] [-n COUNT] [-f FARMERKEY] [-p POOL_KEY] [-c CONTRACT] [-i ID] [-w] [–nonuma] [-v] -d FINALDIR chia plotters bladebit: error: argument -r/–threads: invalid int value: ‘amplot’

    I dont know what do for this works.

    I try to plot at the GUI but it required a temp directory...

    So how I install old version of the bladebit? To a plot at ram?

  • Bladebit with ramplot

    Bladebit with ramplot

    Good morning Harold,

    image Can you please provide some sample strings in using ramplot Everytime I down your version I have to change the name (too many versions out there)

    Can you please provide the string to get help on ramplot as you can see the error. C:\bb>bladebitv201h ramplot -h

    Fatal Error: A farmer public key must be specified.

    C:\bb>

    Thanks again Roy

    Just in case you wanted to know my system spec ( Dell T7610 512gb memory, C:\ Samsung 850EVO 1tb, 2 PCIE 1tb for temp drives and a 18tb end disk to collect the plots :-)

  • [BUG] Windows Bladebit v2.0.1 diskplot --cache 220G failing for 50% of the plots at random stages of Phase 3

    [BUG] Windows Bladebit v2.0.1 diskplot --cache 220G failing for 50% of the plots at random stages of Phase 3

    System: i9-10980xe, 256GB RAM, Samsung 980 Pro, Windows 10 21H2 build 19044.1889

    Bladebit: v2.0.1

    Command line:

    %LOCALAPPDATA%\Programs\Chia\resources\app.asar.unpacked\daemon\bladebit\bladebit.exe ^
        -t 36 ^
        -f ... ^
        -c ...^
        -n 1000 ^
        -v ^
        diskplot ^
        --unbounded ^
        --cache 220G ^
        -t1 %temp_dir_path%\ ^
        %final_dir_path%\
    

    The system is stable, MadMax plots into 120GB RAM disk for days without issues. Bladebit is randomly failing one plot out of two at some point of Phase 3, leaving .tmp files of random sizes in the final dir. When it runs with 220G cache there's plenty of free RAM left. Any suggestions?

    Event Viewer error example:

    Faulting application name: bladebit.exe, version: 0.0.0.0, time stamp: 0x636974ca
    Faulting module name: bladebit.exe, version: 0.0.0.0, time stamp: 0x636974ca
    Exception code: 0xc0000409
    Fault offset: 0x00000000001936d9
    Faulting process id: 0x4b00
    Faulting application start time: 0x01d8f36c520ad42f
    Faulting application path: C:\Users\...\AppData\Local\Programs\Chia\resources\app.asar.unpacked\daemon\bladebit\bladebit.exe
    Faulting module path: C:\Users\...\AppData\Local\Programs\Chia\resources\app.asar.unpacked\daemon\bladebit\bladebit.exe
    Report Id: 
    Faulting package full name: 
    Faulting package-relative application ID: 
    

    Output before failure: Example 1:

    Running Phase 3
    Compressing tables 1 and 2.
    Step 1 Allocated 2969.44 / 4062.89 MiB
    Step 2 using 1.68 / 3.97 GiB.
    

    Example 2:

    Running Phase 3
    Compressing tables 1 and 2.
    Step 1 Allocated 2969.44 / 4062.89 MiB
    Step 2 using 1.68 / 3.97 GiB.
    Table 1 now has 3429416189 / 4294957371 ( 79.85% ) entries.
    Table 1 I/O wait time: 67.80 seconds.
    Finished compressing tables 1 and 2 in 168.15 seconds.
    Compressing tables 2 and 3.
    Step 1 Allocated 3455.82 / 4062.89 MiB
    Step 2 using 1.67 / 3.97 GiB.
    
Graphs the progress of block height in your Chia Full Node.
Graphs the progress of block height in your Chia Full Node.

Chia Height Graph Monitor for Chia Block Height Introduction The chiaheightgraph tool will graph Chia Block Height changes in a linux terminal. Use a

Sep 14, 2021
A family of header-only, very fast and memory-friendly hashmap and btree containers.
A family of header-only, very fast and memory-friendly hashmap and btree containers.

The Parallel Hashmap Overview This repository aims to provide a set of excellent hash map implementations, as well as a btree alternative to std::map

Nov 25, 2022
A fast hash map/hash table (whatever you want to call it) for the C programming language.

C HashMap A fast hash map/hash table (whatever you want to call it) for the C programming language. It can associate a key with a pointer or integer v

Nov 23, 2022
Simple C++ code to benchmark fast division algorithms

fast_division Simple C++ code to benchmark fast division algorithms relying on constant divisors. The code is a companion to the paper Integer Divisio

Oct 31, 2022
This project Orchid-Fst implements a fast text string dictionary search data structure: Finite state transducer (short for FST) in c++ language.This FST C++ open source project has much significant advantages.
This project Orchid-Fst implements a fast text string dictionary search data structure: Finite state transducer (short for FST) in c++ language.This FST C++ open source project has much significant advantages.

Orchid-Fst 1. Project Overview This project Orchid-Fst implements a fast text string dictionary search data structure: Finite state transducer , which

Oct 18, 2022
Another neofetch-like utility but this time it's fast.

SystemFetch Another neofetch-like utility but this time it's fast. Example Speed Here is a table of the time it took to execute all of these programs,

Jul 22, 2021
This is a curve topology verification tool based on Fast Linking Numbers for Topology Verification of Loopy Structures.
This is a curve topology verification tool based on Fast Linking Numbers for Topology Verification of Loopy Structures.

Fast Linking Numbers This tool, called verifycurves, takes input models that consist of closed-loop curves, and outputs a topology certificate as a .t

Jan 26, 2022
merge two sorted lists fast

Py Merge Merge sorted list faster than using list.sort or heapq.merge. import merge # create some sorted lists a = list(range(-100, 1700)) b = list(r

Nov 21, 2021
A fast multi-producer, multi-consumer lock-free concurrent queue for C++11

moodycamel::ConcurrentQueue An industrial-strength lock-free queue for C++. Note: If all you need is a single-producer, single-consumer queue, I have

Dec 2, 2022
C++ implementation of a fast hash map and hash set using hopscotch hashing

A C++ implementation of a fast hash map and hash set using hopscotch hashing The hopscotch-map library is a C++ implementation of a fast hash map and

Nov 26, 2022
🏅State-of-the-art learned data structure that enables fast lookup, predecessor, range searches and updates in arrays of billions of items using orders of magnitude less space than traditional indexes

The Piecewise Geometric Model index (PGM-index) is a data structure that enables fast lookup, predecessor, range searches and updates in arrays of bil

Dec 3, 2022
Fast & memory efficient hashtable based on robin hood hashing for C++11/14/17/20

➵ robin_hood unordered map & set robin_hood::unordered_map and robin_hood::unordered_set is a platform independent replacement for std::unordered_map

Nov 28, 2022
C++ implementation of a fast hash map and hash set using robin hood hashing

A C++ implementation of a fast hash map and hash set using robin hood hashing The robin-map library is a C++ implementation of a fast hash map and has

Nov 26, 2022
A fast Python Common substrings of multiple strings library with C++ implementation

A fast Python Common substrings of multiple strings library with C++ implementation Having a bunch of strings, can I print some substrings which appea

Aug 21, 2022
Typesafe, Generic & Extremely fast Dictionary in C 🚀

CDict.h Typesafe, Generic, and Extremely Fast Dictionary in C ?? Key Features Extremely fast non-cryptographic hash algorithm XXHash Complete Typesafe

Oct 5, 2022
Simple and fast configuration file library (written in C99)

Features Configuration file reading Supported operating systems Ubuntu MacOS Windows Build requirements C99 compiler CMake 3.10+ Cloning git clone htt

May 26, 2022
Very Fast Non-Cryptographic Hash Function

KOMIHASH - Very Fast Hash Function Introduction The komihash() function available in the komihash.h file implements a very fast 64-bit hash function,

Nov 23, 2022
A fast and efficient non-iterating hashmap library

libhash: a fast and efficient non-iterating hashmap library Libhash is a fast and efficient non-iterating hashmap library Usage Usage is easy and simp

Aug 19, 2022