The following code in AddToBlockIndex(main.cpp) is horribly inefficient, and dramatically slows initial block download:
Code:CTxDB txdb; txdb.WriteBlockIndex(CDiskBlockIndex(pindexNew)); // New best if (pindexNew->bnChainWork > bnBestChainWork) if (!SetBestChain(txdb, pindexNew)) return false; txdb.Close();This makes it impossible to use a standard technique for loading large amounts of records into a database (db4 or SQL or otherwise): wrap multiple record insertions into a single database transaction. Ideally, bitcoin would only issue a TxnCommit() for each 1000 blocks or so, during initial block download. If a crash occurs, the database remains in a consistent state.
Furthermore, database open + close for each new block is incredibly expensive. For each database-open and database-close operation, db4
- diagnose health of database, to determine if recovery is needed. this test may require data copying.
- re-init memory pools
- read database file metadata
- acquire file locks
- read and initialize b-tree or hash-specific metadata. build hash table / b-tree roots.
- forces a sync, even if transactions called with DB_TXN_NOSYNC
- fsync memory pool
And, additionally, bitcoin forces a database checkpoint, pushing all transactions from log into main database.
That’s right, that long list of operations is executed per-database (DB), not per-environment (DB_ENV), for a database close+open cycle. To bitcoin, that means we do this for every new block. Incredibly inefficient, and not how db4 was designed to be used.
Recommendations:
1) bitcoin should be opening databases, not just environment, at program startup, and closing database at program shutdown. db4 is designed to handle crashes, if proper transactional use is maintained — and bitcoin already uses db4 transactions properly.
2) For the initial block download, txn commit should occur once every N records, not every record. I suggest N=1000.
EDIT: Updated a couple minor details, and corrected some typos.
It seems like you’re inclined to assume everything is wrong more than is actually so.
Writing the block index is light work. Building the tx index is much more random access per block. I suspect reading all the prev txins is what’s slow. Read caching would help that. It’s best if the DB does that. Maybe it has a setting for how much cache memory to use.
Quote
1) bitcoin should be opening databases, not just environment, at program startup, and closing database at program shutdown.
Already does that. See CDB. The lifetime of the (for instance) CTxDB object is only to support database transactions and to know if anything is still using the database at shutdown.
Quote
And, additionally, bitcoin forces a database checkpoint, pushing all transactions from log into main database.
If it was doing that it would be much slower. It’s supposed to be only once a minute or 500 blocks:
if (strFile == “blkindex.dat” && IsInitialBlockDownload() && nBestHeight % 500 != 0)
nMinutes = 1;
dbenv.txn_checkpoint(0, nMinutes, 0);
Probably should add this:
if (!fReadOnly)
dbenv.txn_checkpoint(0, nMinutes, 0);
Quote
2) For the initial block download, txn commit should occur once every N records, not every record. I suggest N=1000.
Does transaction commit imply flush? That seems surprising to me. I assume a database op wrapped in a transaction would be logged like any other database op. Many database applications need to wrap almost every pair of ops in a transaction, such as moving money from one account to another. (debit a, credit b) I can’t imagine they’re required to batch all their stuff up themselves.
In the following cases, would case 1 flush once and case 2 flush twice?
case 1:
write
write
write
write
checkpoint
case 2:
begin transaction
write
write
commit transaction
begin transaction
write
write
commit transaction
checkpoint
Contorting our database usage will not be the right approach. It’s going to be BDB settings and caching.
88,166 total views, 22 views today
https://bitcointalk.org/index.php?topic=1931.msg25449#msg25449