Proposal: Storage Overhaul

[UnitylChaos]

As a result of the conclusions reached in cosmos/cosmos-sdk#7100, the Cosmos-SDK is undergoing a switch away from the IAVL library which is going to impact the long term usability of Kepler in the Cosmos ecosystem. Currently Kepler relies on a gRPC based IAVL service to provide state storage and commitment, it is unlikely that this service will be updated or maintained in the future.

The Cosmos-SDK is moving to a decoupled storage model, where one module provides state storage, and another provides state commitments. Under this scheme the commitments will be provided by a "Sparse Merkle Tree", through the LazyLedger SMT Library, while the storage will be provided by an in-process KVStore (specific library TBD). Part of the rationale for this change is to allow for easy benchmarking of alternative storage or commitment modules, with a clear pathway to upgrading while maintaining the same standardized interface.

I am proposing that Kepler follow suit, and build support for this decoupled model, including SMT based commitments.
In (very) rough draft this would require:

• New effects: StoreBackend and StateCommitment
• Implementation of StoreBackend effect using Haskell based storage library
  • At minimum this needs support for Read/Write and Begin/Rollback/Commit
  • Currently looking at SQLite based libraries such as Persistent and sqlite-simple, as well as KVStore libraries such as haskey or RocksDB
  • I'm leaning towards the SQLite options to take advantage of the better ACID guarantees, automatic support for Begin/Rollback/Commit, and support for multi-index data / join style queries
  • With a KVStore based implementation begin/rollback/commit functionality would likely to have to be built manually using in-memory write caching. Multi-index data / join style queries could be added manually as well.
• Implementation of StateCommitment effect using Haskell based SMT implementation, on top of raw storage provided by StoreBackend
  • Proof generation / verification
  • Updates to commitment on state writes
  • Returning new root hash on block commit
• Generic evaluator for StoreEffs using combination of StoreBackend and StateCommitment
  • This would handle coordination between reads/writes to state and commitment layer, transactions, block commits, etc.
  • Given that root hash is only returned on block commit, it would likely be most efficient to queue writes to the StateCommitment and submit them as a batch update on commit.
• Replacement for IAVLStore in CoreEffs using combination of above implementations

Completing this overhaul will improve the speed and efficiency of Kepler, as well as simplify future feature upgrades such as State Syncing and IBC compatibility.

I would appreciate feedback on the proposed ideas, both from the FOAM team and anyone else who is or will be a user of Kepler. I have spoken with @hxrts about applying for ICF grant funding to support this and other contributions to Kepler (such as State Syncing or IBC), so I would like to make sure that my proposed changes will be beneficial not just to my own dependent project but to others as well.

Relevant Links:

• What to do about IAVL? cosmos/cosmos-sdk#7100
• ADR-40 Replace IAVL and decouple state commitment from storage cosmos/cosmos-sdk#8297
• ADR-040: Storage and SMT State Commitments cosmos/cosmos-sdk#8430 (rendered ADR)
• Add SMT store type cosmos/cosmos-sdk#8507
• https://github.com/lazyledger/smt

[JonathanLorimer]

This sounds really cool, i'd be happy to help out on some of this!