In this work, we effectively need to rebuild ABCI++ (currently on the v0.36.x branch) on a branch off of v0.34.x, starting just with ABCI++ Prepare/Process Proposal.

The remainder of ABCI++ will be released in subsequent releases.

Proposed Path to Done. Summary

The main idea behind this plan is to proceed in the same way we did for v0.36.x, represented in ABCI++ project board.
The main differences will be:

• We will only be considering PrepareProposal/ProcessProposal related tasks
• We will "follow" our work on v0.36.x. By "following a PR", I mean looking at that PR (similar work done for v0.36.x) and copying over only what makes sense.

The work is structured into two main parts (further described below):

• (1) Preliminary work [Estimation of critical path: 9 days (with 1 thread)]
• (2) Core feature work [Estimation of critical path: 14.5 days (with 5 threads)]

(1) and (2) can mostly proceed in parallel.

N.B.: Let’s manage the protobufs the right way this time 😊 → They should evolve with the code

Proposed Path to Done. Details

(1) Preliminary Work

We could have two threads here, but as all these tasks serialized are shorter than the critical path in "Core Feature Work", there's no point

• Fix UTs on v0.34.x branch (or on new main) [Estimation: 1 day]
• Create feature branch: feature/abci++ppp [Estimation: 0 days]
• Restore CI on feature/abci++ppp branch [Estimation: 0.5 days]
• proto generation uses buf (cherry-pick or follow proto: update proto generation to use buf #7975) [Timebox: 1 day]
• abci++: Port E2E tests from v0.36.x to ABCI++ feature branch #9271

(2) Core feature work

We have three threads that can proceed in parallel:

• PrepareProposal (*) [Estimation: 9.5 days]
• ProcessProposal (*) [Estimation: 8 days]
• Spec and doc work [Estimation: 8 days]

(*) Only when PrepareProposal and ProcessProposal are done, can we proceed with the fourth part:

• Final adjustments [Estimation: 5 days, assuming no parallel work done]
  • tasks here could be further parallelized

PrepareProposal

• Old PrepareProposal #9104 [Estimation: 0.5 days, ready for PR]
  • Description: Cherry-pick abci: PrepareProposal #6544 (Done via Rebased branch abci++ to tip of master #7746 – as a “dry-run” for ABCI++ Option 2 in planning doc)
• Old PrepareProposal-Old VoteExtension integration #9108 [Estimation: 2 days]
  • Description: Cherry-pick PrepareProposal-VoteExtension integration #6915 (very few conflicts expected)
  • careful about protobuf bug we solved on v0.36.x (showed up as a crash in e2e tests, compare with abci: PrepareProposal-VoteExtension integration [2nd try] #7821)
• Sync PrepareProposal with Spec #9118 [Estimation: 7 days]
  • Main part: follow abci++: synchronize PrepareProposal with the newest version of the spec #8094
  • Minor: remove vote extensions from proto: follow abci++: remove app_signed_updates #8128
  • Panic on badly prepared proposal state: panic on ResponsePrepareProposal validation error #8145
  • CheckTx out of the way: follow abci++: remove CheckTx call from PrepareProposal flow #8176
  • Fix Max-size: follow abci++: correct max-size check to only operate on added and unmodified #8242, Remove ModifiedTxStatus from the spec and the code #8210

ProcessProposal

• Old ProcessProposal [Estimation: 3 days, ready for PR]
  • Cherry-pick ABCI++ ProcessProposal #7091
  • Follow Rebased to master the existing ProcessProposal PR #7752
• Sync ProcessProposal with spec [5 days]
  • Follow state: synchronize the ProcessProposal implementation with the latest version of the spec #7961

Final adjustments

Most of these tasks can be done in parallel, but they are quite short.

• Move app_hash from Commit to EndBlock: follow abci: Move app_hash parameter from Commit to FinalizeBlock #8664
• Protobuf: responses should be enums: see what makes sense out of ABCI++: Update new protos to use enum instead of bool #8158 [Estimation: 0.5 days]
• Block data exposed to App #9211: follow abci++: only include meaningful header fields in data passed-through to application #8216 and follow or cherry-pick the PrepareProposal and ProcessProposal parts [Estimation: 1 day]
• (PrepareProposal/ProcessProposal: Compare types.proto in feature/abci++ppp and v0.36.x #9221) Compare the status of abci's types.proto in feature/abci++ppp and v0.36.x and make sure no changes are related to PrepareProposal/ProcessProposal

At this point, external teams should be able to start integration of PrepareProposal/ ProcessProposal

• Add metrics: see what makes sense out of consensus: add additional metrics for abci++ data #8480 [Estimation: 1 day]
• Remove old methods. See what makes sense out of Remove obsolete abci methods, no longer called by ABCI++ Tendermint #8633
• (e2e: add abci delays #9254) Add sleep in e2e app: follow or cherry-pick e2e: programmable ABCI method times #8638 [Estimation: 1.5 days]
• (abci-cli: PrepareProposal and ProcessProposal #9270) abci-cli: PrepareProposal (follow abci-cli: added PrepareProposal command to cli #8656), ProcessProposal (follow abci-cli: Add process_proposal command to abci-cli #8901) [Estimation: 1.5 days]
• (proto: deduplicate consensus params #9287)

Spec and doc work

• [Overall estimation: 9 days] This is low risk, and can proceed completely in parallel with the rest
• (spec: port abci++ spec to main #9201 ) Copy current state of ABCI++ spec on v0.36.x onto feature/abci++ppp branch and remove references to unused features
  - Vote extensions, FinalizeBlock (type of work similar to Remove references to vote extensions from Tendermint spec before releasing v0.36 #7745)
• Remove global client mutex (abci: remove global mutex around local client #7073) won’t be picked up for risk mitigation → “revert” the spec accordingly
• Follow code-related hunks of (Restore Commit to the ABCI++ spec, and other late modifications #8796)
  • Restore data field in Commit
  • Rename Misbehaviour to ByzantineValidators
  • Rename Evidence to Misbheaviour
• Final read of spec for overall coherence
• Review ADRs and RFCs that make sense on feature/abci++ppp branch
• docs: Add ABCI++ tutorials #9272