Make e2e test more deterministic #143
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Benchmark results for
|
| Date (UTC) | 2024-08-25T10:36:36+00:00 |
| Commit | eeaf1feb5a0bd2d4dd5c4fe480fbd84bd2c742dd |
| Base SHA | b84a0d1940dd3392f3ea5d069e1dc54d828b9c45 |
Significant changes
None
ferranbt
added
the
A-e2e
3 tasks
ferranbt
added a commit
that referenced
this pull request
Aug 28, 2024
## 📝 Summary The new integration test introduced in #143 requires the rbuilder to be running to work. However, the playground setup was not waiting for it. This PR fixes that. It adds a loop after the rbuilder creation that checks in the output log for a specific log "RPC server job: started" that signals that the builder is running and the test can start. ## 💡 Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## ✅ I have completed the following steps: * [x] Run `make lint` * [x] Run `make test` * [x] Added tests (if applicable)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📝 Summary
This PR does not fix the e2e test but makes it a bit more deterministic to execute. I feel the problem is that due to flashbots/builder-playground#14 issue, the
rbuilderit not consistent in building blocks at the correct time for the playground. So, it misses building some blocks.The previous e2e test would send the transaction to the EL node (to be picked up by the builder) and the acceptance condition was for the transaction to be mined and that block to be mined by the local builder. However, since the builder was not consistent, the transaction would be mined by the local EL node instead and trigger the acceptance check.
In this fix, I am only sending the transaction to the builder, so that the EL node cannot include it. Then, if the transaction gets mined, it was for sure done by the builder. I have tested it locally multiple times (>10) and it only failed once and I think due to flashbots/builder-playground#14.
💡 Motivation and Context
✅ I have completed the following steps:
make lintmake test