Releases: streamingfast/firehose-core
v0.2.1
Integrators
Note
For integrations updating firehose-core
, check the upgrade procedure to go to v0.1.x -> v0.2.0
Operators
Important
We have had reports of older versions of this software creating corrupted merged-blocks-files (with duplicate or extra out-of-bound blocks)
This release adds additional validation of merged-blocks to prevent serving duplicate blocks from the firehose or substreams service.
This may cause service outage if you have produced those blocks or downloaded them from another party who was affected by this bug.
-
Find the affected files by running the following command (can be run multiple times in parallel, over smaller ranges)
tools check merged-blocks-batch <merged-blocks-store> <start> <stop>
-
If you see any affected range, produce fixed merged-blocks files with the following command, on each range:
tools fix-bloated-merged-blocks <merged-blocks-store> <output-store> <start>:<stop>
-
Copy the merged-blocks files created in output-store over to the your merged-blocks-store, replacing the corrupted files.
Removed
- Removed the
--dedupe-blocks
flag ontools download-from-firehose
as it can create confusion and more issues.
Fixed
- Bumped
bstream
: thefilesource
will now refuse to read blocks from a merged-files if they are not ordered or if there are any duplicate. - The command
tools download-from-firehose
will now fail if it is being served blocks "out of order", to prevent any corrupted merged-blocks from being created. - The command
tools print merged-blocks
did not print the whole merged-blocks file, the arguments were confusing: now it will parse <start_block> as a uint64. - The command
tools unmerge-blocks
did not cover the whole given range, now fixed
Added
- Added the command
tools fix-bloated-merged-blocks
to try to fix merged-blocks that contain duplicates and blocks outside of their range. - Command
tools print one-block and merged-blocks
now supports a new--output-format
jsonl
format.
Bytes data can now printed as hex or base58 string instead of base64 string.
Changed
- Changed
tools check merged-blocks-batch
argument syntax: the output-to-store is now optional.
v0.1.12
- Added
tools check merged-blocks-batch
to simplify checking blocks continuity in batched mode, writing results to a store - Bumped substreams to
v1.1.20
with a fix for some minor bug fixes related to start block processing
v0.1.11
- Bumped substreams to
v1.1.18
with a regression fix for when a substreams has a start block in the reversible segment
v0.1.10
Added
The --common-auth-plugin
got back the ability to use secret://<expected_secret>?[user_id=<user_id>]&[api_key_id=<api_key_id>]
in which case request are authenticated based on the Authorization: Bearer <actual_secret>
and continue only if <actual_secret> == <expected_secret>
.
Changed
- Bumped substreams to
v1.1.17
with provider new metricssubstreams_active_requests
andsubstreams_counter
v0.1.9
Changed
- Bumped susbtreams to
v1.1.14
to fix bugs with start blocks, where substreams would fail if the start block was before the first block of the chain, or if the start block was a block that was not yet produced by the chain. - Improved error message when referenced config file is not found, removed hard-coded mention of
fireacme
.
v0.1.8
Fixed
- More tolerant retry/timeouts on filesource (prevent "Context Deadline Exceeded")
v0.1.7
Operators
Important
The Substreams service exposed from this version will send progress messages that cannot be decoded by substreams clients prior to v1.1.12.
Streaming of the actual data will not be affected. Clients will need to be upgraded to properly decode the new progress messages.
Changed
- Bumped substreams to
v1.1.12
to support the new progress message format. Progression now relates to stages instead of modules. You can get stage information using thesubstreams info
command starting at versionv1.1.12
. - Bumped supervisor buffer size to 100Mb
- Added templating option to
reader-node-arguments
arg, specifically {start-block-num} and {stop-block-num} - Substreams bumped: better "Progress" messages
Removed
- Removed
--substreams-tier1-request-stats
and--substreams-tier1-request-stats
(substreams request-stats are now always sent to clients)
v0.1.6
Fixed
- Fixed bug where
null
dmetering plugin was not able to be registered.
v0.1.5
Changed
fire{chain} tools check forks
now sorts forks by block number from ascending order (so that line you see is the current highest fork).fire{chain} tools check forks --after-block
can now be used to show only forks after a certain block number.- bump
firehose
,dmetering
andbstream
dependencies in order to get latest fixes to meter live blocks.
v0.1.4
This release bumps substreams to v1.1.10
Fixes
- Fixed: jobs would hang when flags
--substreams-state-bundle-size
and--substreams-tier1-subrequests-size
had different values. The latter flag has been completely removed, subrequests will be bound to the state bundle size.
Added
- Added support for continuous authentication via the grpc auth plugin (allowing cutoff triggered by the auth system).