diff --git a/docs/sdk-and-tools/notifier.md b/docs/sdk-and-tools/notifier.md index 97e8c4e0a..c77969f4a 100644 --- a/docs/sdk-and-tools/notifier.md +++ b/docs/sdk-and-tools/notifier.md @@ -30,14 +30,14 @@ Set up at least one observer for each shard in order to handle all the events in ![img](/technology/notifier-overview.png) In the figure above: -- The observer nodes will push block events to Notifier instance, via HTTP POST requests. There are several endpoints for this: - - `/events/push` (POST) -> it will handle all events for each round - - `/events/revert` (POST) -> if there is a reverted block, the event will be +- The observer nodes will push block events to Notifier instance, via WebSocket or HTTP requests. There are several endpoints/topics for this: + - `block events` -> it will handle all events for each round + - `revert events` -> if there is a reverted block, the event will be pushed on this route - - `/events/finalized` (POST) -> when the block has been finalized, the events + - `finalized events` -> when the block has been finalized, the events will be pushed on this route - Notifier checks locker service (via Redis) and applies deduplication -- Notifier will push events to RabbitMQ if enabled, or via Websockets. If Websockets will be enabled an additional endpoint will be available: +- Notifier will push events to RabbitMQ if enabled, or via Websocket. If Websocket will be enabled an additional endpoint will be available: - `/hub/ws` (GET) - this route can be used to manage the websocket connection subscription [comment]: # (mx-context-auto) @@ -48,13 +48,70 @@ In the figure above: ### Observer Client -On the observer side, there is a http client that will push block events to notifier service. +On the observer side, there is a WebSocket client that will push block events to notifier service. +There is also the HTTP Integration, which will be deprecated in the future. -In the observer node's configuration directory, `external.toml` config file can be configured -to enable notifier connector. The config file can be found +In the observer node's configuration directory, the `external.toml` config file can be configured +to enable events notifier connector via WebSocket or via HTTP integrations. The config file can be found [here](https://github.com/multiversx/mx-chain-go/blob/master/cmd/node/config/external.toml). -The supported config variables are as follows: +[comment]: # (mx-context-auto) + +#### WebSocket Integration + +The WebSocket integration is a generic one, and can be used for multiple outport driver integrations. +In case Elasticsearch integration is already being used with WebSocket connector, a separate config +section `HostDriversConfig` has to be set for event notifier. + +The corresponding config section for enabling the driver: + +```toml +[[HostDriversConfig]] + # This flag shall only be used for observer nodes + Enabled = true + + # This flag will start the WebSocket connector as server or client (can be "client" or "server") + Mode = "client" + + # URL for the WebSocket client/server connection + # This value represents the IP address and port number that the WebSocket client or server will use to establish a connection. + URL = "127.0.0.1:22111" + + # After a message will be sent it will wait for an ack message if this flag is enabled + WithAcknowledge = true + + # The duration in seconds to wait for an acknowledgment message, after this time passes an error will be returned + AcknowledgeTimeoutInSec = 60 + + # This flag defines the marshaller type. Currently supported: "json", "gogo protobuf" + MarshallerType = "gogo protobuf" + + # The number of seconds when the client will try again to send the data + RetryDurationInSec = 5 + + # Sets if, in case of data payload processing error, we should block or not the advancement to the next processing event. Set this to true if you wish the node to stop processing blocks if the client/server encounters errors while processing requests. + BlockingAckOnError = true + + # Set to true to drop messages if there is no active WebSocket connection to send to. + DropMessagesIfNoConnection = false + + # Defines the payload version. Version will be changed when there are breaking + # changes on payload data. The receiver/consumer will have to know how to handle different + # versions. The version will be sent as metadata in the websocket message. + Version = 1 +``` + +In this case, observer node will act as client and events notifier service will act as a server. +`Mode` option should be set to `client`. It is important to have `WithAcknowledge` set to `true` +since observer node should continue only if there is an acknowledge that the event was processed +successfully. `MarshallerType` field has to be aligned with `DataMarshallerType` on events notifier +configuration file. + +[comment]: # (mx-context-auto) + +#### HTTP Integration + +For http integration, the supported config variables are as follows: - `Enabled`: signals whether a driver should be attached when launching the node. - `UseAuthorization`: signals whether to use authorization. For testing purposes it can be set to `false`. @@ -84,6 +141,10 @@ The corresponding config section for enabling the driver: Password = "" ``` +:::info +HTTP Integration will be deprecated in the future. +::: + :::tip Due to the possible high data volume, it's not recommended to use validators as nodes to push events to Notifier Service. @@ -99,6 +160,49 @@ delays due to outport driver. In the notifier configuration directory (`cmd/notifier/config`), there is the `config.toml` file that can be used to configure the service. +There are 2 ways to connect observer node with events notifier service: +- via WebSocket integration +- via HTTP integration (which will be deprecated in the future) + +[comment]: # (mx-context-auto) + +#### WebSocket Integration + +There is a separate config section `WebSocketConnector` that has to be aligned with +`HostDriversConfig` from observer node. + +```toml +[WebSocketConnector] + # Enabled will determine if websocket connector will be enabled or not + Enabled = false + + # URL for the WebSocket client/server connection + # This value represents the IP address and port number that the WebSocket client or server will use to establish a connection. + URL = "localhost:22111" + + # This flag describes the mode to start the WebSocket connector. Can be "client" or "server" + Mode = "server" + + # Possible values: json, gogo protobuf. Should be compatible with mx-chain-node outport driver config + DataMarshallerType = "gogo protobuf" + + # Retry duration (receive/send ack signal) in seconds + RetryDurationInSec = 5 + + # Signals if in case of data payload processing error, we should send the ack signal or not + BlockingAckOnError = false + + # After a message will be sent it will wait for an ack message if this flag is enabled + WithAcknowledge = true + + # The duration in seconds to wait for an acknowledgment message, after this time passes an error will be returned + AcknowledgeTimeoutInSec = 60 +``` + +[comment]: # (mx-context-auto) + +#### HTTP Integration + The supported config variables are: - `Port`: the port on which the http server listens on. Should be the same as the port in the `ProxyUrl` described above. @@ -123,6 +227,10 @@ The `ConnectorApi` section has to be aligned with the one from observer node: CheckDuplicates = true ``` +[comment]: # (mx-context-auto) + +#### Deduplication + If `CheckDuplicates` will be set to true, notifier service will try to connect to a redis instance. In this context, redis will be used as a locker service mechanism for deduplication. This is useful in scenarios when multiple observer nodes from same shard are used to send @@ -135,9 +243,6 @@ The `Redis` section includes the following parameters as described below: # The url used to connect to a pubsub server Url = "redis://localhost:6379/0" - # The pubsub channel used for publishing/subscribing - Channel = "pub-sub" - # The master name for failover client MasterName = "mymaster" @@ -205,7 +310,7 @@ There are multiple event types: - Finalized Block event: when the block is finalized In RabbitMQ there is a separate exchange for each event type. -In Websockets setup, there is a event type field in each message. +In Websocket setup, there is a event type field in each message. The WS event is defined as follows: