Development

docs/developers/developer-reference/serialization-format.md

@@ -574,8 +574,12 @@ export const ExampleTable = () => (
   </table>
 );
 
+[comment]: # (mx-context-auto)
+
 ## Defaults
 
+[comment]: # (mx-context-auto)
+
 ### Built-in defaults
 
 The serialization format naturally supports defaults for most types.
@@ -606,6 +610,8 @@ For instance, for all numeric types, zero is the default value, because we repre
 | `DayOfWeek` (see example above)           | `DayOfWeek::Monday`            |
 | `EnumWithEverything` (see example above)  | `EnumWithEverything::Default`  |
 
+[comment]: # (mx-context-auto)
+
 ### Types that have no defaults 
 
 Certain types have no values that can be represented as an empty buffer, and therefore they have no default value.
@@ -637,6 +643,8 @@ If you need the default, one workaround is to place these structures inside an `
 
 There is, however, another way to do it: for custom structures it is possible to define custom defaults, as we will see in the next section.
 
+[comment]: # (mx-context-auto)
+
 ### Custom defaults
 
 A structure that does not have a natural default value can receive one via custom code. First of all this applies to structures, but it can also be useful for some enums.

docs/sdk-and-tools/elastic-indexer.md

@@ -0,0 +1,117 @@
+---
+id: indexer
+title: Elasticindexer service
+---
+
+[comment]: # (mx-abstract)
+
+## Overview
+
+:::tip
+This feature will work starting from `rc/v1.6.0` version of the node
+:::
+
+
+A MultiversX observer node can send messages over `WebSocket` to an elastic indexer service, which will process and index the data in an Elasticsearch database.
+
+The GitHub repository for the `elasticindexer` service can be found [here](https://github.com/multiversx/mx-chain-es-indexer-go/tree/rc/v1.6.0).
+
+[comment]: # (mx-context-auto)
+
+## Architectural Overview
+
+The observer node in the network will be connected to `elasticindexer` service.
+
+:::important
+Set up one observer for each shard in order to handle all the data in the chain.
+:::
+
+![img](/technology/indexer.png)
+
+In the figure above:
+- The observer nodes will send WebSocket messages to the `elasticindexer` service.
+- The `elasticindexer` service will receive and process the WebSocket messages and index data in an Elasticsearch cluster.
+
+[comment]: # (mx-context-auto)
+
+## Set up observer and elasticindexer
+
+[comment]: # (mx-context-auto)
+
+### Observer Client
+
+On the observer side, there is a WebSocket host that will send messages to the elasticindexer service.
+
+In the observer node's configuration directory, `external.toml` config file can be configured
+to enable host driver config. The config file can be found
+[here](https://github.com/multiversx/mx-chain-go/blob/rc/v1.6.0/cmd/node/config/external.toml).
+
+The corresponding config section for enabling the driver:
+
+```toml
+[[HostDriversConfig]]
+    # This flag shall only be used for observer nodes
+    Enabled = true
+    # This flag will start the WebSocket connector as server or client (can be "client" or "server")
+    Mode = "client"
+    # URL for the WebSocket client/server connection
+    # This value represents the IP address and port number that the WebSocket client or server will use to establish a connection.
+    URL = "127.0.0.1:22111"
+    # After a message will be sent it will wait for an ack message if this flag is enabled
+    WithAcknowledge = true
+    # The duration in seconds to wait for an acknowledgment message, after this time passes an error will be returned
+    AcknowledgeTimeoutInSec = 60
+    # Possible values: json, gogo protobuf. Should be compatible with mx-chain-es-indexer-go config
+    MarshallerType = "json"
+    # The number of seconds when the client will try again to send the data
+    RetryDurationInSec = 5
+    # Sets if, in case of data payload processing error, we should block or not the advancement to the next processing event. Set this to true if you wish the node to stop processing blocks if the client/server encounters errors while processing requests.
+    BlockingAckOnError = true
+    # Set to true to drop messages if there is no active WebSocket connection to send to.
+    DropMessagesIfNoConnection = false
+```
+
+[comment]: # (mx-context-auto)
+
+### Elasticindexer service
+
+In the `elasticindexer` configuration directory (`cmd/elasticindexer/config`), there is the `prefs.toml`
+file that can be used to configure the service.
+
+
+The `config.web-socket` section has to be aligned with the one from observer node:
+
+```toml
+[config]
+    disabled-indices = []
+    [config.web-socket]
+        # URL for the WebSocket client/server connection
+        # This value represents the IP address and port number that the WebSocket client or server will use to establish a connection.
+        url = "localhost:22111"
+        # This flag describes the mode to start the WebSocket connector. Can be "client" or "server"
+        mode = "server"
+        # Possible values: json, gogo protobuf. Should be compatible with mx-chain-node outport driver config
+        data-marshaller-type = "json"
+        # Retry duration (receive/send ack signal) in seconds
+        retry-duration-in-seconds = 5
+        # Signals if in case of data payload processing error, we should send the ack signal or not
+        blocking-ack-on-error = true
+        # After a message will be sent it will wait for an ack message if this flag is enabled
+        with-acknowledge = true
+        # The duration in seconds to wait for an acknowledgment message, after this time passes an error will be returned
+        acknowledge-timeout-in-seconds = 50
+```
+
+The corresponding config section for the Elasticsearch section:
+
+```toml
+[config.elastic-cluster]
+        use-kibana = false
+        url = "http://localhost:9200"
+        username = ""
+        password = ""
+        bulk-request-max-size-in-bytes = 4194304 # 4MB
+```
+
+For more details on `elasticindexer` service setup, please follow the **Install** and **Launching**
+sections from [README](https://github.com/multiversx/mx-chain-es-indexer-go) in the repository.

docs/sdk-and-tools/elastic-search.md

@@ -19,6 +19,21 @@ Our implementation uses a concept of a queue and makes sure that everything is b
 
 ## Setup
 
+[comment]: # (mx-context-auto)
+
+### Option 1
+
+Set up four observers, each corresponding to a shard, with the `WebSocketHost` enabled, and create an instance of the `elasticindexer` service.
+Follow the instructions provided on [this](/sdk-and-tools/indexer) page to accomplish this.
+
+[comment]: # (mx-context-auto)
+
+### Option 2 
+
+:::warning
+We plan to remove this option in the `rc/v1.7.0` release.
+:::
+
 In order to set up an observer that indexes in Elasticsearch, one has to update the `external.toml` file from the node's
 configuration directory. A minimum configuration would have `Enabled` set to `true` and the rest of the fields updated
 accordingly (`URL`, `Username`, `Password`).

docs/sdk-and-tools/sdk-dapp/internal-processes/guardians.md

@@ -30,11 +30,14 @@ transaction.setOptions(TransactionOptions.withOptions({
 }));
 ```
 
+[comment]: # (mx-context-auto)
+
 ### Signing the transaction
 
 Once these fields are set, the transaction must be signed by both Guardian (adding the `guardianSignuature` field) and UserSigner (adding the `signature` field).
 All signing providers (except Ledger) take care internally of formatting the transaction, as described above (excluding adding the extra `gasLimit`).
 
+[comment]: # (mx-context-auto)
 
 #### Signing the transaction with Ledger 
 
@@ -43,7 +46,6 @@ After formatting the transaction and applying the signature provided by the Ledg
 ```js
 import { WalletProvider } from '@multiversx/sdk-web-wallet-provider';
 
-
 const walletProvider = new WalletProvider('https://www.wallet.multiversx.com/dapp/init');
 walletProvider.guardTransactions(transactions, {
     callbackUrl: encodeURIComponent('https://my-dapp.com'),
@@ -55,4 +57,3 @@ Once transactions are back from the web wallet, they can be retrieved as follows
 const signedTransactions = new WalletProvider('https://www.wallet.multiversx.com/dapp/init').getTransactionsFromWalletUrl();
 ```
 
-

docs/validators/node-configuration.md

@@ -3,11 +3,15 @@ id: node-configuration
 title: Node Configuration
 ---
 
+[comment]: # (mx-context-auto)
+
 ## Introduction
 
 The node relies on some configuration files that are meant to allow the node operator to easily change some values
 that won't require a code change, a new release, or so on.
 
+[comment]: # (mx-context-auto)
+
 ## Configuration files
 
 All the configuration files are located by default in the `config` directory that resides near the node's binary. The paths can be changed
@@ -73,6 +77,8 @@ Below you can find an example of how the configuration files look like for the `
 - `ratings.toml` contains the parameters used for the nodes' rating mechanism, for example, the start rating, decrease steps, and so on
 - `systemSmartContractsConfig.toml` contains System Smart Contracts configurable values, such as parameters for Staking, ESDT, or Governance
 
+[comment]: # (mx-context-auto)
+
 ### Overriding config.toml values
 
 As mentioned in the above descriptions, `prefs.toml` is not overwritten by the installation scripts when performing an upgrade.

docs/wallet/web-wallet.md

@@ -257,6 +257,8 @@ The Guardians feature allows you to change your guardian in situations where you
 
 This can be achieved by clicking th *Change Guardian* button from the Guardian's main dashboard. Afterwards you will be faced with two options:
 
+[comment]: # (mx-context-auto)
+
 #### **Changing guardian with 20 epochs pre-registration time**
 
 1. **If you have lost access to the authenticator app**, you can address this issue by selecting the option *"I cannot access my guardian"* from the window mentioned below.
@@ -287,7 +289,6 @@ Showcasing the details of the request:
 
 ![img](/wallet/web-wallet/guardian_change6.png)
 
-
 If you were the one who initiated the request to change the guardian and you still have access to the currently active guardian, you will encounter two additional options:
 
 - Skip the cooldown period by clicking the **Confirm Change** button: By selecting this option, you can proceed with the change of the guardian without waiting for the cooldown period to elapse.
@@ -304,6 +305,8 @@ If it was not you the one triggering the change of the guardian, it may be the c
 We recommend the first solution. Even though you have your funds staked, the unbound period (10 epochs) is shorther than the guardian activation period so that you have enough time to unstake and safely move them to a safe account.
 :::
 
+[comment]: # (mx-context-auto)
+
 #### **Instantly changing guardian**
 
 To instantly change your guardian you must use the Change Guardian. You must follow a different set of steps compared to the previous *Change Guardian* way:

docs/wallet/xportal.md

@@ -53,7 +53,7 @@ Yes, the Invisible Guardian can be changed, and there are two ways in which this
 
 - If the user has access to the current Guardian the change can be done immediately by signing a transaction;
 - If the user doesn’t have access to the current Guardian the process implies a 20-day cooldown period before the change of Guardian takes place.
- 
+
 [comment]: # (mx-context-auto)
 
 ## Why do I have to wait 20 days to activate or change an Invisible Guardian?
@@ -75,6 +75,7 @@ Aside from the extra security and the peace of mind that goes hand in hand with
 ## What if something happens to my device or I change it?
 
 The Invisible Guardian is stored locally on your device, but the first step of setting it up is creating an encrypted backup containing the Guardian. This makes importing it on a different device as easy as it gets, all you need to do is to provide the password associated with the guarded backup.
+
 [comment]: # (mx-context-auto)
 
 ## What happens if my secret phrase is compromised?

sidebars.js

@@ -171,6 +171,7 @@ const sidebars = {
         label: "Elasticsearch",
         items: [
           "sdk-and-tools/elastic-search",
+          "sdk-and-tools/indexer",
           "sdk-and-tools/es-index-wrong-mapping",
           {
             type: "category",