Merge pull request #1 from re-nft/naz/nft-1645-add-ability-to-index-m…

…ultiple-chains

nft 1645 add ability to index multiple chains

.eslintignore

@@ -0,0 +1,7 @@
+node_modules/
+build/
+dist/
+lib/
+abi/
+db/
+model/

.eslintrc.json

@@ -0,0 +1,18 @@
+{
+    "env": {
+        "browser": true,
+        "es2021": true
+    },
+    "extends": [
+        "standard-with-typescript",
+        "plugin:prettier/recommended"
+    ],
+    "parserOptions": {
+        "ecmaVersion": "latest",
+        "sourceType": "module"
+    },
+    "rules": {
+        "max-len": ["error", { "code": 80 }],
+        "quotes": ["error", "double"]
+    }
+}

.prettierignore

@@ -0,0 +1,7 @@
+node_modules/
+build/
+dist/
+lib/
+abi/
+db/
+model/

.prettierrc

@@ -0,0 +1,5 @@
+{
+    "semi": true,
+    "singleQuote": false,
+    "printWidth": 80
+}

.vscode/settings.json

@@ -0,0 +1,8 @@
+{
+  "editor.codeActionsOnSave": {
+    "source.fixAll.eslint": true
+  },
+  "editor.formatOnSave": true,
+  "prettier.eslintIntegration": true
+}
+

README.md

@@ -1,98 +1,49 @@
-# Readme
+# reNFT v3 Indexing - Subsquid
 
 Squid is generated from `squidgen.yaml`. Reason we are using yaml file instead of using `squidgen` is because our source of truth
-is more than one contract. And the only way to index more than one contract at the same time and have all artifacts generated,
-you must use squidgen.yaml config file: https://docs.subsquid.io/basics/squid-gen/#configuration
-
-Therefore, **ignore everything below**. Keeping it for reference for now. Eventually will remove it.
-
-# Squid ABI template
-
-An experimental template is used to generate a squid that indexes EVM logs and transactions of choice from a contract address. Supports automatic ABI lookups for public contracts using the Etherscan API
-
-## Usage
-
-0. Install the [Squid CLI](https://docs.subsquid.io/squid-cli/):
-
-```sh
-npm i -g @subsquid/cli
-```
-
-1. Init the template and install the dependencies
-
-```bash
-sqd init my-abi-squid --template https://github.com/subsquid/squid-abi-template
-cd my-abi-squid
-npm i
-```
-
-2. Run `sqd generate` with the appropriate flags.
-
-```bash
-Usage: sqd generate [options]
-
-Options:
-  --address <contract>      contract address
-  --archive <url>           archive endpoint 
-  --abi <path>              (Optional) path or URL to the abi file. If omitted, the Etherscan API is used.
-  -e, --event <name...>     one or multiple events to be indexed. '*' will index all events
-  -f, --function <name...>. one or multiple contract functions to be indexed. '*' will index all functions
-  --from <block>            start indexing from the given block. 
-  --etherscan-api <url>     (Optional) an Etherscan-compatible API to fetch contract ABI by a known address. Default: https://api.etherscan.io/
-```
-
-3. Build and run the squid
-
-```bash
-sqd build
-sqd up
-sqd migration:generate
-sqd process
-```
-The indexing will start.
-
-In a separate window, start the GraphQL API server at `localhost:4350/graphql`:
-```bash
-sqd serve
-```
-
-4. Inspect `schema.graphql`, `src/processor.ts` and start hacking!
-
-For more details on how to build and deploy a squid, see the [docs](https://docs.subsquid.io).
-
-## Example
-### Generate
-```bash
-npx squid-gen-abi \
---address 0x2E645469f354BB4F5c8a05B3b30A929361cf77eC \
---archive https://eth.archive.subsquid.io \
---event NewGravatar \
---event UpdatedGravatar \
---function createGravatar \
---from 6000000
-```
-### Explore
-```gql
-query MyQuery {
-  events(orderBy: block_timestamp_ASC) {
-    id
-    name
-    block {
-      number
-      timestamp
-    }
-    ... on NewGravatarEvent {
-      id0
-      imageUrl
-      owner
-    }
-    ... on UpdatedGravatarEvent {
-      id0
-      imageUrl
-      owner
-    }
-  }
-}
-```
-<img width="1000" alt="Example" src="https://user-images.githubusercontent.com/61732514/214889375-20cd1945-0124-4924-a1dd-3f1a07ddd6ab.png">
+is more than one contract, see [this](https://docs.subsquid.io/basics/squid-gen/#configuration).
 
+Once it's generated there is some manual work to be done, however. You cannot use the generated code out of the box.
+
+## Development
+
+To spin up a docker db:
+
+`sqd up`
+
+If you need to generate a new migrations script, because you have made changes to `schema.graphql`, run:
+
+`sqd migrations:generate`
+
+You can learn about all the commands at your disposal in `commands.json`
+
+To connect to the database and explore the data you can use various clients, for example:
+
+- DBeaver
+- DBVisualiser
+
+Here is the default data you'd need to use to connect to the db:
+
+## Deploying to Aquarium
+
+I had issues when both v1 and v2 were syncing at the same time. API endpoint wouldn't be available.
+
+To remedy, I had to hibernate v1 and restart v2.
+
+![squid db config](./assets/squid-db.png)
+
+Password is `postgres`
+
+---
+
+To start sepolia indexing run:
+
+`sqd process:eth-sepolia`
+
+## Optimisations
+
+Right now, the data goes into aquarium hosted db. Then, api will pull that data by making post requests to the squid graphql server.
+
+If we are seeing slow responses, then we can save indexed data straight into the same db as the api.
+
+This wasn't done originally because of migrations. Subsquid has its own migrations and our api does too. To avoid potential conflicts, it was decided to keep the dbs separate to begin with.

commands.json

@@ -34,11 +34,11 @@
       "migration:generate": {
         "description": "Generate a DB migration matching the TypeORM entities",
         "deps": ["build", "migration:clean"],
-        "cmd": ["squid-typeorm-migration", "generate"],
+        "cmd": ["squid-typeorm-migration", "generate"]
       },
       "migration:clean": {
         "description": "Clean the migrations folder",
-        "cmd": ["npx", "--yes", "rimraf", "./db/migrations"],
+        "cmd": ["npx", "--yes", "rimraf", "./db/migrations"]
       },
       "migration": {
         "deps": ["build"],
@@ -53,15 +53,26 @@
         "description": "Generate data access classes for an ABI file(s) in the ./abi folder",
         "cmd": ["squid-evm-typegen", "./src/abi", {"glob": "./abi/*.json"}, "--multicall"]
       },
-      "process": {
-        "description": "Load .env and start the squid processor",
+      "process:eth-sepolia": {
+        "description": "Load .env and start the ETH-SEPOLIA squid processor",
+        "deps": ["build", "migration:apply"],
+        "cmd": ["node", "--require=dotenv/config", "lib/eth-sepolia/main.js"]
+      },
+      "process:prod:eth-sepolia": {
+        "description": "Start the squid processor",
+        "deps": ["migration:apply"],
+        "cmd": ["node", "lib/eth-sepolia/main.js"],
+        "hidden": true
+      },
+      "process:polygon-mumbai": {
+        "description": "Load .env and start the ETH-SEPOLIA squid processor",
         "deps": ["build", "migration:apply"],
-        "cmd": ["node", "--require=dotenv/config", "lib/main.js"]
+        "cmd": ["node", "--require=dotenv/config", "lib/polygon-mumbai/main.js"]
       },
-      "process:prod": {
+      "process:prod:polygon-mumbai": {
         "description": "Start the squid processor",
         "deps": ["migration:apply"],
-        "cmd": ["node", "lib/main.js"],
+        "cmd": ["node", "lib/polygon-mumbai/main.js"],
         "hidden": true
       },
       "serve": {