All contracts should have a startBlock that is public + set at deploy time

sqlite file sent in dm (exceeded github size limit).

Query to replace:

select  tx_hash, block_number from receipts where tx_hash not in (select tx_hash from eth_txes);

Issue does not seem to occur the oether way around:

select tx_hash from eth_txes where tx_hash not in (select tx_hash from receipts);`

Issue

last indexed is stored here

but in the event of an error we just keep indexing and skip that log entirely.

Fix

Loop should return an error. When last indexed is used in the next block, 1 will be subtracted and this will get inserted

One secondary fix is to insert everything in a single transaction. Also may be a bit faster. This should help maintain monotonic validity of blocks throughout #114

Right now, when responding to incidents, monitoring, etc we have a lot of issues finding a public rpc to check against. For these systems, it's crucial to have a live, accessible rpc to be able to check against. To that end, we're going to build an rpc proxy with the following functionality

• given a list of rpcs, select the one with the lowest latency intermittently
• get a list of rpcs for a chainid and get the one w/ the lowest latency (#134 )
• run a proxy that supports lowest-latency handling and fallbacks to other rpcs
• take a config file for non-public rpcs
• configs that allow double rpc checking for harmony
• integration w/ #114 for get logs request (based on config)

TODOs after #755 is merged:

• Replace handle with receiveBaseMessage
• Leave a single "BaseClient" implementation that enforces the optimistic period upon receiving a base message (after #756 SystemRouter will not be inheriting from the clients).

ReplicaManager should only contain logistical code, and as minimal as state as possible.

One workflow we have a lot in our codebase looks like this:

It'd be kind of nice for debuggability if we didn't have to generate these by hand and could use abigen to handle it for us. I think I'd still use the json parser to tie the events to the topic, but we could build in empty hash checking and constant generation

Testing cases for:

• Multiple messages should be dispatched with different latencies being set
• Messages should be able to be processed in a non-FIFO way, being processed as each leaf's optimistic message timing is hit.

Optimistic Message Timing is:

• Signed Updater Message + Optimistic Second leaf
• Time of Relayer confirming relayed update on Replica
• Time confirmed + Optimistic Seconds = Acceptable tx to process

Publish actions (composite and contrib) from monorepo into new repos automaitcally so they can be used by other repos

#753 replaced prettier with forge fmt.

The dependencies were not removed in order to avoid merge conflicts, when the branch is eventually going to be merged into master.

However, once #654 is merged, the dependencies could be safely removed.

https://pkg.go.dev/go.opentelemetry.io/otel/metric#NewAddConfig

Sending a message incurs costs on both local and remote chain. The costs are:

• Signing updates on local Home contract. This needs to be done every X minutes (assuming there were new messages dispatched). Requires spending local chain gas.
• Relaying updates to the remote ReplicaManager contracts. This needs to be done whenever a new update is signed on Home for every remote chain that has incoming messages in that update. Requires spending remote chain gas.
• Calling proveAndProcess on the remote ReplicaManager to execute the message. Requires spending remote chain gas.
  • prove part consumes a fixed amount of gas regardless of message.
  • process part consumes a variable amount of gas, and might additionally require a gas airdrop.

Thus, it makes sense to split the total fee this way:

• Amortize "signing updates" costs among all messages between two updates.
• Amortize "relay updates on given remote" costs among all messages to given remote chain between the updates.
• Assign a fixed (assuming fixed remote chain gas token/unit price) fee to cover proveAndProcess part without executing the message.
• Assign a variable fee to cover executing the message (including gas airdrop and potentially other executing options).

    Hmmm maybe keep a removed list?

Originally posted by @joecroninallen in #325 (comment)

There are a few things that have come up in production that would greatly improve the usability of omnirpc.

• Submit transactions to all endpoints
• RPC Types: Right now, all requests go through all endpoints. It would be nice to have a meter for this
• Better rate limiting (see: https://github.com/ethereum-optimism/optimism/tree/develop/proxyd)
• Definable max batch sizes per rpc endpoint and omnirpc level batch splitting
• Way to make a bath request across endpoints
• Metrics (Especially on latest block number of all chains)
• Better backoff support

Right now, between sanguine/scribe and sanguine/agents, there is a lot of replicated DB code. Some attempts to resolve this have been in #125 where there is a new dbcommon dir, but it is incomplete, only filling in some helpers and generic functions.

Currently, clickhouse primary keys are done using mysql style primaryKey directives:

These won't work in clickhouse since PrimaryKey is specified at table creation time (see here)

You need to add primary keys based on this. OrderBy should use the same thing.

Let me know when this is done and we'll merge the pr then move on to implementing a ReplacingMergeTree

For this

#692 paves the way for automatically generating typecast files based on the contracttype file

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• fix(deps): update dependency @openzeppelin/contracts to v5
• fix(deps): update dependency @openzeppelin/contracts-upgradeable to v5
• chore(deps): update packages/contracts-core/lib/create3-factory digest to e3e57d9
• chore(deps): update packages/contracts-core/lib/forge-std digest to 978ac6f
• fix(deps): update github.com/syndtr/goleveldb digest to 126854a
• chore(deps): update dependency @babel/eslint-parser to v7.24.5
• chore(deps): update dependency @types/chai to v4.3.16
• chore(deps): update dependency autoprefixer to v10.4.19
• chore(deps): update dependency dayjs to v1.11.11
• chore(deps): update dependency hardhat to v2.22.4
• chore(deps): update dependency husky to v8.0.3
• chore(deps): update dependency postcss to v8.4.38
• chore(deps): update dependency tailwindcss to v3.4.3
• chore(deps): update dependency typescript to v4.9.5
• chore(deps): update graphqlcodegenerator monorepo (@graphql-codegen/cli, @graphql-codegen/client-preset, @graphql-codegen/introspection)
• chore(deps): update mariadb docker tag to v10.11.7
• chore(deps): update yarn to v1.22.22
• fix(deps): update dependency @cloudflare/next-on-pages to v1.11.3
• fix(deps): update dependency @headlessui/react to v1.7.19
• fix(deps): update dependency @popperjs/core to v2.11.8
• fix(deps): update dependency @rainbow-me/rainbowkit to v2.0.8
• fix(deps): update dependency @reduxjs/toolkit to v2.2.5
• fix(deps): update dependency @tailwindcss/typography to v0.5.13
• fix(deps): update dependency clsx to v2.1.1
• fix(deps): update dependency dayjs to v1.11.11
• fix(deps): update dependency react-redux to v9.1.2
• fix(deps): update dependency tailwindcss to v3.4.3
• fix(deps): update emotion monorepo to v11.11.4 (@emotion/react, @emotion/styled)
• fix(deps): update module github.com/99designs/gqlgen to v0.17.47
• fix(deps): update module github.com/99designs/gqlgen to v0.17.47
• fix(deps): update module github.com/99designs/gqlgen to v0.17.47
• fix(deps): update module github.com/augustoroman/hexdump to v0.0.0-20231204223853-3694912baadb
• fix(deps): update module github.com/btcsuite/btcd/btcec/v2 to v2.3.3
• fix(deps): update module github.com/coinbase/rosetta-sdk-go to v0.8.5
• fix(deps): update module github.com/denrianweiss/tracely to v0.0.0-20240320084351-309c1780e6e1
• fix(deps): update module github.com/dgraph-io/ristretto to v0.1.1
• fix(deps): update module github.com/gartnera/gcloud to v0.0.19
• fix(deps): update module github.com/gen2brain/beeep to v0.0.0-20240516210008-9c006672e7f4
• fix(deps): update module github.com/googleapis/gax-go/v2 to v2.12.4
• fix(deps): update module github.com/hashicorp/terraform-provider-kubernetes to v1.13.4
• fix(deps): update module github.com/hedzr/log to v1.6.25
• fix(deps): update module github.com/kendru/darwin/go/depgraph to v0.0.0-20230809052043-4d1c7e9d1767
• fix(deps): update module github.com/libp2p/go-libp2p-pubsub to v0.10.1
• fix(deps): update module github.com/migueleliasweb/go-github-mock to v0.0.23
• fix(deps): update module github.com/migueleliasweb/go-github-mock to v0.0.23
• fix(deps): update module github.com/miguelmota/go-ethereum-hdwallet to v0.1.2
• fix(deps): update module github.com/multiformats/go-multiaddr to v0.12.4
• fix(deps): update module github.com/nsmithuk/local-kms to v0.0.0-20231025161602-e4d9425700fe
• fix(deps): update module github.com/sirupsen/logrus to v1.9.3
• fix(deps): update module github.com/thoas/go-funk to v0.9.3
• fix(deps): update module github.com/ugorji/go/codec to v1.2.12
• fix(deps): update module github.com/uptrace/opentelemetry-go-extra/otelzap to v0.2.4
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/urfave/cli/v2 to v2.27.2
• fix(deps): update module github.com/vburenin/ifacemaker to v1.2.1
• fix(deps): update module github.com/vektah/gqlparser/v2 to v2.5.12
• fix(deps): update module github.com/vektah/gqlparser/v2 to v2.5.12
• fix(deps): update module github.com/vishalkuo/bimap to v0.0.0-20230830142743-a9fb9b52066c
• fix(deps): update module go.uber.org/automaxprocs to v1.5.3
• fix(deps): update module golang.org/x/exp to v0.0.0-20240506185415-9bf2ced13842
• fix(deps): update module golang.org/x/exp to v0.0.0-20240506185415-9bf2ced13842
• fix(deps): update module golang.org/x/exp to v0.0.0-20240506185415-9bf2ced13842
• fix(deps): update module golang.org/x/exp to v0.0.0-20240506185415-9bf2ced13842
• fix(deps): update module golang.org/x/exp to v0.0.0-20240506185415-9bf2ced13842
• fix(deps): update module golang.org/x/exp to v0.0.0-20240506185415-9bf2ced13842
• fix(deps): update module golang.org/x/exp to v0.0.0-20240506185415-9bf2ced13842
• fix(deps): update module golang.org/x/exp to v0.0.0-20240506185415-9bf2ced13842
• fix(deps): update module golang.org/x/exp to v0.0.0-20240506185415-9bf2ced13842
• fix(deps): update module google.golang.org/genproto/googleapis/api to v0.0.0-20240520151616-dc85e6b867a5
• fix(deps): update module gorm.io/driver/mysql to v1.5.6
• fix(deps): update module gorm.io/driver/mysql to v1.5.6
• fix(deps): update module gorm.io/driver/mysql to v1.5.6
• fix(deps): update module gorm.io/driver/mysql to v1.5.6
• fix(deps): update module gorm.io/driver/mysql to v1.5.6
• fix(deps): update module gorm.io/driver/mysql to v1.5.6
• fix(deps): update module gorm.io/driver/mysql to v1.5.6
• fix(deps): update module gorm.io/driver/mysql to v1.5.6
• fix(deps): update module gorm.io/driver/mysql to v1.5.6
• fix(deps): update module gorm.io/gorm to v1.25.10
• fix(deps): update module gorm.io/gorm to v1.25.10
• fix(deps): update module gorm.io/gorm to v1.25.10
• fix(deps): update module gorm.io/gorm to v1.25.10
• fix(deps): update module gorm.io/gorm to v1.25.10
• fix(deps): update module gorm.io/gorm to v1.25.10
• fix(deps): update module gorm.io/gorm to v1.25.10
• fix(deps): update module gorm.io/gorm to v1.25.10
• fix(deps): update module gorm.io/gorm to v1.25.10
• fix(deps): update module gorm.io/gorm to v1.25.10
• fix(deps): update module gorm.io/gorm to v1.25.10
• chore(deps): update crazy-max/ghaction-import-gpg action to v5.4.0
• chore(deps): update crytic/slither-action action to v0.4.0
• chore(deps): update dependency @babel/core to v7.24.5
• chore(deps): update dependency @storybook/testing-library to ^0.2.0
• chore(deps): update dependency @testing-library/react to v14.3.1
• chore(deps): update dependency @typechain/ethers-v5 to v10.2.1
• chore(deps): update dependency @types/react to v18.3.2
• chore(deps): update dependency lint-staged to v11.2.6
• chore(deps): update dependency markdownlint to ^0.34.0
• chore(deps): update dependency markdownlint-cli2 to v0.13.0
• chore(deps): update dependency node to v18.20.2
• chore(deps): update dependency node to v18.20.2
• chore(deps): update dependency size-limit to v8.2.6
• chore(deps): update dependency solhint to v3.6.2
• chore(deps): update dependency solhint to v4.5.4
• chore(deps): update dependency solhint-plugin-prettier to ^0.1.0
• chore(deps): update dependency tailwindcss to v3.4.3
• chore(deps): update dependency tslib to v2.6.2
• chore(deps): update dependency typechain to v8.3.2
• chore(deps): update dependency typedoc to ^0.25.0
• chore(deps): update dependency typescript to v4.9.5
• chore(deps): update dependency typescript to v5.4.5
• chore(deps): update dependency typescript to v5.4.5
• chore(deps): update dependency typescript to v5.4.5
• chore(deps): update dependency typescript to ~5.4.0
• chore(deps): update dependency vercel to v33.7.1
• chore(deps): update franzdiebold/github-env-vars-action action to v1.3.0
• chore(deps): update helm release clickhouse to v3.8.0
• chore(deps): update helm release jaeger to v0.74.1
• chore(deps): update helm release mysql to v9.23.0
• chore(deps): update helm release mysql to v9.23.0
• chore(deps): update helm release mysql to v9.23.0
• chore(deps): update helm release mysql to v9.23.0
• chore(deps): update helm/chart-releaser-action action to v1.6.0
• chore(deps): update helm/chart-testing-action action to v2.6.1
• chore(deps): update helm/kind-action action to v1.10.0
• chore(deps): update jaxxstorm/action-install-gh-release action to v1.12.0
• chore(deps): update mathieudutour/github-tag-action action to v6.2
• chore(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• chore(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• chore(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• chore(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• chore(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• chore(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• chore(deps): update module github.com/ipfs/go-ipld-format to v0.6.0
• chore(deps): update module github.com/ipfs/go-ipld-format to v0.6.0
• chore(deps): update module golang.org/x/oauth2 to v0.20.0
• chore(deps): update module golang.org/x/oauth2 to v0.20.0
• chore(deps): update module golang.org/x/oauth2 to v0.20.0
• chore(deps): update module golang.org/x/oauth2 to v0.20.0
• chore(deps): update module google.golang.org/api to v0.181.0
• chore(deps): update module k8s.io/kubectl to v0.30.1
• chore(deps): update node.js to v14.18.2
• chore(deps): update ppremk/lfs-warning action to v3.3
• chore(deps): update prom/prometheus docker tag to v2.52.0
• chore(deps): update react monorepo (@types/react, @types/react-dom, react, react-dom)
• chore(deps): update rinchsan/renovate-config-validator action to v0.2.0
• chore(deps): update scribemd/docker-cache action to v0.5.0
• chore(deps): update wandalen/wretry.action action to v1.4.10
• fix(deps): update aws-sdk-go-v2 monorepo (github.com/aws/aws-sdk-go-v2, github.com/aws/aws-sdk-go-v2/config, github.com/aws/aws-sdk-go-v2/service/kms)
• fix(deps): update dependency @apollo/client to v3.10.4
• fix(deps): update dependency @babel/core to v7.24.5
• fix(deps): update dependency @changesets/cli to v2.27.2
• fix(deps): update dependency @ethersproject/address to v5.7.0
• fix(deps): update dependency @headlessui/react to v1.7.19
• fix(deps): update dependency @mui/material to v5.15.18
• fix(deps): update dependency @segment/analytics-next to v1.69.0
• fix(deps): update dependency @types/node to v18.19.33
• fix(deps): update dependency @vercel/analytics to v1.2.2
• fix(deps): update dependency @wagmi/connectors to v4.3.10
• fix(deps): update dependency @wagmi/core to v2.10.2
• fix(deps): update dependency babel-jest to v29.7.0
• fix(deps): update dependency chart.js to v3.9.1
• fix(deps): update dependency date-fns to v2.30.0
• fix(deps): update dependency dotenv to v16.4.5
• fix(deps): update dependency ethers to v6.12.1
• fix(deps): update dependency graphql to v16.8.1
• fix(deps): update dependency next to v14.2.3
• fix(deps): update dependency react-chartjs-2 to v4.3.1
• fix(deps): update dependency recharts to v2.12.7
• fix(deps): update dependency sharp to ^0.33.0
• fix(deps): update dependency solidity-coverage to v0.8.12
• fix(deps): update dependency tailwind-merge to v1.14.0
• fix(deps): update dependency tiny-invariant to v1.3.3
• fix(deps): update dependency ts-node to v10.9.2
• fix(deps): update dependency typescript to v5.4.5
• fix(deps): update dependency viem to v2.10.9
• fix(deps): update dependency wagmi to v2.9.2
• fix(deps): update docusaurus monorepo to v3.3.2 (@docusaurus/core, @docusaurus/logger, @docusaurus/module-type-aliases, @docusaurus/plugin-content-docs, @docusaurus/preset-classic, @docusaurus/theme-common, @docusaurus/tsconfig, @docusaurus/types, @docusaurus/utils, @docusaurus/utils-common, @docusaurus/utils-validation)
• fix(deps): update kubernetes packages to v0.30.1 (k8s.io/apiextensions-apiserver, k8s.io/apimachinery, k8s.io/client-go, k8s.io/kubectl)
• fix(deps): update module cloud.google.com/go/bigtable to v1.23.0
• fix(deps): update module cloud.google.com/go/kms to v1.17.0
• fix(deps): update module github.com/aws/smithy-go to v1.20.2
• fix(deps): update module github.com/aws/smithy-go to v1.20.2
• fix(deps): update module github.com/aws/smithy-go to v1.20.2
• fix(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• fix(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• fix(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• fix(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• fix(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• fix(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• fix(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• fix(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• fix(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• fix(deps): update module github.com/brianvoe/gofakeit/v6 to v6.28.0
• fix(deps): update module github.com/clickhouse/clickhouse-go/v2 to v2.24.0
• fix(deps): update module github.com/decred/dcrd/dcrec/secp256k1/v4 to v4.3.0
• fix(deps): update module github.com/decred/dcrd/dcrec/secp256k1/v4 to v4.3.0
• fix(deps): update module github.com/dubonzi/otelresty to v1.3.0
• fix(deps): update module github.com/gin-contrib/cors to v1.7.2
• fix(deps): update module github.com/gin-gonic/gin to v1.10.0
• fix(deps): update module github.com/gin-gonic/gin to v1.10.0
• fix(deps): update module github.com/gin-gonic/gin to v1.10.0
• fix(deps): update module github.com/gin-gonic/gin to v1.10.0
• fix(deps): update module github.com/gin-gonic/gin to v1.10.0
• fix(deps): update module github.com/gin-gonic/gin to v1.10.0
• fix(deps): update module github.com/gin-gonic/gin to v1.10.0
• fix(deps): update module github.com/gin-gonic/gin to v1.10.0
• fix(deps): update module github.com/gin-gonic/gin to v1.10.0
• fix(deps): update module github.com/go-git/go-git/v5 to v5.12.0
• fix(deps): update module github.com/go-git/go-git/v5 to v5.12.0
• fix(deps): update module github.com/go-resty/resty/v2 to v2.13.1
• fix(deps): update module github.com/go-resty/resty/v2 to v2.13.1
• fix(deps): update module github.com/go-resty/resty/v2 to v2.13.1
• fix(deps): update module github.com/google/uuid to v1.6.0
• fix(deps): update module github.com/google/uuid to v1.6.0
• fix(deps): update module github.com/google/uuid to v1.6.0
• fix(deps): update module github.com/google/uuid to v1.6.0
• fix(deps): update module github.com/google/uuid to v1.6.0
• fix(deps): update module github.com/google/uuid to v1.6.0
• fix(deps): update module github.com/grpc-ecosystem/grpc-gateway/v2 to v2.20.0
• fix(deps): update module github.com/hashicorp/consul/sdk to v0.16.0
• fix(deps): update module github.com/hashicorp/go-hclog to v1.6.3
• fix(deps): update module github.com/hashicorp/go-plugin to v1.6.1
• fix(deps): update module github.com/hashicorp/hcl/v2 to v2.20.1
• fix(deps): update module github.com/hashicorp/terraform-exec to v0.21.0
• fix(deps): update module github.com/hashicorp/terraform-plugin-go to v0.23.0
• fix(deps): update module github.com/hashicorp/terraform-plugin-go to v0.23.0
• fix(deps): update module github.com/hashicorp/terraform-plugin-log to v0.9.0
• fix(deps): update module github.com/hashicorp/terraform-plugin-mux to v0.16.0
• fix(deps): update module github.com/hashicorp/terraform-plugin-sdk/v2 to v2.34.0
• fix(deps): update module github.com/hashicorp/terraform-plugin-sdk/v2 to v2.34.0
• fix(deps): update module github.com/hashicorp/terraform-plugin-sdk/v2 to v2.34.0
• fix(deps): update module github.com/hashicorp/terraform-plugin-sdk/v2 to v2.34.0
• fix(deps): update module github.com/hedzr/cmdr to v1.11.26
• fix(deps): update module github.com/invopop/jsonschema to v0.12.0
• fix(deps): update module github.com/jarcoal/httpmock to v1.3.1
• fix(deps): update module github.com/jarcoal/httpmock to v1.3.1
• fix(deps): update module github.com/jellydator/ttlcache/v3 to v3.2.0
• fix(deps): update module github.com/josephburnett/jd to v1.8.1
• fix(deps): update module github.com/libp2p/go-libp2p to v0.34.0
• fix(deps): update module github.com/libp2p/go-libp2p to v0.34.0
• fix(deps): update module github.com/lmittmann/w3 to v0.16.1
• fix(deps): update module github.com/lmittmann/w3 to v0.16.1
• fix(deps): update module github.com/lmittmann/w3 to v0.16.1
• fix(deps): update module github.com/lmittmann/w3 to v0.16.1
• fix(deps): update module github.com/lmittmann/w3 to v0.16.1
• fix(deps): update module github.com/otiai10/copy to v1.14.0
• fix(deps): update module github.com/prometheus/client_golang to v1.19.1
• fix(deps): update module github.com/prometheus/client_golang to v1.19.1
• fix(deps): update module github.com/puzpuzpuz/xsync to v1.5.2
• fix(deps): update module github.com/ravilushqa/otelgqlgen to v0.15.0
• fix(deps): update module github.com/ravilushqa/otelgqlgen to v0.15.0
• fix(deps): update module github.com/richardwilkes/toolbox to v1.113.0
• fix(deps): update module github.com/richardwilkes/toolbox to v1.113.0
• fix(deps): update module github.com/richardwilkes/toolbox to v1.113.0
• fix(deps): update module github.com/richardwilkes/toolbox to v1.113.0
• fix(deps): update module github.com/richardwilkes/toolbox to v1.113.0
• fix(deps): update module github.com/richardwilkes/toolbox to v1.113.0
• fix(deps): update module github.com/sethvargo/go-githubactions to v1.2.0
• fix(deps): update module github.com/sethvargo/go-githubactions to v1.2.0
• fix(deps): update module github.com/shopspring/decimal to v1.4.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module github.com/synapsecns/sanguine/ethergo to v0.10.0
• fix(deps): update module github.com/synapsecns/sanguine/ethergo to v0.10.0
• fix(deps): update module github.com/synapsecns/sanguine/ethergo to v0.10.0
• fix(deps): update module github.com/synapsecns/sanguine/ethergo to v0.10.0
• fix(deps): update module github.com/synapsecns/sanguine/ethergo to v0.10.0
• fix(deps): update module github.com/synapsecns/sanguine/ethergo to v0.10.0
• fix(deps): update module github.com/synapsecns/sanguine/ethergo to v0.10.0
• fix(deps): update module github.com/synapsecns/sanguine/ethergo to v0.10.0
• fix(deps): update module github.com/synapsecns/sanguine/ethergo to v0.10.0
• fix(deps): update module github.com/synapsecns/sanguine/ethergo to v0.10.0
• fix(deps): update module github.com/synapsecns/sanguine/services/rfq to v0.21.4
• fix(deps): update module github.com/synapsecns/sanguine/services/scribe to v0.9.2
• fix(deps): update module github.com/uptrace/opentelemetry-go-extra/otelgorm to v0.2.4
• fix(deps): update module github.com/valyala/fasthttp to v1.53.0
• fix(deps): update module github.com/vektra/mockery/v2 to v2.43.1
• fix(deps): update module github.com/vektra/mockery/v2 to v2.43.1
• fix(deps): update module github.com/viant/toolbox to v0.37.0
• fix(deps): update module github.com/yamashou/gqlgenc to v0.23.1
• fix(deps): update module github.com/yamashou/gqlgenc to v0.23.1
• fix(deps): update module github.com/yamashou/gqlgenc to v0.23.1
• fix(deps): update module github.com/zclconf/go-cty to v1.14.4
• fix(deps): update module go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin to v0.51.0
• fix(deps): update module go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc to v0.51.0
• fix(deps): update module go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc to v0.51.0
• fix(deps): update module go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp to v0.51.0
• fix(deps): update module go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp to v0.51.0
• fix(deps): update module go.opentelemetry.io/contrib/propagators/b3 to v1.26.0
• fix(deps): update module golang.org/x/mod to v0.17.0
• fix(deps): update module golang.org/x/mod to v0.17.0
• fix(deps): update module golang.org/x/mod to v0.17.0
• fix(deps): update module golang.org/x/mod to v0.17.0
• fix(deps): update module golang.org/x/oauth2 to v0.20.0
• fix(deps): update module golang.org/x/oauth2 to v0.20.0
• fix(deps): update module golang.org/x/oauth2 to v0.20.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/sync to v0.7.0
• fix(deps): update module golang.org/x/time to v0.5.0
• fix(deps): update module golang.org/x/time to v0.5.0
• fix(deps): update module golang.org/x/tools to v0.21.0
• fix(deps): update module golang.org/x/tools to v0.21.0
• fix(deps): update module golang.org/x/tools to v0.21.0
• fix(deps): update module google.golang.org/api to v0.181.0
• fix(deps): update module google.golang.org/grpc to v1.64.0
• fix(deps): update module google.golang.org/grpc to v1.64.0
• fix(deps): update module google.golang.org/grpc to v1.64.0
• fix(deps): update module google.golang.org/grpc to v1.64.0
• fix(deps): update module google.golang.org/grpc to v1.64.0
• fix(deps): update module google.golang.org/protobuf to v1.34.1
• fix(deps): update module google.golang.org/protobuf to v1.34.1
• fix(deps): update module gorm.io/driver/clickhouse to v0.6.0
• fix(deps): update module k8s.io/apimachinery to v0.30.1
• fix(deps): update module k8s.io/apimachinery to v0.30.1
• fix(deps): update module k8s.io/apimachinery to v0.30.1
• fix(deps): update module k8s.io/apimachinery to v0.30.1
• fix(deps): update module k8s.io/apimachinery to v0.30.1
• fix(deps): update opentelemetry-go monorepo (go.opentelemetry.io/otel, go.opentelemetry.io/otel/exporters/otlp/otlptrace, go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc, go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp, go.opentelemetry.io/otel/exporters/prometheus, go.opentelemetry.io/otel/metric, go.opentelemetry.io/otel/sdk, go.opentelemetry.io/otel/sdk/metric, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/metric, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/metric, go.opentelemetry.io/otel/sdk, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/metric)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/metric, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/metric, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/metric, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/trace)
• fix(deps): update opentelemetry-go monorepo to v1.26.0 (go.opentelemetry.io/otel, go.opentelemetry.io/otel/trace)
• fix(deps): update react monorepo to v18.3.1 (react, react-dom)
• fix(deps): update testing-library monorepo (@testing-library/jest-dom, @testing-library/react)
• chore(deps): update actions/cache action to v4
• chore(deps): update actions/github-script action to v7
• chore(deps): update actions/github-script action to v7
• chore(deps): update actions/github-script action to v7
• chore(deps): update actions/labeler action to v5
• chore(deps): update actions/setup-go action to v5
• chore(deps): update actions/setup-node action to v4
• chore(deps): update actions/setup-python action to v5
• chore(deps): update actions/stale action to v9
• chore(deps): update azure/setup-helm action to v4
• chore(deps): update crazy-max/ghaction-import-gpg action to v6
• chore(deps): update debian docker tag to v12
• chore(deps): update dependency @graphql-codegen/typescript-rtk-query to v3
• chore(deps): update dependency @testing-library/react to v15
• chore(deps): update dependency @typechain/ethers-v5 to v11
• chore(deps): update dependency @types/jest to v29
• chore(deps): update dependency babel-loader to v9
• chore(deps): update dependency chai to v5
• chore(deps): update dependency eslint to v9
• chore(deps): update dependency eslint to v9
• chore(deps): update dependency eslint-config-prettier to v9
• chore(deps): update dependency eslint-config-standard to v17
• chore(deps): update dependency eslint-plugin-jsdoc to v48
• chore(deps): update dependency eslint-plugin-prettier to v5
• chore(deps): update dependency eslint-plugin-promise to v6
• chore(deps): update dependency eslint-plugin-unicorn to v53
• chore(deps): update dependency husky to v9
• chore(deps): update dependency image-minimizer-webpack-plugin to v4
• chore(deps): update dependency imagemin to v9
• chore(deps): update dependency imagemin-svgo to v11
• chore(deps): update dependency jest to v29
• chore(deps): update dependency lerna to v8
• chore(deps): update dependency lint-staged to v15
• chore(deps): update dependency mkdirp to v3
• chore(deps): update dependency mocha to v10 (mocha, @types/mocha)
• chore(deps): update dependency node to v20 (node, @types/node)
• chore(deps): update dependency node to v20
• chore(deps): update dependency node-fetch to v3
• chore(deps): update dependency prettier to v3
• chore(deps): update dependency prettier to v3
• chore(deps): update dependency rimraf to v5
• chore(deps): update dependency rollup to v4
• chore(deps): update dependency size-limit to v11
• chore(deps): update dependency solhint to v5
• chore(deps): update dependency solhint to v5
• <!-- unli

Historically, we've had a lot of issues with missing events from blockchains we operate on. This has resulted in missing transactions and poor/unreliable analytics. Additionally, every service we operate has to tail head in some capacity (for example: #102 is an external monitoring service).

We can fix this through a generic event sourcing microservice. All off-chain agents + services can run an instance of this service (embedded or separately) and query it rather than querying the chain directly. Notably, this service can and will be eth specific so the domain abstractions used in core need not apply here.

The service is completely separate from core (new folder in sanguine) and has a dedicated config (see: https://github.com/synapsecns/sanguine/tree/master/core/config)

The config might look something like the following:

chains:
   - id: 1 # chain id 
     url: "http://127.0.0.1:8545" # rpc url
     confirmation_threshold: 15 # how many blocks to wait before indexing events (so we can account for re-orgs)
   - id: 137
     url: "http://127.0.0.1:8546"

contracts:
   - address: "0x0c4229E35D61d51559Bc450c17337E623179f50b"
    chain_id: 1
    start_block: 2
   - address: 0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48
    chain_id: 137
    start_block: 4

The only job of this service is to allow storing and querying of the results of eth_getLogs for these contracts. Implementation wise, here's what that looks like:

• an indexer: starts eth_getLogs at a given block and runs it continuously until head and then keeps listening to chain tip. The service will very closely resemble: https://github.com/synapsecns/synapse-node/blob/master/pkg/evm/watcher/contract_watcher.go#L139 Crucially, failure of any kind will not result in continuing to index future blocks outside of the failed range. The failed range will be retried until successful, even across restarts. This means that the latest height has to be persisted.

Events are then idempotently inserted into a database in two parts (and seperate tables)

types.Log: all the log data in individual tables so we can query here
types.Receipt: the receipt (which you can fetch using eth_getTransactionByReceipt) for the txhash in the log above. Logs in this type (see link) can be fetched using a foreign key constraint. Any logs for contracts we're not watching should be inserted into the logs table as well w/ a foreign key constraint to txhash.

• a query interface: Once the database is done, we want to be able to query over (rest, grpc, etc: up to you) for the following:
  • events for a contract in a given block/time range (GetBlockByTime implemented here: https://github.com/synapsecns/synapse-node/blob/master/pkg/evm/block.go#L21)

That's it. This allows our other services to event source in indexers without worrying about the reliability of the chain and allows individual indexers to break without needing to implement backfilling

Currently, Foundry tests are unstructured, reuse repeated code and don't have a single test function naming convention.

This should be fixed sooner than later. Ideally, at a point where there are very few solidity PRs into master.

After many attempts (#139), I've spent too much time trying to get github package registry auth for npm to work. This should be done at some point in the future.

Run make chart-test on each new image build and test the helm chart against that new iamage

XAppConfig / ReplicaManager should work together to enroll / un-enroll replicas from active / failed / archived replicas.

SystemRouter contract features a method allowing to do a few system calls to different system contracts with different data at the same time, whether on local or remote chain:

In reality, a lot of system calls are going to be either:

• Call a few contracts with the same calldata (e.g. add a new Notary on these contracts)
• Call a single contract a few times with different calldata (e.g. add a bunch of "remote Notaries" to Destination contract)

Thus, to avoid setting up an extra array every time, and to make the code cleaner, these two wrappers are required.

This is probably best done after testing suite from #57 is implemented.

https://github.com/seatgeek/docker-mirror

Message recipients are supposed to check if the snapshot root was submitted at least "recipient optimistic period" seconds ago (Destination-enforced optimistic period specified in the message may differ).

For this reason it makes sense to pass block.timestamp - rootSubmittedAt instead of rootSubmittedAt to all the message recipients.

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: .github/renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: Invalid configuration option: forkProcessing

ReplicaManager needs general coverage for proving and processing messages.

Overview

With the near completion of Scribe (#114), we're ready to start indexing events for our analytics api. The current state of the analytics api is quite convoluted. analytics.synapseprotocol.com is currently broken on several chains and missing lots of data. You can see that code here along w/ the explorer code here.

A second iteration of analytics, comprised of synapse-indexer and analytics-api requires too much complexity/is too stateful to deploy (which was part of the motivation for #114 along with issues like #153 popping up all over the place rather than in one place where they can be fixed all at once).

The finished product will be a graphql api that looks like this over go, but the first step is to replicate the indexer.

Let's walk through a few real bridging transactions and how they should be indexed. Since this is you're first contribution, I'll run through some steps to get started further below:

The Indexing Process

Here we take an example from the live bridge and walk through the indexing process. Your indexer will take a yaml config file that should look something like the following. I only define two chains since those are the two used for the example. These config values will make sense as I go through the example

The Config:

chains:
   - id: 1 # chain id 
     url: "http://127.0.0.1:8545" # rpc url
     contracts: 
       # this is a list since in some cases we have multiple versions of the same contract. You'll need to define these as an enum somewhere
       - type: bridge
       # this will be sourced by the person writing the config for abi.receipt. blockNumber, e.g. this is from https://github.com/synapsecns/synapse-contracts/blob/master/deployments/mainnet/SynapseBridge.json
         start_block:  13033669
       # some contracts (really only bridgeconfig/poolconfig: an older iteration of bridge config) are only on ethereum
        - type:  "bridgeconfig"
          address: "0x5217c83ca75559B1f8a8803824E5b7ac233A12a1"
          # see: https://github.com/synapsecns/synapse-contracts/blob/master/deployments/mainnet/BridgeConfigV3.json#L1100
          start_block: 14259367
       # an older verison of bridge config
       - type: "bridgeconfig"
         address: "0xAE908bb4905bcA9BdE0656CC869d0F23e77875E7"
         start_block: 13949327
        # when we start using v3. 
         end_block: 14259367
   - id: 42161
     url: "http://127.0.0.1:8546"
     contracts:
        - type: bridge
           # see: [0x6F4e8eBa4D337f874Ab57478AcC2Cb5BACdc19c9](https://github.com/synapsecns/synapse-contracts/blob/master/deployments/arbitrum/SynapseBridge.json#L2)
          address: "0x6F4e8eBa4D337f874Ab57478AcC2Cb5BACdc19c9"
          # see: https://github.com/synapsecns/synapse-contracts/blob/master/deployments/arbitrum/SynapseBridge.json#L1462
          start_block:  657404
       - type: pool
         # https://github.com/synapsecns/synapse-contracts/blob/master/deployments/arbitrum/nUSDPoolV3.json#L2
         address: 0x9Dd329F5411466d9e0C488fF72519CA9fEf0cb40
        # see: https://arbiscan.io/tx/0x500afe6cf8e927ccad7a8a2e01f7d3bfc2fa9ef3af6a55f841d71bd5b62c84d3, older deploys don't have the receipt so we pull it from the top right corner of the contract address in the explorer, arbiscan in this case
         start_block: 5152261
# url of the scribe service, should probably also be embedable 
scribe: http://scribe:1231

The Example:

Let's look at a live example. Here is a transaction which occured on arbitrum. As we can see from the data, the user is bridging to ethereum.

Note: I've chosen the most complicated bridge type here, other types such as mint do not require bridgeconfig, etc

Bridge Parsing

This transaction is going to trigger a few events that will get populated in the contracts we watch on scribe. The first is the bridge event. This particular event triggered on the bridge is TokenRedeemAndRemove We can see it contains the following items:

We now know that on ethereum, 0x59719d517208b306eA9c7a9FD90D6215163323Ee will receive a minimum of 5330566953 nusd (which will then be swapped for tokenIndexTo: 0 which is usdc) before 1662394851 (Monday, September 5, 2022 4:20:51 PM) on ethereum (chain id 1). If the swap can't be completed, the user will receive nusd on the other end which they can then trade for any token in the pool.

We can also look at the raw data (for most transactions, txes triggered this can't be used for indexing because other contracts can call ours, but it is helpful for understanding the flow) and see the method called:

Pool Parsing:

Since it's a swapAndRedeemAndRemove, we can see exactly what methods are called for the contract to execute in L2BridgeZap:

In addition to being pased in the input, these are also passed as a log:
that can be parsed by the abi we generated and inserted

We also have another event to index here: a swap.

We can see the raw swap data here:

. If we look at the swap:

We can see exactly what happened here. We're going to want to index this so we can calculate pool volume.

The Receiving Chain

This transaction triggered a bridge that was then received at the other end. Let's take a look at the transaction here. We can see here that withdrawAndRemove was called.

One of the challenges of parsing transactions on the other end is the pool is never emitted directly:

We can see from the contract that in cases where the swap is not successful, we simply transfer the token (nusd in this case) to the user. Since there's nothing more to index here, we can finish up after just indexing the receiving TokenWithdrawAndRemove without any pool data.

Bridge Config

In cases where expectedOutput >= swapMinAmount (most cases), we'll also receive an event from a pool. But how do the validators know which pool to pass here? And why is the token different than the address on the origin chain)

This is where bridgeconfig comes in. Two calls are made to BridgeConfigV3, in your case, these should be archive calls at the block_number of the transaction. First we call getTokenID(0x2913E812Cf0dcCA30FB28E6Cac3d2DCFF4497688, 42161). This is the token address in the call above and the chain id from above. This should be called on 0x5217c83ca75559B1f8a8803824E5b7ac233A12a1 rather than the other bridge config since the the current block number is greater than start block. If this tx were between 13949327 and 14259367 we'd use 0xAE908bb4905bcA9BdE0656CC869d0F23e77875E7 instead.

We can try this out on etherscan here. This won't be an archive call, but it's good enough for us to see what happened, since bridge config hasn't changed in the meantime. We can see the tokenID is nusd:

Now, let's figure out the token address we want to use on chainID 1 using the token id we just got:

This data corresponds to this struct, in order:

We can see here the the token address 0x1b84765de8b7566e4ceaf4d0fd3c5af52d3dde4f matches nusd on ethreum. Since this transaction is a swap, we want to query the pool config as well to see what pool we've swapped (or attempted to swap on). Let's call getPoolconfig with the token address we received above:

. We can see the first argument is nusd and the second is a SwapFlashLoan contract. This is where the swap from nusd to usdc happened in our contract.

If we go back to the event logs for the tx we're inspecting here we can see an event emitted by this contract:

. Our topic map will tell is this is RemoveLiquidityOne. We'll need to store this for swap analytics. We can also see the amount of tokens the user actually received this way and use that for volume calculations.

We can also see from the logs a TokenWithdrawAndRemove events:

. We'll want to index this.

One final thing to note. You can see the last indexed topic here is bytes32 kappa. Kappa is simply the keccac256(origin_tx_hash).

So in this transaction, we should've indexed the following:

• TokenRedeemAndRemove: on arbitrum
• TokenSwap: on arbitrum
• [TokenWithdrawAndRemove](https://github.com/synapsecns/synapse-contracts/blob/9e390f7c826ab09c48c3c8fe3d040226ee8b3aa0/contracts/bridge/SynapseBridge.sol#L108): on ethereum
• RemoveLiquidityOne

From this, we'll be able to compute a few things:

• We can compute the price of usdc and nusd against usdc.
• We can calculate the volume
• We can calculate the fees earned

Steps to building the service

Abigen

First, you're going to create a new service in services/explorer, next you're going to need to generate some contracts. This readme will walk you through the process. (Note: prior to the merge of #166, you could've imported synapse-node and used its contracts. The topics file and bridge folder generally are worth referencing. I'd recommend adding the contracts repo as a submodule in order to abigen against them. I'd also reccomend giving the contracts a versioned name, as it's quite possible we'll have to generate multiple ersions in order to parse events against them. For instance, we've had several iterations of the BridgeConfig so far.

There are a few contracts you'll have to generate abi's for in order to succesfully track events from the bridge:

• SwapFlashLoan: used for swapping
• SynapseBridge: used for bridging
• BridgeConfigV3: used for bridge config

In general, all events from these contracts should be indexed in a standardize way (e.g. store all data in the db as structured data). Many of the bridge events are indexed here so you should straight up be able to copy and paste the code. Ordinarily, copying and pasting code is a big no-no, but in this case since we're deprecating synapse-node it's fine. Crucially, you'll need the topicMap and the standardized parsing

Config

Create a config parser for the config defined above, you should be able to use this file and the corresponding test. You'll use this to decide which contracts to index/their types.

Scribe Client

Create a graphql client against scribe, @CryptoMaxPlanck should be able to walk you through this, but your goal is to be able query continiously and index against the JSON. You're best bet here is going to be to use the raw JSON scalar and call UnmarshallJSON on the ethereum types, e.g. for logs this method. These can then be used to parse out events, like so

DB

I'd use DBService here for reference. You're going to want to store all these events in a format that they can easily be aggregated in real time. You'll need a tiny bit of additional data, namely the prices. I'd probably handle this with a sql join.

GraphQL server:

You should be able to staright up copy this schema. This doesn't include analytics methods, but should be a good start to the sever.

The server should be run independently of the indexer.

Should also check if solidity was changed

As outlined in 97a6c84, System Call is likely to cover all the usecases for a System Message. Thus, the Flag in the SystemMessage can be deleted and the whole library should be renamed/transformed into SystemCall.

Also, the library needs a proper explanation as to how the system calls are working.

A System Call works like this:

1. Recipient system contract exposes one or more functions that are supposed to be called only by other system contracts (whether in a on-chain or cross-chain way).
2. These functions need to have following format:

// Some arbitrary data, last three arguments are always the same
// Function is protected by onlySystemRouter modifier, so that only local chain system router can call it
function someFunction(<...>, uint32 origin, SystemEntity caller, uint256 rootSubmittedAt) external onlySystemRouter;

3. To perform a system call, a system contract triggers:

bytes memory payload = abi.encodeWithSelector(someFunction.selector, <...>);
systemRouter.systemCall(uint32 destination, uint32 optimisticSeconds, SystemEntity recipient, payload);

4. System Router routes this call to another system contract for a on-chain call, or sends a message to SystemRouter on destination chain to do the routing in case of a cross-chain call.
5. System Router on origin chain adds following information to the payload, effectively adding two security parameters (origin domain and which contract called the router) to the payload

payload = abi.encodePacked(payload, abi.encode(origin, caller));

6. System Router on destination chain adds time when root was submitted to the payload (block.timestamp is used for on-chain calls) and calls recipient:

payload = abi.encodePacked(payload, abi.encode(rootSubmittedAt));
recipient.call(payload);
// this calls recipient.someFunction(<...>, origin, caller, rootSubmittedAt);

7. Every System Router keeps track of all on-chain system contracts, allowing system contracts to use enum SystemEntity instead of actual contract address. address <> SystemEntity matching is done within the System Router.

enum SystemEntity {
    Origin,
    Destination,
    ...
}

8. Recipient receives origin, caller, rootSubmittedAt parameters verified by system routers. Using them, recipient can restrict the prerequisites for calling the function to any combination of the following:

• Calls from only given domain, like "only on-chain local", "only cross-chain call", "only call from Synapse Chain", etc.
• Calls from only a subset of system contracts, like "only from Origin", or "only from BodngManager", or "from every system contract but Destiantion", etc.
• Calls with given optimistic period like "at least 1 day old".

  On-chain calls will always use block.timestamp, so this restriction will disable them.

One of the interesting things an rpc proxy allows us to do is to run additional sanity checks on bridge transactions. While these checks will differ for v2, we'd like to implement something in omnirpc for the v1 bridge that can identify and stop non-key compromise based attacks before they happen. It works by simulating the state of the bridge on the destination chain and making sure no more tokens are issued than were burned on the origin chain. The diagram above briefly describes what I go into more detail with here:

1. What we want here is a plug-and-playable way to enable/disable checks in code based on rpc method. The rpc request object will contain the method name so if it's eth_sendRawTransaction we should trigger this check workflow. One nice to have here: if we could figure out a generalizable way to implement these checks, it might be helpful going forward. For interesting for eth_getReceipt we may want to do (v,r,s) verification for chain id.
2. Decode the transaction and check the address against a list of nodegroup addressess. If the from address is in the list (defined in config.yaml), proceed with the bridge check.
3. Fork the destination chain. There are a few ways I can think of to do this with varying levels of complexity:

• Use docker to fork the network using ganache w/ confirmability of 1 omnirpc/confirmations/1/rpc/[chainid]. This can be a bit challenging in kubernetes
• Do the same thing as above, but w/ anvil
• Embed anvil into omnirpc. this should be possible with rustgo but don't know the complexity of this or the compile time consequences.
• Investigate go native solutions: ethereum/go-ethereum#21224 and ledgerwatch/erigon#1613, no progress seems to have been made on either of these
• Rust/js microservice for quickly spinning up/down forks for this purpose.

4. Fetch the token in bridgeconfig, see here for details.
5. Using the decoded transaction, make sure the total tokens given to the user at the end of the tx (or liquidity pool in the case of a swap) are less than the tokens in. and that a redeem (burn) cannot result in a withdraw. If they are not, do not send the tx and send an sos message to the team (probably through webhook, this will allow us to setup phone calls, etc or slack. If the tx is fine, do nothing.

Challenges

1. Hot forking is fairly novel and may or may not require archive nodes for rpc endpoints. We should investigate this and see what's required in terms of infrastructure to get this done.

Blockers:

#225, potentially infrastructure helm charts

Right now, abi is automatically generated using go:generate from the abigen module. This requires manually running go generate ./... periodically.

The problem with this is it breaks some of the implied functionality of a monorepo: namely that go tests don't fail when solidity is updated with breaking changes. There are three potential solutions here:

1. Always run go:generate ./... in ci (and probably only in the core/contracts directory). This introduces a few problems. First, the docker pull process & abi generation steps are somewhat time consuming. This also runs other go generate steps with the potential to flake (some may be networked). Additionally, this has the disadvantage of not making users update the repo with new auto-generated abi.
2. Use a diff checker on packages/contracts and and only run generate` if needed. Additionally, this has the disadvantage of not making users update the repo with new auto-generated abi.
3. Force users to run go generate ./.... use a diff check and fail if results are different. This has the disadvantage of making changes to the packages/contracts repo not usable without go.
4. Run go generate ./... as part of the typechain generation process?

There's likely a better, more generic solution here, but this is what I've got so far.

Currently, the updates of Home merkle roots on remote Replica contracts is done in the synchronous way, i.e. all signed updates need to be applied to every Replica. This does not scale well.

Need to implement the updates in asynchronous way, i.e. where updater signs a bunch of updates of Home merkle root, within the signing rules (i.e. without getting slashed). Any of the updates can be applied on any of the Relplica, allowing to skip the updates that doesn't include any messages to given Replica.

Currently, the makefile does not download the latest version or pin a version which can result in versioning differences between local and ci

Apply latest updates from the original repo:

https://github.com/summa-tx/memview-sol

If you look at #130 you can see that despite changes to files in the .github folder, no new labels are added.

The issue likely resides in .github/labeler.yml

Add a way for Home or ReplicaManagercontracts to send messages to another Home or ReplicaManager contracts on other chains. This could be used for variety of reasons:

• Cross-chain Home setups: adding new Updater, WatchTower, etc.
• Sending information about collected tips by the remote chain agents.

As the security is crucial here, these messages could be only sent with a long optimistic period (to prevent fraud updates screwing the protocol itself).

Current set of tests is good, but lacking a few things:

• Doesn't use fuzzed inputs, meaning something potentially might have been missed.
• Functions that are using assembly to write in memory should be tested for dirty writes (as they update the free memory pointer).

In PRs such #576 , PR's that might have some go changes, and revert them do not remove the go generate tag.

Deprecate typecast in ethergo

As part of efforts to separate indexing logic from agents, we've built (scribe #114) and (omnirpc #135). The final step here to making the off-chain agents invulnerable to chain-watching induced errors is configuring the indexer at a domain level. Currently, for notary (#77) we start an indexer to sync messages. This performs getLogs on the contracts and ignores every event from origin except for dispatch. Leaving starting this loop up to the notary will create the need to seperate dbs at an agent level despite these agents performing the same requests and makes testing harder.

Additionally, storing both agent level and indexer level data in the MessageDB interface creates the potential for logic errors in future work. Instead, we can do the following.

func NewIndexer(config DomainConfig, db ){}

DomainConfig struct {
    // Standard domain config (e.g. rpc, chain id, etc)
   DestinationAddress common.Address
   OriginAddress common.Address
   ConfirmationThreshold uint
   StartHeight uint32
   AttestationCollector common.Address
}

The indexer can then sync all events whichever of these contracts is not blank common.Adress{} != domainConfig.x and any agent can pull from that. Crucially, the IndexerDB exposes two types IndexWriter which is only used by the indexer and IndexReader which all agents use.

Each Notary, Broadcaster, Guard etc each have their own seperate db stores with the prefix based off the id speficied in the config.

#316 implements the MVP for Bonding Managers, contracts that are responsible for syncing the information about the off-chain actors, who have committed their staking bond on Synapse Chain.

Why do we need that?

The actions tied to bonding, and especially unbonding, require the confirmation that the new piece of information has been successfully passed to all relevant domains. For example, if an off-chain agent wants to unstake their bond, we need to do both:

• Send information that the agent is no longer active to all chains, where this actor signature is valid.
• Receive confirmation from all such chains that the actor is removed.

Only after both are completed, the staking bond could be unlocked.

Information to sync

1. A new Notary/Guard staked a bond. Originates on Synapse Chain, need to inform all other chains.
2. An active Notary/Guard initiates the unbonding. Originates on Synapse Chain, need to inform all other chains.
3. An active Notary/Guard was slashed. Originates on any chain, should be forwarded to Synapse Chain first. Then to all the remaining chains.

The incentives should be structured in a way, where the slashed actor will have been reported already (assuming rational actors) by the time the information about slashing is finally forwarded.

Considerations

1. Implementation should resemble a ping-pong scheme:
   a. First, BondingPrimary on Synapse Chain forwards new information to all the BondingSecondary contracts (pings).
   b. BondingSecondary responds with a pong upon handling the information.
   c. BondingPrimary should ensure that number of received pongs is equal to number of sent pings.
2. What happens if one of the pongs gets delayed? Due to contract error, or simply due to the destination domain being halted?
3. Do we need some kind of timeout mechanism to prevent deadlocks?
4. Do we need to send a new wave of pings once all the pongs are received? So that every BondingSecondary knows that the new piece of information is synced everywhere.
5. Do we need to bulk the synchronization process? For example, send information about all updated agents statuses at most once per day (or once there's at least N status changes to prevent out-of-gas DoS attacks).

TODOs after #755 is merged:

• Use sendSystemMessage() instead of sendBaseMessage()
• Use receiveSystemMessage() instead of handle/receiveBaseMessage
• Remove the concept of local "system call": local contracts are supposed to know each other address
• Remove the concept of remote multi-call: could be done via a vanilla remote call that triggers a multicall when received (local contracts are supposed to know each other address).