Webhook invocation

What is a webhook invocation?

A webhook invocation is the URL to which an HTTP request should be sent if this webhook wallet participates in a blockchain transaction (E.g fund transfer)

This architectural decision was made to provide the system with flexibility, a guarantee of webhook delivery, and even to load distribution to instances

Single Responsibility principle states that every object should have one responsibility and this responsibility must be completely encapsulated. Following this principle, it was divided into several processors, each one responsible for its specific task. Processors in this contest are jobs that wake up at a certain time and perform a task

• Blockchain processor
• Webhook processor
• Failover processor

Blockchain processor (Details can be found in the Blockchain tracking section)

• After the blockchain processor has processed the block and found a transaction in it that involves a wallet in the system, it adds to a special queue for webhooks in Redis, which will be further processed by special jobs.
• The structure of storing webhooks in Redis is as follows: Since Redis stores data in the form of "key and value" and provides a queue system, we utilize these capabilities

1. The first queue stores information about the wallet. The key will be [queue: webhook], and the value will be a json object that stores the wallet address and blockchain type. It would be correct to mention that the values for the key will be unique, since this will allow webhooks to be performed for a specific wallet consistently.
2. And the second queue stores transactions in the wallet, which needs to send a webhook. I.e. the key will be [address: type: webhook], and the value is the transaction hash.

Webhook processor

A webhook processor makes only one webhook invocation for one wallet at a time. This job wakes up at a certain time with fixed delay after each execution. This allows for the avoidance of spam and overloading of remote servers. In addition, this starts several instances of Open State to make webhook invocation faster. Logically, the architecture of a webhook invocation can be divided into 2 main parts:

• Webhook queue
• Webhook invocation

The webhook invocation algorithm is presented on the following diagram:

Webhook queue

All detected transactions in a blockchain processor are added to a webhook queue. A webhook queue consists of 2 parts:

• A Wallet execution queue contains a set of objects with information about a wallet attempts counter and execution timestamp. There is only one such object for a wallet in queue. A task in a queue can be re-scheduled (i.e. changed execution timestamp) or removed if webhooks are invocated for all transactions of a wallet. The Queue is implemented as ZSet in Redis, and an added timestamp is used as a score.
• A List of transactions is created for each wallet in queue. It is implemented as a list in Redis. All transactions from a blockchain processor are added to the end of the list. Processed transactions are removed from list, and processing starts from the first record in the list.

Webhook invocation

The Webhook invocation processor takes the task that is closest to the current timestamp from the webhooks queue and tries to execute it. Every task is executed. It retries task execution if the webhook invocation fails. The Webhook invocation processor can be configured with the following properties:

• Webhook invocation processor execution delay - amount of milliseconds between each processor invocation
• Lock time out - amount of milliseconds to lock an object in wallet queue
• Short time period attempts count - max count of attempts for webhook invocation retries in a short time period. The delay is equal to the number of seconds taken from a Fibonachi row for each attempt. Default value is 10.
• Long time period attempts count - max count of attempts for webhook invocation retries in a long time period. Each attempt would run once per day during a specified count of days. Default value is 17.

The Webhook invocation executes as follows:

• Get the next task from the webhook queue and lock it. If the lock fails, this means that task was already executed by another process, and WIP will try to get the next task from the queue
• Get the first record from the transactions list associated with the current wallet and invoke an HTTP request
• If the HTTP response is successful (2xx HTTP response code), remove the transaction from the list. If the list becomes empty, remove the wallet from the webhook queue and delete the associated list of transactions.
• If the webhook fails, it would be re-scheduled to execute later after a delay. The delay is calculated as follows: fb(n) * second. After reaching a short time period attempts count, the webhook would be re-scheduled to run once per day. After reaching a long time period attempts count, the webhook invocation would be blocked for the wallet
• If all attempts fail, the webhook task would be removed from the queue and a list of transactions for the wallet would be moved to a special collection in MongoDB. The wallet's transactions would not be processed until a webhook url or a webhook status is changed
• Save webhook invocation information to MongoDB (number of attempts, last received HTTP response code, message from remote server, and an invocation timestamp)
• Remove lock for wallet

Webhook invocation payload

A Webhook payload would be sent via HTTP POST request and include the following data:

• blockchain - Name of blockchain
• walletAddress - Address of wallet that received transaction
• transaction - information about transaction

A Transaction object would include the following data:

• hah - transaction hash
• blockHash - hash of block
• blockHeight - height of block
• amount - amount of transaction
• date - date of transaction
• to - address of receiver wallet
• from - address of sender wallet

Example of payload:

{ "blockchain": "Ethereum", "walletAddress": "0x9D86b1B2554ec410ecCFfBf111A6994910111340", "transaction": { "hash": "0x9D86b1B2554ec410ecCFfBf111A6994910111340", "blockHash": "0x3aabcf5ca3ec5e459d9aa1314d67c6595beac810", "blockHeight": 124414145, "amount": 0.314141, "date": "2020-01-12T00:22:43.511Z", "to": "0x9D86b1B2554ec410ecCFfBf111A6994910111340", "from": "0x3aabcf5ca3ec5e459d9aa1314d67c6595beac810" } }

Failover processor

• This processor is needed in order to periodically check that the Webhook processor has not exceeded the processing time.
• This job works in the following ways: Takes an item from the [lock: webhook] queue in Redis, which will already be sorted by date added in descending order. Next, it checks that the webhook execution time for this wallet has not been exceeded. If so, it will count that the Webhook processor has not processed this webhook and queues it for processing [queue: webhook] This processor guarantees delivery if the Webhook processor does not finish processing the webhook.