[MEX-923] Performance optimisations

[mad2sm0key]

Reasoning

• degraded performance when interacting with the persistence layer directly; cache warmers OOM crashes

Proposed Changes

• Persistence updates :

  • add snappy compressor to mongoDB requests
  • decorator for collecting mongoDB performance metrics + add decorator to all persistence services
  • update filteredTokensAggregationPipeline to only return token identifiers (full tokens are to be fetched from cache)
  • add lean option for mongoDB find queries
  • add ordered: false for mongoDB bulkWrite queries
  • execute sequencialy + add delays on pair and token analytics refresh execution
  • execute analytics refresh cron directly (the task does not benefit from orchestration)
  • remove unused methods from pair and token persistence services
  • use redis hashes to cache pairs and tokens (tokens.all, pairs.all)

• Caching updates :

  • add hash operations to cache service
  • hash getMany util function (get-or-set mechanism)
  • pair caching methods - set/get/refresh
  • add getTokensByIdentifier method w/ caching
  • integrate caching updates in state changes consumer
  • use data from persistence layer in compute heavy cache warmers

How to test

• N/A

[codecov]

Codecov Report

❌ Patch coverage is 21.23552% with 204 lines in your changes missing coverage. Please review.
✅ Project coverage is 51.27%. Comparing base (70ed122) to head (36b3556).
⚠️ Report is 27 commits behind head on feat/event-driven-backend.

Files with missing lines	Patch %	Lines
...s/persistence/services/pair.persistence.service.ts	9.45%	65 Missing and 2 partials ⚠️
.../persistence/services/token.persistence.service.ts	10.60%	57 Missing and 2 partials ⚠️
src/services/caching/cache.service.ts	8.00%	20 Missing and 3 partials ⚠️
src/utils/get.many.utils.ts	5.00%	16 Missing and 3 partials ⚠️
src/modules/persistence/utils/bulk.write.utils.ts	7.14%	10 Missing and 3 partials ⚠️
...elpers/decorators/persistence.metrics.decorator.ts	68.75%	9 Missing and 1 partial ⚠️
.../modules/router/services/router.compute.service.ts	12.50%	7 Missing ⚠️
src/utils/metrics.collector.ts	33.33%	4 Missing ⚠️
src/modules/tokens/services/token.service.ts	66.66%	2 Missing ⚠️

Additional details and impacted files

@@                      Coverage Diff                      @@
##           feat/event-driven-backend    #1708      +/-   ##
=============================================================
- Coverage                      53.00%   51.27%   -1.73%     
=============================================================
  Files                            302      306       +4     
  Lines                          14790    15465     +675     
  Branches                        1930     2076     +146     
=============================================================
+ Hits                            7839     7930      +91     
- Misses                          6945     7139     +194     
- Partials                           6      396     +390     

Flag	Coverage Δ
unittests	51.27% <21.23%> (-1.73%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.