chore: Streaming images for pdf transcript #38263

KevLehman · 2026-01-19T18:45:33Z

Proposed changes (including videos or screenshots)

Issue(s)

Steps to test or reproduce

Further comments

Memory Measurements

Scenario	Total RSS Baseline	Total RSS During Job	Δ	Key Process Deltas
Non-streaming	5969.8 MiB	7378.5 MiB	+1408.7 MiB	`programs/server`: +822.7 MiB `apps/omnichannel-transcript`: +980.2 MiB
Streaming	5651.2 MiB	6291.7 MiB	+640.5 MiB	`programs/server`: +426.8 MiB `apps/omnichannel-transcript`: +152.6 MiB

Conclusions

Streaming attachments/PDF data cuts the transcript service spike by ~85% and overall job RSS by ~770 MiB compared to the buffered implementation.

Extra

Measurement was made with a custom script that measured per process cpu/ram usages for every service.
The room in question (used for the tests) had 10 attachments, varying size, but summing up 200MB (in processable files, aka, images) and some messages.

dionisio-bot · 2026-01-19T18:45:38Z

Looks like this PR is not ready to merge, because of the following issues:

This PR is missing the 'stat: QA assured' label
This PR is missing the required milestone or project

Please fix the issues and try again

If you have any trouble, please check the PR guidelines

changeset-bot · 2026-01-19T18:45:38Z

⚠️ No Changeset found

Latest commit: 9d7c70d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

coderabbitai · 2026-01-19T18:45:41Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch feat/streaming-pdf-images

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-01-19T18:54:34Z

Codecov Report

❌ Patch coverage is 4.00000% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.77%. Comparing base (a28bc73) to head (9d7c70d).
⚠️ Report is 10 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop   #38263      +/-   ##
===========================================
+ Coverage    70.71%   70.77%   +0.06%     
===========================================
  Files         3142     3142              
  Lines       108925   108953      +28     
  Branches     19577    19652      +75     
===========================================
+ Hits         77022    77108      +86     
+ Misses       29902    29848      -54     
+ Partials      2001     1997       -4

Flag	Coverage Δ
unit	`71.91% <5.88%> (+0.08%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2026-01-19T18:59:21Z

📦 Docker Image Size Report

➡️ Changes

Service	Current	Baseline	Change
sum of all images	0B	0B	0B
account-service	0B	0B	0B
authorization-service	0B	0B	0B
ddp-streamer-service	0B	0B	0B
omnichannel-transcript-service	0B	0B	0B
presence-service	0B	0B	0B
queue-worker-service	0B	0B	0B
rocketchat	0B	0B	0B

📊 Historical Trend

---
config:
  theme: "dark"
  xyChart:
    width: 900
    height: 400
---
xychart
  title "Image Size Evolution by Service (Last 30 Days + This PR)"
  x-axis ["11/18 22:53", "11/19 23:02", "11/21 16:49", "11/24 17:34", "11/27 22:32", "11/28 19:05", "12/01 23:01", "12/02 21:57", "12/03 21:00", "12/04 18:17", "12/05 21:56", "12/08 20:15", "12/09 22:17", "12/10 23:26", "12/11 21:56", "12/12 22:45", "12/13 01:34", "12/15 22:31", "12/16 22:18", "12/17 21:04", "12/18 23:12", "12/19 23:27", "12/20 21:03", "12/22 18:54", "12/23 16:16", "12/24 19:38", "12/25 17:51", "12/26 13:18", "12/29 19:01", "12/30 20:52", "01/21 16:47 (PR)"]
  y-axis "Size (GB)" 0 --> 0.5
  line "account-service" [0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.00]
  line "authorization-service" [0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.00]
  line "ddp-streamer-service" [0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.00]
  line "omnichannel-transcript-service" [0.14, 0.14, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.00]
  line "presence-service" [0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.00]
  line "queue-worker-service" [0.14, 0.14, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.00]
  line "rocketchat" [0.35, 0.35, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.34, 0.00]

Statistics (last 30 days):

📊 Average: 1.5GiB
⬇️ Minimum: 1.4GiB
⬆️ Maximum: 1.6GiB
🎯 Current PR: 0B

ℹ️ About this report

This report compares Docker image sizes from this build against the develop baseline.

Tag: pr-38263
Baseline: develop
Timestamp: 2026-01-21 16:47:28 UTC
Historical data points: 30

Updated: Wed, 21 Jan 2026 16:47:28 GMT

d-gubert · 2026-01-19T22:31:42Z

apps/meteor/app/file-upload/server/lib/FileUpload.ts

apps/meteor/server/services/upload/service.ts

d-gubert · 2026-01-20T20:40:39Z

apps/meteor/app/file-upload/server/lib/FileUpload.ts

 			// We can't use _streamToTmpFile at this stage since the file hasn't been validated yet,
 			// and for security reasons we must not write it to disk before validation
 			content = await streamToBuffer(content);
+			fileData.size = fileData.size ?? (content as Buffer).length;


I think we could stream the contents to a temp file here and overwrite content with the file path - then it's gonna follow the new file upload flow

I can change the service to upload to a temp file right before calling this one 👀

stream

1d631b7

Merge branch 'develop' into feat/streaming-pdf-images

e24a39f

KevLehman commented Jan 20, 2026

View reviewed changes

apps/meteor/app/file-upload/server/lib/FileUpload.ts Outdated Show resolved Hide resolved

apps/meteor/server/services/upload/service.ts Outdated Show resolved Hide resolved

Apply suggestions from code review

3bd48d8

d-gubert reviewed Jan 20, 2026

View reviewed changes

stream to file upon receive

9d7c70d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore: Streaming images for pdf transcript #38263

chore: Streaming images for pdf transcript #38263

KevLehman commented Jan 19, 2026 •

edited

Loading

Uh oh!

dionisio-bot bot commented Jan 19, 2026

Uh oh!

changeset-bot bot commented Jan 19, 2026 •

edited

Loading

Uh oh!

coderabbitai bot commented Jan 19, 2026

Review skipped

Uh oh!

codecov bot commented Jan 19, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 19, 2026 •

edited

Loading

Uh oh!

d-gubert commented Jan 19, 2026

Uh oh!

Uh oh!

Uh oh!

d-gubert Jan 20, 2026

Uh oh!

KevLehman Jan 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chore: Streaming images for pdf transcript #38263

Are you sure you want to change the base?

chore: Streaming images for pdf transcript #38263

Conversation

KevLehman commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes (including videos or screenshots)

Issue(s)

Steps to test or reproduce

Further comments

Conclusions

Extra

Uh oh!

dionisio-bot bot commented Jan 19, 2026

Uh oh!

changeset-bot bot commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

coderabbitai bot commented Jan 19, 2026

Review skipped

Uh oh!

codecov bot commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Docker Image Size Report

➡️ Changes

📊 Historical Trend

Uh oh!

d-gubert commented Jan 19, 2026

Uh oh!

Uh oh!

Uh oh!

d-gubert Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

KevLehman Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

KevLehman commented Jan 19, 2026 •

edited

Loading

changeset-bot bot commented Jan 19, 2026 •

edited

Loading

codecov bot commented Jan 19, 2026 •

edited

Loading

github-actions bot commented Jan 19, 2026 •

edited

Loading