Skip to content

【BUG】Connect runtime out of memory  #542

@VictoryAnn

Description

@VictoryAnn

BUG REPORT

  1. Please describe the issue you observed:
  • What did you do (The steps to reproduce)?

  • What is expected to see?
    task status is RUNNING

  • What did you see instead?
    task status is FAILED, and reason is OutOfMemory.

  1. Please tell us about your environment:

  2. Other information (e.g. detailed explanation, logs, related issues, suggestions on how to fix, etc):
    a. run jmap -heap 16 found memory top 1 is org.apache.rocketmq.connect.runtime.stats.ConnectStatsService.
    image
    b. use arthas watch sourceTaskTimesList and sinkTaskTimesList
    image
    image

SOLUTION:

    private void sampling() {
        this.lockSampling.lock();
        try {
            this.sourceTaskTimesList.add(new CallSnapshot(System.currentTimeMillis(), sourceTaskTimesTotal()));
            if (this.sourceTaskTimesList.size() > (MAX_RECORDS_OF_SAMPLING + 1)) {
                this.sourceTaskTimesList.removeFirst();
            }
            // !!! sourceTaskTimesList is wrong !!!
           // !!! should be sinkTaskTimesList !!!
            this.sinkTaskTimesList.add(new CallSnapshot(System.currentTimeMillis(), sinkTaskTimesTotal()));
            if (this.sourceTaskTimesList.size() > (MAX_RECORDS_OF_SAMPLING + 1)) {
                this.sourceTaskTimesList.removeFirst();
            }
        } finally {
            this.lockSampling.unlock();
        }
    }

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions