Skip to content

A little confusion about one image of the paper #3

@penghuima

Description

@penghuima

The author's work is great!There's a question i want to ask:
context:

cf96628da6f872b67495d6222f7e9d2`
Figure 3(a) shows the number of instances activated for serving ResNet-20 with and without OTP batching. By aggregating the user requests into batches (we set 𝑏𝑎𝑡𝑐ℎ𝑠𝑖𝑧𝑒 = 4), the total number of function invocations declines by 72%. As an instance may be reused by a later-arriving request,the total number oflaunched instances under batching also declines by 35%.

Does w/o batching mean there is no batch processing? If the orange dotted line represents multiple concurrent request processing for a single instance, shouldn't the number of instances represented by the orange dotted line be less than the number of instances represented by the blue dotted line.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions