-
Notifications
You must be signed in to change notification settings - Fork 14
Open
Description
The author's work is great!There's a question i want to ask:
context:
`
Figure 3(a) shows the number of instances activated for serving ResNet-20 with and without OTP batching. By aggregating the user requests into batches (we set 𝑏𝑎𝑡𝑐ℎ𝑠𝑖𝑧𝑒 = 4), the total number of function invocations declines by 72%. As an instance may be reused by a later-arriving request,the total number oflaunched instances under batching also declines by 35%.
Does w/o batching mean there is no batch processing? If the orange dotted line represents multiple concurrent request processing for a single instance, shouldn't the number of instances represented by the orange dotted line be less than the number of instances represented by the blue dotted line.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
