Shared memory data transfer between Functions host and out-of-proc workers for binary data

[gohar94]

What problem would the feature you're requesting solve? Please describe.

When transferring parameters to/from the out-of-proc workers (input/output bindings), the data flows over gRPC which does not provide high throughput. Since the Functions host and the worker processes are generally on the same VM, it would make more sense to share parameters between the two over shared memory.

Describe the solution you'd like

• Azure Functions Host receives a function invocation request. It goes through the binding process and obtains a Stream to the input being read
• Two of the following options can be taken:
  • The Stream is directly written into a MemoryMappedFile (lower memory footprint, faster, trickier to implement)
  • The Stream is read into a byte[] which is then written into a MemoryMappedFile (slower, easier to implement for a first draft)
• A unique name is generated for the MemoryMappedFile from two of the following options:
  • A GUID is generated
    • Much more generic for all binding types
  • InvokeString of the object or the URI is used
• When sending the InvocationRequest message to the worker, the ParameterBinding has a new TypedData field called SharedMemoryData with the following sub-fields:
  • MemoryMappedFileName
  • Offset (offset within the specified MemoryMappedFile to start reading data from)
  • Count (number of bytes starting from the offset to read)
  • ConversionType (after reading the bytes, how to convert them before passing them to the function. e.g. convert to a string or leave as a byte[] etc.)
• The worker upon receiving the InvocationRequest reads each parameter as it does currently; if it encounters a parameter of with TypedData as SharedMemoryData then it will read the specified MemoryMappedFile from Offset, read Count bytes and finally convert the read bytes into ConversionType before passing it to the function
• The worker will produce an output which will also be transferred over shared memory so almost the same process which was performed by the Functions Host for InvocationRequest will now be performed by the worker for InvocationResponse
• The worker will also generate a new MemoryMappedFile (or in case of Python worker, called mmap) with a unique name, write the output data into it, and then respond back to the Functions Host with the same details in a SharedMemoryData field
• The Functions Host upon receiving InvocationResponse will open the MemoryMappedFile, read the output data and write it to the bound output Stream so it can be persisted (e.g. to Azure Blob Storage)

Sequence of Events

1. Invocation come to Functions Host
2. Downloads content (i.e. binding process)
3. Write to MemoryMappedFile
4. Hold a reference open to the MemoryMappedFile
5. As part of the invocation message, send the MemoryMappedFile name/offset/count
6. Python reads from MemoryMappedFile
7. Function invocation happens
8. If any outputs produced, Python writes to MemoryMappedFile
9. Python also holds a reference open to the MemoryMappedFile it produces (until Functions Host is done reading it, otherwise the MemoryMappedFile gets cleaned up if all references go away)
10. Functions Host gets back response of function invocation from the worker
11. Frees the MemoryMappedFile that Functions Host was holding (inputs)
12. Sends another message (using gRPC) to Python to now also free the MemoryMappedFile it produced (outputs) - this cane be done in the background (in a fire-and-forget manner) to not add any more latency to the function invocation compared to what we have today
13. Done

Additional context

[gohar94]

cc: @goiri

@pragnagopa Opening an issue here to track and document as we go. You can assign this to me. :)

[pragnagopa]

Tagging language worker owners as FYI. No action needed at this time. This is a heads up on a new feature coming to Rpc layer that language workers can opt-in.

• @mhoeger - NodeJs Worker
• @AnatoliB - Powershell Worker
• @vrdmr - Python worker
• @amamounelsayed - Java Worker

[pragnagopa]

@gohar94 - Ok to assign this issue to Sprint 88 ?