Project

General

Profile

Feature #11474

Allow fields to be individually marshalled

Added by Richard Neswold almost 4 years ago. Updated about 1 month ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Parser
Target version:
-
Start date:
01/21/2016
Due date:
% Done:

0%

Estimated time:
Duration:

Description

The "Problem"

DPM routes data from front-ends to clients and, as the data is picked apart and sent, it is possibly scaled. This works well and, since the new DPMs aren't heavily loaded (yet), we haven't seen performance problems. There is a concern, however, that we won't scale up as much as we would like. Charlie King and Rich Neswold, during a discussion about DPM, realized that data is routed as follows:

  1. An Erlang process receives data from a front-end as a binary.
  2. It sends pieces of the binary to processes communicating with ACNET clients.
  3. If the client asks for raw data, the binary can be directly placed in a protocol message and marshaled. If the data needs to be scaled, the process takes the contents of the binary, scales it, puts the floating point value in the protocol message and the marshaller turns it into a binary.

Scaled data for a single scalar isn't too bad, but scaling an array of data is pretty wasteful because the process loops through the binary and generates a list of doubles which the marshaller turns back into a binary. It would be nice to avoid the intermediate list from being generated.

A Solution

The protocol source file could include an annotation that says we want to marshall a field ourselves. The generator would then create a function that marshals all fields of the message except the marked one. The function would take a parameter that would provide the marshaled field.

However:

  • Do we only allow one field per message to be annotated this way?
  • This needs to be done in a type-safe way (i.e. we can't let the programmer provide an arbitrary binary as the marshaled value; it needs to be properly formed.)

Related issues

Related to Protocol Compiler - Feature #11463: Add constraints to the grammarNew01/21/2016

History

#1 Updated by Richard Neswold almost 4 years ago

#2 Updated by Richard Neswold about 1 month ago

  • Status changed from New to Closed
  • Category set to Parser

This needs to be done in a type-safe way (i.e. we can't let the programmer provide an arbitrary binary as the marshaled value; it needs to be properly formed.)

The fact that the data needs to be validated removes any massive speed-up we would get copying a field from one message to another (because the marshaling function can't know the data came from a validated, received message.)

Plus, annotating a field in the protocol to support an optimization in a service implementation doesn't feel right.



Also available in: Atom PDF