The "Problem"¶

DPM routes data from front-ends to clients and, as the data is picked apart and sent, it is possibly scaled. This works well and, since the new DPMs aren't heavily loaded (yet), we haven't seen performance problems. There is a concern, however, that we won't scale up as much as we would like. Charlie King and Rich Neswold, during a discussion about DPM, realized that data is routed as follows:

1. An Erlang process receives data from a front-end as a binary.
2. It sends pieces of the binary to processes communicating with ACNET clients.
3. If the client asks for raw data, the binary can be directly placed in a protocol message and marshaled. If the data needs to be scaled, the process takes the contents of the binary, scales it, puts the floating point value in the protocol message and the marshaller turns it into a binary.

Scaled data for a single scalar isn't too bad, but scaling an array of data is pretty wasteful because the process loops through the binary and generates a list of doubles which the marshaller turns back into a binary. It would be nice to avoid the intermediate list from being generated.

A Solution¶

The protocol source file could include an annotation that says we want to marshall a field ourselves. The generator would then create a function that marshals all fields of the message except the marked one. The function would take a parameter that would provide the marshaled field.

However:

• Do we only allow one field per message to be annotated this way?
• This needs to be done in a type-safe way (i.e. we can't let the programmer provide an arbitrary binary as the marshaled value; it needs to be properly formed.)