Project

General

Profile

Python Generator

We now have some preliminary support for Python scripts to communicate using ACNET. In order to use DPM, Python needed to become a target of the protocol compiler. This page describes how to use Python with the protocol compiler.

Running the protocol source though this generator results in a file with the same base name as the .proto file appended with "_protocol.py". Python has only one level of name-spacing, which is at the module level. Because of this limitation, more information is packed into the names of classes to guarantee uniqueness.

Type-mapping

The protocol compiler types are mapped to Python types in the following way:

Protocol Compiler Type Python v2.x Python v3.x
bool False or True
int16 int
int32 int
int64 long int
double float
string str
binary bytearray
enum T defined as attributes of the module with the form T_ValName
T [] list
optional T attribute is defined as T or the attribute doesn't exist
Type Name{T1 f1; T2 f1} generates classes with the name Name_Type --
the fields of the message will be attributes of the class.

Python v2.x has two integer sizes; 16-bit and 32-bit use the efficient int type and 64-bit integers use the unlimited precision long type. Python v3.x just has int types. For enumerations, the set of names of the enumerated values are converted to module-scoped variable names. Optional fields either contain a value or aren't set as an attribute. You can test for an optional field using the .has_attr() method.

The struct, request and reply keywords, in the protocol source file, define new data types and are implemented in Python as a new class. The name of the class will consist of the protocol name followed the type ("struct", "request" or "reply") separated by an underscore. These classes will have a .marshal() method which returns a generator that generates the character stream of the marshaled message. Note that calling .marshal() does not marshal the message, iterating across the generator does.

The module exports two functions, unmarshal_request and unmarshal_reply. These take an iterator and return a request or reply object, respectively. If there's any error in the incoming stream, they throw a run-time exception.

The module also includes documentation which can be accessed via Python's help module.

Example

For this example, we'll define a protocol for a simple ACNET chat system (it won't have chat rooms.) The source for the protocol is

request register {
    string nickname;
}

reply registered {
    int32 id;
}

reply message {
    int64 time;
    string who; 
    string what;
}

request deliver {
    int32 id;
    string message;
}

reply delivered {
}

The protocol compiler only creates encoders/decoders for the messages. It doesn't care how you transfer them; it could be via ACNET, TCP, or simply written to a file. In this example, we're going to use ACNET and we've added fields to the messages which lend themselves to ACNET communication. Communication between chat clients and the server works like this:

  • The chat server creates a CHAT ACNET handle. This is the handle where clients will send their messages.
  • A client sends a register request to the server as a multiple-reply request. The nickname field will get filled with the user's nickname. The first reply for the request will be a registered reply where the id field holds a unique value for the client.
  • When a client wants to send a chat message, it send a single-reply request using the deliver request. the id field is the client's unique ID and the message field holds the text. The server will return a single delivered reply. Each text message is a new, single-reply request.
  • When the server receives a deliver request, it translates it into a message reply (filling in when the message arrived, who sent it, and the message.) This reply is sent to all open multiple-reply requests. Viewed another way, a client makes a multiple-reply request. The first reply is a registered reply. All remaining replies are message replies.
  • When a client cancels the initial, multiple-reply request, the server removes it from its tables/resources.

Running this source through the protocol compiler ("pc -l python Chat.proto") yields Chat_protocol.py. Creating a register request of the protocol, using the nickname "Dude", is simple:

msg = Chat_protocol.register_request()
msg.nickname = "Dude" 

The .marshal() method returns a generator which can automatically be used as an iterator. Many Python library containers directly use iterators, so the result of .marshal() can be directly passed to files or sockets, for instance. Datagrams, however, need to be passed the whole message and so we need to do the iteration ourselves. Fortunately the byte array constructor also accepts an iterator, so getting a binary buffer of the marshaled message is also easy:

bin = bytearray(msg.marshal())

To decode the reply, we need an iterator over a string. When reading a TCP socket or file content, an iterator can be made which starts at the current point in a file. A UDP socket, like those in ACNET will return a binary message, so to decode it, we need one more step:

bin = get_next_binary_message()
msg = Chat_protocol.unmarshal_reply(iter(str(bin)))

If Chat_protocol.unmarshal_reply() doesn't throw an exception, then msg will contain a properly formed protocol reply message. We don't yet have a Receiver class, like the C++ generator, to properly route the incoming messages, so right now your Python script will have to check to see which instance it is. This results in chaining if statements:

if isinstance(msg, Chat_protocol.registered_reply):
    # Handle the register reply
elif isinstance(msg, Chat_protocol.message_reply):
    # Handle the message reply
else:
    # Handle the delivered reply