Google Protocol Buffer
11594 ワード
Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler.
By defining the protocol buffer message types in
Each message type has one or more uniquely numbered fields, and each field has a name and a value type, where values types can be numbers (integer or floating point), booleans, strings, raw bytes, or even other protocol buffer message types, allowing us to structure our data hierarchically. The field can be
Once we have the message defined, we can run the protocal buffer compiler for our chosen language on the
In Java we can write codes to utilize the
And similar codes in C++.
With Protocol Buffer, we can add new fields to our message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. Therefore, we can extend our protocol without worrying about breaking existing code.
Define message formats in a Use the protocol buffer compiler. Use the Java/C++/Python protocol buffer API to write and read messages.
The definitions in a
The
The
Next, we have the
A more complicated structure is supported. As shown above,
The
Each field must be annotated with one of the following modifiers: required: a value for the field must be provided, otherwise the message will be considered "uninitialized". Trying to build an uninitialized message will throw a optional: the field may or may not be set. If an optional field value isn't set, a default value is used. For simple types, we can specify our own default value, as we've done for the phone number type in the example. Otherwise, a system default is used: zero for numeric types, the empty string for strings, false for bools. repeated: the field may be repeated any number of times (including zero). The order of the repeated values will be preserved in the protocol buffer.
We'll find a complete guide to writing
Now that we have a If we haven't installed the compiler, download the package. Now run the compiler, specifying the source directory (where our application's source code lives with the current directory as default), the destination directory (where we want the generated code to go; often the same as the source directory), and the path to our
Assuming we want Java classes, we use the
This generates com/example/tutorial/AddressBookProtos.java in our specified destination directory.
If we look in
Here are some of the accessors for the
Meanwhile,
As we can see, there are simple JavaBeans-style getters and setters for each field.
Here is an example about how to create an instance of
To persist the data, we can simply run the following codes.
Once persisted, we can read the data as such.
For example in C++, we can write codes like:
Then later on we can read the message back like:
Check out this post for more information:)
Google Developer Protocol Buffer
How does protocol buffer work?
By defining the protocol buffer message types in
.proto
files, we can specify how our information to be serialized and how they are structured. Here is a basic example of a .proto
file that defines a message containing information about a person: message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}
Each message type has one or more uniquely numbered fields, and each field has a name and a value type, where values types can be numbers (integer or floating point), booleans, strings, raw bytes, or even other protocol buffer message types, allowing us to structure our data hierarchically. The field can be
optional
or required
. Once we have the message defined, we can run the protocal buffer compiler for our chosen language on the
.proto
file to generate data access classes Person
. We can then use this class in our application to populate, serialize, and retrieve Person
protocol buffer messages. In Java we can write codes to utilize the
Person
class like this. Person john = Person.newBuilder()
.setId(1234)
.setName("John Doe")
.setEmail("[email protected]")
.build();
output = new FileOutputStream(args[0]);
john.writeTo(output);
And similar codes in C++.
Person john;
fstream input(argv[1],
ios::in | ios::binary);
john.ParseFromIstream(&input);
id = john.id();
name = john.name();
email = john.email();
With Protocol Buffer, we can add new fields to our message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. Therefore, we can extend our protocol without worrying about breaking existing code.
Basic Steps
.proto
file. Define message formats in a .proto file
The definitions in a
.proto
file are simple: we add a message for each data structure we want to serialize, then specify a name and a type for each field in the message. Let's use the an address book application as an example. To define our messages, we start with the addressbook.proto
. package tutorial;
option java_package = "com.example.tutorial";
option java_outer_classname = "AddressBookProtos";
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}
message AddressBook {
repeated Person person = 1;
}
package & java_package
The
.proto
file starts with a package declaration package tutorial
, which helps to prevent name conflicting. It will by default used as the Java package unless we explicitly specify the java_package
as shown above. It is recommended to always specify a package to avoid name collisions in Protocol Buffer name spaces even in non-Java languages. java_outer_classname
The
java_outer_classname
option defines the class name which should contain all of the classes in this file. If we don't give a java_outer_classname
explicitly, it will be generated by converting the file name to camel case. For example, "my_proto.proto"would, by default, use "MyProto"as the outer class name. Message
Next, we have the
message
definitions. A message is just an aggregate containing a set of typed fields. Several basic types are available: bool
, int32
, float
, double
and string
. Complex message type
A more complicated structure is supported. As shown above,
Person
message contains PhoneNumber
messages, while the AddressBook
message contains Person
messages. We can even define message types nested inside other messages – as we can see, the PhoneNumber
type is defined inside Person
. Enum is also supported– here we have a phone number that can be one of MOBILE, HOME, or WORK. Number tag
The
" = 1"
, " = 2"
markers on each element identify the unique "tag"that field uses in the binary encoding. Tag numbers 1-15 require one less byte to encode than higher numbers, so as an optimization we can decide to use those tags for the commonly used or repeated elements, leaving tags 16 and higher for less-commonly used optional elements. Each element in a repeated field requires re-encoding the tag number, so repeated fields are particularly good candidates for this optimization. Required Optional Repreated
Each field must be annotated with one of the following modifiers:
RuntimeException
. Parsing an uninitialized message will throw an IOException
. More
We'll find a complete guide to writing
.proto
files – including all the possible field types – in the Protocol Buffer Language Guide. Compiling Protocol Buffers
Now that we have a
.proto
, the next thing we need to do is generate the classes we'll need to read and write AddressBook
messages. To do this, we need to run the protocol buffer compiler protoc on our .proto
: .proto
. protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/addressbook.proto
Assuming we want Java classes, we use the
--java_out
option here. Similar options are provided for other supported languages. This generates com/example/tutorial/AddressBookProtos.java in our specified destination directory.
The Protocol Buffer API - Java
If we look in
AddressBookProtos.java
, we can see that it defines a class called AddressBookProtos
, nested within which is a class for each message we specified in addressbook.proto
. Each class has its own Builder class that we use to create instances of that class. Here are some of the accessors for the
Person
class. // required string name = 1;
public boolean hasName();
public String getName();
// required int32 id = 2;
public boolean hasId();
public int getId();
// optional string email = 3;
public boolean hasEmail();
public String getEmail();
// repeated .tutorial.Person.PhoneNumber phone = 4;
public List getPhoneList();
public int getPhoneCount();
public PhoneNumber getPhone(int index);
Meanwhile,
Person.Builder
has the same getters plus setters: // required string name = 1;
public boolean hasName();
public java.lang.String getName();
public Builder setName(String value);
public Builder clearName();
// required int32 id = 2;
public boolean hasId();
public int getId();
public Builder setId(int value);
public Builder clearId();
// optional string email = 3;
public boolean hasEmail();
public String getEmail();
public Builder setEmail(String value);
public Builder clearEmail();
// repeated .tutorial.Person.PhoneNumber phone = 4;
public List getPhoneList();
public int getPhoneCount();
public PhoneNumber getPhone(int index);
public Builder setPhone(int index, PhoneNumber value);
public Builder addPhone(PhoneNumber value);
public Builder addAllPhone(Iterable value);
public Builder clearPhone();
As we can see, there are simple JavaBeans-style getters and setters for each field.
Create instances
Here is an example about how to create an instance of
Person
. Person john = Person.newBuilder()
.setId(1234)
.setName("John Doe")
.setEmail("[email protected]")
.addPhone(
Person.PhoneNumber.newBuilder()
.setNumber("555-4321")
.setType(Person.PhoneType.HOME))
.build();
Serialization and Deserialization
To persist the data, we can simply run the following codes.
Person john = Person.newBuilder()
.setId(1234)
.setName("John Doe")
.setEmail("[email protected]")
.addPhone(
Person.PhoneNumber.newBuilder()
.setNumber("555-4321")
.setType(Person.PhoneType.HOME))
.build();
// Write to file
FileOutputStream output = new FileOutputStream("target/person.ser");
john.writeTo(output);
output.close();
Once persisted, we can read the data as such.
// Read from file
Person person = Person.parseFrom(new FileInputStream("target/person.ser");
The Protocol Buffer API - C++
For example in C++, we can write codes like:
Person person;
person.set_name("John Doe");
person.set_id(1234);
person.set_email("[email protected]");
fstream output("myfile", ios::out | ios::binary);
person.SerializeToOstream(&output);
Then later on we can read the message back like:
fstream input("myfile", ios::in | ios::binary);
Person person;
person.ParseFromIstream(&input);
cout << "Name: " << person.name() << endl;
cout << "E-mail: " << person.email() << endl;
Check out this post for more information:)
Reference
Google Developer Protocol Buffer