Re: [ANN] lua-pb Lua Protocol Buffers

[Josh Haberman <jhaberman@...>]

Robert G. Jakabosky <bobby <at> sharedrealm.com> writes:
> -- person.proto:
> message Person {
>   required string name = 1;
>   required int32 id = 2;
>   required string email = 3;
> }
>
> -- example_person.lua:
> require"pb" -- first load lua-pb
> require"person" -- load the above .proto file
> msg = person.Person() -- create a Person message

I'm all for magic where there's a benefit, but what is
the benefit over:

  person = pb.load("person.proto")

This seems clearer to me.

If you're going the magical route, I think it makes more
sense to load things into the top-level namespace, based
on the package name specified in the .proto file:

  // person.proto
  package foo.bar
  message Person { ... }

  -- test.lua
  require "person"
  msg = foo.bar.Person()

> I am planning on adding these methods to the message interface:
> msg:MergeFrom(msg1)
> msg:CopyFrom(msg1)
> msg:Clear()
> msg:IsInitialized()
> msg:MergeFromString(str)
> msg:ParseFromString(str)
> msg:SerializeToString()
> msg:SerializePartialToString()
> msg:ByteSize()

I'm thinking more and more that it makes sense to separate
the in-memory representation of a protobuf from its
serializations (text format, binary format, JSON, etc),
both code-wise and API-wise.

I have plans to write a C-based protobuf extension for
Lua, and my plans were to have something like:

  -- These are as you mentioned, because they are not
  -- specific to any one serialization format.
  msg:Clear()
  msg:IsInitialized()
  msg:CopyFrom(msg1)

  -- These are specific to one or more serializations:
  pb.Serialize(msg)
  pb.SerializeText(msg)
  pb.SerializeJSON(msg)
  msg = pb.Parse(str)
  msg = pb.ParseText(str)
  msg = pb.ParseJSON(str)

Don't worry, I don't plan to use the "pb" namespace --
I'm planning to put everything under "upb", since that's
the name of my project:
  https://github.com/haberman/upb/wiki

Some other things to consider:

- do you plan to allow reparenting of nested messages?  eg.
  msg.foo = Foo().  I ask because you say you're emulating
  Python proto, which does not allow this AFAIK and instead
  uses the C++ convention of: msg.mutable_foo.  I've always
  thought this was awkward for a dynamic language, so plan
  to allow reparenting, but in that case you have to watch
  out for cycles that the user may create.

- watch out for 64-bit integers, which lua_Number can't
  fully represent in its default configuration (since
  a double only has 52 bits of precision.  Probably the best
  you can do is just warn your users about the loss of
  precision.

Josh