Copy a std::vector to a repeated field from protobuf with memcpy

akristmann picture akristmann · Mar 19, 2013 · Viewed 31.4k times · Source

At first I have this simple protobuf file

message messagetest
{
    ...
    repeated float samples = 6;
    ....
}

Which creates a headerfile with this methods

    //repeated float samples = 6;
      inline int samples_size() const;
      inline void clear_samples();
      static const int kSamplesFieldNumber = 6;
      inline float samples(int index) const;
      inline void set_samples(int index, float value);
      inline void add_samples(float value);
      inline const ::google::protobuf::RepeatedField< float >&  samples() const;
      inline ::google::protobuf::RepeatedField< float >* mutable_samples();

What I'm basically doing is to copy all data one by one in a for loop.

int main(int argc, char** argv)
{    
    messagetest fMessage;

    vector<float> fData (1000, 0);

    // Create 1000 random values
    for (int i = 0; i < fData.size(); i++)
    {
        fData[i] = rand() % 1001;
    }

    for (int j = 0; j < fData.size(); j++)
    {
        fMessage.add_samples(fData[j]);    
    }

    return 0;
}

But I want to use a method like memcpy to accelerate the copy process. It is just an idea that comes to my mind. If it's completely wrong correct me. The last declaration in the headerfile is:

inline ::google::protobuf::RepeatedField< float >* mutable_samples();

I have no idea what this method does (lack of skill). But it kind of looks like a vector. Maybe that's the solution for my problem. If so, I have no idea how to implement it.


Edit SOLVED:

Memcpy is in any case faster, because you have one method call and then you copy the data. Compared to the for loop which has to call the "fMessage.add_samples()" method 1000 times.

One flaw of this approach is that you have to know the size of your vector to reserve data for your samples, but then you can allocate at programm start the memory for fMessage.sample (this works for me).

fMessage.mutable_samples()->Reserve(fData.size());

for (int j = 0; j < fData.size(); j++)
{
   fMessage.add_samples(0);          
}

And now i can get the data from a stream in a while loop and copy them with memcpy to my datastructer

while(42)
{
   //fancy streaming things to get vector fData

  memcpy(fMessage.mutable_samples()->mutable_data(),
         &fData[0],
         sizeof(float)*fData.size());
}

Answer

mgild picture mgild · Apr 13, 2018

Since this isn't here yet and I like one-liners:

*fMessage.mutable_samples() = {fData.begin(), fData.end()};