Segment Fault occurred due to data of string/bytes type field isn't allocated on arena

[Ginakira]

Describe the bug
When I use arena mode with cyber example code (cyber/examples/common_component_example), I found that reader will crash when the writer wrote the content field with a string that length greater than 16 (null terminator included).

To Reproduce
Proto:

message Driver {
  optional string content = 1;
  optional uint64 msg_id = 2;
  optional uint64 timestamp = 3;
};

Cyber config:

shm_conf {
  arena_shm_conf {
    arena_channel_conf {
      channel_name: "/apollo/test"
      max_msg_size: 33554432
      max_pool_size: 32
    }
  }
}

Writer:

#include "cyber/examples/proto/examples.pb.h"

#include "cyber/cyber.h"
#include "cyber/time/rate.h"
#include "cyber/time/time.h"

using apollo::cyber::Rate;
using apollo::cyber::Time;
using apollo::cyber::examples::proto::Driver;

int main(int argc, char* argv[]) {
  // init cyber framework
  apollo::cyber::Init(argv[0]);
  // create talker node
  auto talker_node = apollo::cyber::CreateNode("channel_test_writer");
  // create talker
  auto talker = talker_node->CreateWriter<Driver>("/apollo/test");
  Rate rate(2.0);

  std::string content("apollo_testrr"); // length 13
  while (apollo::cyber::OK()) {
    static uint64_t seq = 0;

    auto msg = talker->AcquireMessage();
    msg->set_timestamp(Time::Now().ToNanosecond());
    msg->set_msg_id(seq++);
    msg->set_content(content + std::to_string(seq - 1));
    talker->Write(msg);

    AINFO << "/apollo/test sent message, seq=" << (seq - 1) << ";";
    rate.Sleep();
  }
  return 0;
}

Log:

I1218 17:05:00.766216  6061 common_component_example.cc:25] [mainboard]Start common component Proc [161] [98]apollo_testrr98
I1218 17:05:01.099596  6051 common_component_example.cc:25] [mainboard]Start common component Proc [162] [98]apollo_testrr98
I1218 17:05:01.432871  6055 common_component_example.cc:25] [mainboard]Start common component Proc [163] [99]apollo_testrr99
Segmentation fault

Desktop:

• OS: Ubuntu 18.04

Additional context
IMO, the reason it works fine with the content field's length less than 16 is because of std::string's SSO (Small String Optimization). When its length is less than a specific number (maybe 16B), the data will embedded in the std::string object. When the length exceeds, std::string will allocate memory on the heap to store it.
So, if string data is allocated on the heap instead of arena-managed memory, the address returned by std::string::data() will be on the writer's heap; to the reader, it is an illegal address.

Protobuf says the string fields store their data on the heap even when the containing message is on the arena. The same goes for bytes type field.

This is a critical issue because it means any message including string or bytes could not transmit with arena mode on shared memory. The vast majority of protos include the string/bytes type fields.
Is it a known issue? Is there already any solution in the internal version of apollo?

Thanks!

[hearto1314]

sadly, we don't have a solution either.

[Ginakira]

sadly, we don't have a solution either.

Got it. Thanks for your reply!