Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iox-#2128 Support segment name configuration #1

Open
wants to merge 1 commit into
base: iox-#2128-segment-mapping-design
Choose a base branch
from
Open

Iox-#2128 Support segment name configuration #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/design/draft/named-segments.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ In order to determine which segments to map:

## Development Roadmap

- [ ] Extend the RouDi config to support specifying segment names to be added to the `SegmentConfig`
- [x] Extend the RouDi config to support specifying segment names to be added to the `SegmentConfig`
- [ ] Add the name to the `MePooSegment` data structure and allow multiple write-access segments to be mapped.
- [ ] Update producer options structs to include segment names
- [ ] Update producer segment selection logic to take the segment name into account
Expand Down
2 changes: 1 addition & 1 deletion doc/website/advanced/configuration-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ int main(int argc, char* argv[])
mepooConfig.addMemPool({265, 10000});

auto currentGroup = iox::PosixGroup::getGroupOfCurrentProcess();
roudiConfig.m_sharedMemorySegments.push_back({currentGroup.getName(), currentGroup.getName(), mepooConfig});
roudiConfig.m_sharedMemorySegments.emplace_back(currentGroup.getName(), currentGroup.getName(), currentGroup.getName(), mepooConfig);

// configure the chunk count for the introspection; each introspection topic gets this number of chunks
roudiConfig.introspectionChunkCount = 10;
Expand Down
6 changes: 3 additions & 3 deletions iceoryx_examples/ice_access_control/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,9 @@ iox::mepoo::MePooConfig mepooConfig;
mepooConfig.addMemPool({128, 1000});

// Create an entry for a new shared memory segment from the mempooConfig and add it to the roudiConfig
// Parameters are {"ReaderGroup", "WriterGroup", MemoryPoolConfig}
roudiConfig.m_sharedMemorySegments.push_back({"unprivileged", "privileged", mepooConfig});
roudiConfig.m_sharedMemorySegments.push_back({"infotainment", "infotainment", mepooConfig});
// Parameters are {"SegmentName", "ReaderGroup", "WriterGroup", MemoryPoolConfig}
roudiConfig.m_sharedMemorySegments.emplace_back("privileged", "unprivileged", "privileged", mepooConfig);
roudiConfig.m_sharedMemorySegments.emplace_back("infotainment", "infotainment", "infotainment", mepooConfig);
```

The `roudiConfig` is composed of a memory pool config called `mepooConfig`. When the segment is created, one needs to
Expand Down
6 changes: 3 additions & 3 deletions iceoryx_examples/ice_access_control/roudi_main_static_segments.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ int main(int argc, char* argv[])
mepooConfig.addMemPool({128, 1000});

// Create an entry for a new shared memory segment from the mempooConfig and add it to the roudiConfig
// Parameters are {"ReaderGroup", "WriterGroup", MemoryPoolConfig}
roudiConfig.m_sharedMemorySegments.push_back({"unprivileged", "privileged", mepooConfig});
roudiConfig.m_sharedMemorySegments.push_back({"infotainment", "infotainment", mepooConfig});
// Parameters are {"SegmentName", "ReaderGroup", "WriterGroup", MemoryPoolConfig}
roudiConfig.m_sharedMemorySegments.emplace_back("privileged", "unprivileged", "privileged", mepooConfig);
roudiConfig.m_sharedMemorySegments.emplace_back("infotainment", "infotainment", "infotainment", mepooConfig);
//! [config]

IceOryxRouDiApp roudi(cmdLineArgs.value(), roudiConfig);
Expand Down
15 changes: 7 additions & 8 deletions iceoryx_examples/iceperf/roudi_main_static_config.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (c) 2021 by Apex.AI Inc. All rights reserved.
// Copyright (c) 2024 by Latitude AI. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -53,19 +54,17 @@ int main(int argc, char* argv[])
auto currentGroup = iox::PosixGroup::getGroupOfCurrentProcess();

/// Create an Entry for a new Shared Memory Segment from the MempoolConfig and add it to the RouDiConfig
roudiConfig.m_sharedMemorySegments.push_back({currentGroup.getName(), currentGroup.getName(), mepooConfig});
roudiConfig.m_sharedMemorySegments.emplace_back(currentGroup.getName(), currentGroup.getName(), currentGroup.getName(), mepooConfig);

/// For the case that you want to give accessrights to the shm segments, you need to set groupnames as fixed string.
/// These names defines groups whose members are either to read/write from/to the respective shared memory segment.
/// @note the groups needs to be registered in /etc/groups.
/// @code
/// iox::PosixGroup::string_t readerGroup{iox::TruncateToCapacity, "readerGroup"};
/// iox::PosixGroup::string_t writerGroup{iox::TruncateToCapacity, "writerGroup"};
/// iox::mepoo::SegmentConfig::SegmentEntry segentry({readerGroup, writerGroup, mepooConfig});
/// roudiConfig.m_sharedMemorySegments.push_back(
/// {iox::PosixGroup::string_t(iox::TruncateToCapacity, reader),
/// iox::PosixGroup::string_t(iox::TruncateToCapacity, writer),
/// mempoolConfig})
/// roudiConfig.m_sharedMemorySegments.emplace_back(
/// "segmentName",
/// "readerGroup",
/// "writerGroup",
/// mempoolConfig)
/// @endcode

/// configure the chunk count for the introspection; each introspection topic gets this number of chunks
Expand Down
8 changes: 6 additions & 2 deletions iceoryx_posh/include/iceoryx_posh/mepoo/segment_config.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (c) 2019 by Robert Bosch GmbH. All rights reserved.
// Copyright (c) 2024 by Latitude AI. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -31,18 +32,21 @@ struct SegmentConfig
{
struct SegmentEntry
{
SegmentEntry(const PosixGroup::groupName_t& readerGroup,
SegmentEntry(const ShmName_t& name,
const PosixGroup::groupName_t& readerGroup,
const PosixGroup::groupName_t& writerGroup,
const MePooConfig& memPoolConfig,
iox::mepoo::MemoryInfo memoryInfo = iox::mepoo::MemoryInfo()) noexcept
: m_readerGroup(readerGroup)
: m_name(name)
, m_readerGroup(readerGroup)
, m_writerGroup(writerGroup)
, m_mempoolConfig(memPoolConfig)
, m_memoryInfo(memoryInfo)

{
}

ShmName_t m_name;
PosixGroup::groupName_t m_readerGroup;
PosixGroup::groupName_t m_writerGroup;
MePooConfig m_mempoolConfig;
Expand Down
15 changes: 13 additions & 2 deletions iceoryx_posh/include/iceoryx_posh/roudi/roudi_config_file_provider.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
// Copyright (c) 2019 by Robert Bosch GmbH. All rights reserved.
// Copyright (c) 2021 by Apex.AI Inc. All rights reserved.
// Copyright (c) 2024 by Latitude AI. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -35,6 +36,8 @@ namespace roudi
/// MAX_NUMBER_OF_MEMPOOLS_PER_SEGMENT_EXCEEDED - the max number of mempools per segment is exceeded
/// MEMPOOL_WITHOUT_CHUNK_SIZE - chunk size not specified for the mempool
/// MEMPOOL_WITHOUT_CHUNK_COUNT - chunk count not specified for the mempool
/// EXCEPTION_IN_PARSER - An exception occurred in the toml file parsing library
/// NAMED_SEGMENT_IN_V1_CONFIG - A segment name was specified in a v1 config
enum class RouDiConfigFileParseError
{
FILE_OPEN_FAILED,
Expand All @@ -46,7 +49,11 @@ enum class RouDiConfigFileParseError
MAX_NUMBER_OF_MEMPOOLS_PER_SEGMENT_EXCEEDED,
MEMPOOL_WITHOUT_CHUNK_SIZE,
MEMPOOL_WITHOUT_CHUNK_COUNT,
EXCEPTION_IN_PARSER
EXCEPTION_IN_PARSER,
NAMED_SEGMENT_IN_V1_CONFIG,
SEGMENT_NAME_EXCEEDS_MAX_LENGTH,
WRITER_GROUP_NAME_EXCEEDS_MAX_LENGTH,
READER_GROUP_NAME_EXCEEDS_MAX_LENGTH
};

constexpr const char* ROUDI_CONFIG_FILE_PARSE_ERROR_STRINGS[] = {"FILE_OPEN_FAILED",
Expand All @@ -58,7 +65,11 @@ constexpr const char* ROUDI_CONFIG_FILE_PARSE_ERROR_STRINGS[] = {"FILE_OPEN_FAIL
"MAX_NUMBER_OF_MEMPOOLS_PER_SEGMENT_EXCEEDED",
"MEMPOOL_WITHOUT_CHUNK_SIZE",
"MEMPOOL_WITHOUT_CHUNK_COUNT",
"EXCEPTION_IN_PARSER"};
"EXCEPTION_IN_PARSER",
"NAMED_SEGMENT_IN_V1_CONFIG",
"SEGMENT_NAME_EXCEEDS_MAX_LENGTH",
"WRITER_GROUP_NAME_EXCEEDS_MAX_LENGTH",
"READER_GROUP_NAME_EXCEEDS_MAX_LENGTH"};

/// @brief Base class for a config file provider.
class RouDiConfigFileProvider
Expand Down
3 changes: 2 additions & 1 deletion iceoryx_posh/roudi_env/source/minimal_roudi_config.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (c) 2023 by Mathias Kraus <[email protected]>. All rights reserved.
// Copyright (c) 2024 by Latitude AI. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -26,7 +27,7 @@ RouDiConfig_t MinimalRouDiConfigBuilder::create() const noexcept
mepoo::MePooConfig mepooConfig;
mepooConfig.addMemPool({m_payloadChunkSize, m_payloadChunkCount});
auto currentGroup = PosixGroup::getGroupOfCurrentProcess();
roudiConfig.m_sharedMemorySegments.push_back({currentGroup.getName(), currentGroup.getName(), mepooConfig});
roudiConfig.m_sharedMemorySegments.emplace_back(currentGroup.getName(), currentGroup.getName(), currentGroup.getName(), mepooConfig);

roudiConfig.introspectionChunkCount = m_introspectionChunkCount;
roudiConfig.discoveryChunkCount = m_discoveryChunkCount;
Expand Down
3 changes: 2 additions & 1 deletion iceoryx_posh/source/mepoo/segment_config.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (c) 2019 by Robert Bosch GmbH. All rights reserved.
// Copyright (c) 2024 by Latitude AI. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand All @@ -24,7 +25,7 @@ namespace mepoo
SegmentConfig& SegmentConfig::setDefaults() noexcept
{
auto groupName = PosixGroup::getGroupOfCurrentProcess().getName();
m_sharedMemorySegments.push_back({groupName, groupName, MePooConfig().setDefaults()});
m_sharedMemorySegments.emplace_back(groupName, groupName, groupName, MePooConfig().setDefaults());
return *this;
}

Expand Down
33 changes: 28 additions & 5 deletions iceoryx_posh/source/roudi/roudi_config_toml_file_provider.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) 2019 by Robert Bosch GmbH. All rights reserved.
// Copyright (c) 2022 by Apex AI Inc. All rights reserved.
// Copyright (c) 2022 by NXP. All rights reserved.
// Copyright (c) 2024 by Latitude AI. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -108,7 +109,7 @@ TomlRouDiConfigFileProvider::parse(std::istream& stream) noexcept
return iox::err(iox::roudi::RouDiConfigFileParseError::NO_GENERAL_SECTION);
}
auto configFileVersion = general->get_as<uint32_t>("version");
if (!configFileVersion || *configFileVersion != 1)
if (!configFileVersion || *configFileVersion == 0 || *configFileVersion > 2U)
{
return iox::err(iox::roudi::RouDiConfigFileParseError::INVALID_CONFIG_FILE_VERSION);
}
Expand All @@ -128,8 +129,29 @@ TomlRouDiConfigFileProvider::parse(std::istream& stream) noexcept
iox::RouDiConfig_t parsedConfig;
for (auto segment : *segments)
{
auto maybe_name = segment->get_as<std::string>("name");
if (maybe_name && *configFileVersion == 1U)
{
return iox::err(iox::roudi::RouDiConfigFileParseError::NAMED_SEGMENT_IN_V1_CONFIG);
}
auto name = maybe_name.value_or(into<std::string>(groupOfCurrentProcess));
if (name.size() > ShmName_t::capacity())
{
return iox::err(iox::roudi::RouDiConfigFileParseError::SEGMENT_NAME_EXCEEDS_MAX_LENGTH);
}

auto writer = segment->get_as<std::string>("writer").value_or(into<std::string>(groupOfCurrentProcess));
if (writer.size() > PosixGroup::groupName_t::capacity())
{
return iox::err(iox::roudi::RouDiConfigFileParseError::WRITER_GROUP_NAME_EXCEEDS_MAX_LENGTH);
}

auto reader = segment->get_as<std::string>("reader").value_or(into<std::string>(groupOfCurrentProcess));
if (reader.size() > PosixGroup::groupName_t::capacity())
{
return iox::err(iox::roudi::RouDiConfigFileParseError::READER_GROUP_NAME_EXCEEDS_MAX_LENGTH);
}

iox::mepoo::MePooConfig mempoolConfig;
auto mempools = segment->get_table_array("mempool");
if (!mempools)
Expand All @@ -156,10 +178,11 @@ TomlRouDiConfigFileProvider::parse(std::istream& stream) noexcept
}
mempoolConfig.addMemPool({*chunkSize, *chunkCount});
}
parsedConfig.m_sharedMemorySegments.push_back(
{PosixGroup::groupName_t(iox::TruncateToCapacity, reader.c_str(), reader.size()),
PosixGroup::groupName_t(iox::TruncateToCapacity, writer.c_str(), writer.size()),
mempoolConfig});
parsedConfig.m_sharedMemorySegments.emplace_back(
ShmName_t(iox::TruncateToCapacity, name.c_str(), name.size()),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we care if the name gets truncated? I'm wondering about cases where we have two or more segments with names that are too long, but whose prefixes match up to the point where they are truncated, like:

a_really_long_name_for_a_segment_that_is_idential_up_to_here
a_really_long_name_for_a_segment_that_is_idential_up_to_there

But maybe the truncation of both just yields: a_really_long_name_for_a_segment_that_is_idential_up_to_ and we have a weird name conflict which is not apparent by reading the config file.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this is kind of an annoying point. FWIW in the next PR I replace the "multiple writers" error with a "duplicate segment name" error which causes the program to crash. You get an informative error message. As you will see it could probably be handled more close to the config parsing better but I'm trying to balance locality of changes vs taking on larger refactoring.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I presume there are also going to be weird errors if the writer/reader groups are too large.

I might poke around a bit at adding a config-time check for the length of these.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah makes sense and I don't expect we'll encounter the string size limit as a problem all that often anyway.

PosixGroup::groupName_t(iox::TruncateToCapacity, reader.c_str(), reader.size()),
PosixGroup::groupName_t(iox::TruncateToCapacity, writer.c_str(), writer.size()),
mempoolConfig);
}

return iox::ok(parsedConfig);
Expand Down
4 changes: 2 additions & 2 deletions iceoryx_posh/test/integrationtests/test_posh_mepoo.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,8 @@ class Mepoo_IntegrationTest : public Test

auto currentGroup = iox::PosixGroup::getGroupOfCurrentProcess();
iox::RouDiConfig_t roudiConfig;
roudiConfig.m_sharedMemorySegments.push_back(
{currentGroup.getName(), currentGroup.getName(), mempoolConfig});
roudiConfig.m_sharedMemorySegments.emplace_back(
currentGroup.getName(), currentGroup.getName(), currentGroup.getName(), mempoolConfig);
return roudiConfig;
}
else
Expand Down
11 changes: 6 additions & 5 deletions iceoryx_posh/test/moduletests/test_mepoo_segment_management.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
// Copyright (c) 2019 by Robert Bosch GmbH. All rights reserved.
// Copyright (c) 2021 - 2022 by Apex.AI Inc. All rights reserved.
// Copyright (c) 2024 by Latitude AI. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -68,16 +69,16 @@ class SegmentManager_test : public Test
SegmentConfig getSegmentConfig()
{
SegmentConfig config;
config.m_sharedMemorySegments.push_back({"iox_roudi_test1", "iox_roudi_test2", mepooConfig});
config.m_sharedMemorySegments.push_back({"iox_roudi_test2", "iox_roudi_test3", mepooConfig});
config.m_sharedMemorySegments.emplace_back("segment_name1", "iox_roudi_test1", "iox_roudi_test2", mepooConfig);
config.m_sharedMemorySegments.emplace_back("segment_name2", "iox_roudi_test2", "iox_roudi_test3", mepooConfig);
return config;
}

SegmentConfig getInvalidSegmentConfig()
{
SegmentConfig config;
config.m_sharedMemorySegments.push_back({"iox_roudi_test1", "iox_roudi_test1", mepooConfig});
config.m_sharedMemorySegments.push_back({"iox_roudi_test3", "iox_roudi_test1", mepooConfig});
config.m_sharedMemorySegments.emplace_back("segment_name1", "iox_roudi_test1", "iox_roudi_test1", mepooConfig);
config.m_sharedMemorySegments.emplace_back("segment_name2", "iox_roudi_test3", "iox_roudi_test1", mepooConfig);
return config;
}

Expand All @@ -86,7 +87,7 @@ class SegmentManager_test : public Test
SegmentConfig config;
for (uint64_t i = 0U; i < iox::MAX_SHM_SEGMENTS; ++i)
{
config.m_sharedMemorySegments.push_back({"iox_roudi_test1", "iox_roudi_test1", mepooConfig});
config.m_sharedMemorySegments.emplace_back("segment_name1", "iox_roudi_test1", "iox_roudi_test1", mepooConfig);
}
return config;
}
Expand Down
Loading
Loading