getml / reflect-cpp Goto Github PK

Given this program:

#include <iostream>
#include <rfl.hpp>
#include <rfl/json.hpp>

struct foo
{
    std::string a;
    int         b;
};

int
main(int argc, char *argv[])
{
    auto x = foo { .a = "Hello, World!", .b = 100 };
    std::cout << rfl::json::write(x) << '\n';
}

How do I fix this error?

[build] /usr/bin/ld: CMakeFiles/scratch3.dir/scratch3/main.cpp.o: in function `auto rfl::internal::get_field_names<iron::foo>()':
[build] .../_deps/reflect-cpp-src/include/rfl/internal/get_field_names.hpp:126: undefined reference to `rfl::internal::fake_object<iron::foo>'
[build] collect2: error: ld returned 1 exit status

The program is linked with target reflectcpp_static.

Thank you.

Pertaining to the discussion related to @zerolfx's great PR (#29), it is necessary to use some kind of testing framework to fully reap the benefits of Github Actions. We are looking to introduce more serialization formats over the next 2-3 weeks. Being able to test them all in an automated manner will be a great benefit that I really look forward to.

Unlike textual formats, binary formats like CBOR, BSON, etc also support bytestrings. This should also be reflects in our library.

The JSON-only build does not work if you haven't installed vcpkg. You get error messages like this:

CMake Error at /usr/share/cmake-3.22/Modules/CMakeDetermineSystem.cmake:130 (message):
  Could not find toolchain file:
  /home/ubuntu/reflect-cpp/vcpkg/scripts/buildsystems/vcpkg.cmake
Call Stack (most recent call first):
  CMakeLists.txt:15 (project)

In some tests with CBOR I have noticed that reflect-cpp adds a significant overhead over tinycbor. This makes it less appealing to use this library in performance critical environments.

E.g., a minimal struct

struct Minimal {
  double foo;
  double bar;
  double baz;
  double qux;
};

has the following benchmark results:

~/projects/reflect-cpp$ ./build/linux-release/reflectcpp-bm
2024-03-04T21:29:54+01:00
Running ./build/linux-release/reflectcpp-bm
Run on (4 X 400 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x2)
  L1 Instruction 32 KiB (x2)
  L2 Unified 512 KiB (x2)
  L3 Unified 4096 KiB (x1)
Load Average: 0.06, 0.05, 0.01
---------------------------------------------------------------
Benchmark                     Time             CPU   Iterations
---------------------------------------------------------------
BM_CBOR_Write               160 ns          160 ns      4360721
BM_CBOR_Read                427 ns          427 ns      1645479
BM_CBOR_Write_tinycbor     28.9 ns         28.9 ns     24334360
BM_CBOR_Read_tinycbor       225 ns          225 ns      3099686

I've tried replacing usages of Box with stack objects in CBOR's Writer (see zann1x@9addb7c). This improves performance a bit, but there is still a 10x difference in comparison to tinycbor.

When I try to get a json from map<string, any> everything works as expected, for example:

struct TestStringKey {
    rfl::Field<"Field", std::map<std::string, std::string>> Field;
} testString = {std::map<std::string, std::string>{{"1", "first"}, {"2", "second"}}};

After conversion to json becomes:
{"Field":{"1":"first","2":"second"}}

But when I try to do the same for map<int, any> I see some strange output, foe example(I love examples):

struct TestIntKey {
    rfl::Field<"Field", std::map<int, std::string>> Field;
} testInt =  {std::map<int, std::string>{{1, "first"}, {2, "second"}}};

After conversion to json becomes:
{"Field":[[1,"first"],[2,"second"]]}

It became a list of ?unnamed? fields. This is weird, I am sure that one of the key features of json format is that we should be able to convert json to the same types in a different programming language. Let's check how the same serialization works in GoLang for example, I don't expect you to know it so let me explain a bit. C++ map<Key, Value> is equal to Go map[Key]Value syntax (actually unordered_map, but who cares). So I can write the same simple code on Go:

type TestIntKey struct {
	Field map[int]string
}
intKey := TestIntKey{map[int]string{
		1: "first",
		2: "second",
}}

res, _ := json.Marshal(&intKey)
fmt.Println(string(res))

The output will be:
{"Field":{"1":"first","2":"second"}}

It just took a key and converted it to a string. I decided to check if I can deserialize your library output to this go struct:

text := `{"Field":[[1,"first"],[2,"second"]]}`
err := json.Unmarshal([]byte(text), &intKey)
if err != nil {
    fmt.Println(err.Error())
}

And I got an error
json: cannot unmarshal array into Go struct field TestIntKey.Field of type map[int]string

For numeric types it is easy to fix, but what should happen when key is a bool type or event a struct? I saw that in some C# libraries it can serialize Dictionary to sth like:

[
  {
    "Key": {
      "Id": 69,
      "Name": "Gomer"
    },
    "Value": {
      "Id": 1488,
      "City": "Vavilon"
    }
  },
  {
    "Key": {
      "Id": 322,
      "Name": "Bart"
    },
    "Value": {
      "Id": 228,
      "City": "Florida"
    }
  }
]

But I also find it weird.

I want something like rfl::get_field_ref<0>(x), so I can assign value to it.

struct A {
  int a;
  double b;
};

A a;
rfl::get_field_ref<0>(x) = 1;

I dug the code a little and find rfl::internal::to_ptr_named_tuple, but this can only get a const pointer

auto ptr_named_tuple = rfl::internal::to_ptr_named_tuple(homer);
ptr_named_tuple.get<0>(); // this is a const pointer.

So how to get an non-const reference/pointer?

Could anyone help me, thanks in advance!

My example program and compile log (reflect-cpp is cloned from main):

[greg@greg-spc-arch build]$ bat ../CMakeLists.txt ../main.cc
───────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: ../CMakeLists.txt
───────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ cmake_minimum_required( VERSION 3.28 )
   2   │
   3   │ project( TestRefl )
   4   │
   5   │ set( CMAKE_CXX_STANDARD 20 )
   6   │ add_subdirectory( reflect-cpp )
   7   │ set( REFLECTCPP_JSON ON )
   8   │
   9   │ add_executable( main main.cc )
  10   │ target_link_libraries( main PRIVATE reflectcpp )
───────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
───────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: ../main.cc
───────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ #include <rfl/json.hpp>
   2   │ #include <rfl.hpp>
   3   │
   4   │ #include <iostream>
   5   │
   6   │ struct S { int a; float b; };
   7   │
   8   │ int main() {
   9   │     S s{};
  10   │     std::cout << rfl::json::write(s) << '\n';
  11   │ }
───────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[greg@greg-spc-arch build]$ cmake .. && make
-- The C compiler identification is GNU 13.2.1
-- The CXX compiler identification is GNU 13.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done (1.5s)
-- Generating done (0.0s)
-- Build files have been written to: /home/greg/tmp/reproduce/build
[ 25%] Building C object reflect-cpp/CMakeFiles/reflectcpp.dir/src/yyjson.c.o
[ 50%] Linking C static library libreflectcpp.a
[ 50%] Built target reflectcpp
[ 75%] Building CXX object CMakeFiles/main.dir/main.cc.o
In file included from /home/greg/tmp/reproduce/reflect-cpp/include/rfl/parsing/Parser_default.hpp:13,
                 from /home/greg/tmp/reproduce/reflect-cpp/include/rfl/parsing/Parser.hpp:7,
                 from /home/greg/tmp/reproduce/reflect-cpp/include/rfl/json/Parser.hpp:6,
                 from /home/greg/tmp/reproduce/reflect-cpp/include/rfl/json.hpp:4,
                 from /home/greg/tmp/reproduce/main.cc:1:
/home/greg/tmp/reproduce/reflect-cpp/include/rfl/internal/is_attribute.hpp:20:20: error: ‘Attribute’ was not declared in this scope; did you mean ‘is_attribute’?
   20 | class is_attribute<Attribute<Type>> : public std::true_type {};
      |                    ^~~~~~~~~
      |                    is_attribute
/home/greg/tmp/reproduce/reflect-cpp/include/rfl/internal/is_attribute.hpp:20:34: error: template argument 1 is invalid
   20 | class is_attribute<Attribute<Type>> : public std::true_type {};
      |                                  ^~
/home/greg/tmp/reproduce/reflect-cpp/include/rfl/internal/is_attribute.hpp:23:20: error: ‘Attribute’ was not declared in this scope; did you mean ‘is_attribute’?
   23 | class is_attribute<Attribute<Type>*> : public std::true_type {};
      |                    ^~~~~~~~~
      |                    is_attribute
/home/greg/tmp/reproduce/reflect-cpp/include/rfl/internal/is_attribute.hpp:23:34: error: template argument 1 is invalid
   23 | class is_attribute<Attribute<Type>*> : public std::true_type {};
      |                                  ^
/home/greg/tmp/reproduce/reflect-cpp/include/rfl/internal/is_attribute.hpp:23:36: error: expected unqualified-id before ‘>’ token
   23 | class is_attribute<Attribute<Type>*> : public std::true_type {};
      |                                    ^
/home/greg/tmp/reproduce/reflect-cpp/include/rfl/internal/is_attribute.hpp:26:40: error: ISO C++ forbids declaration of ‘type name’ with no type [-fpermissive]
   26 | class is_attribute<const Attribute<Type>*> : public std::true_type {};
      |                                        ^
/home/greg/tmp/reproduce/reflect-cpp/include/rfl/internal/is_attribute.hpp:26:42: error: template argument 1 is invalid
   26 | class is_attribute<const Attribute<Type>*> : public std::true_type {};
      |                                          ^
In file included from /home/greg/tmp/reproduce/reflect-cpp/include/rfl/parsing/Parser_named_tuple.hpp:4,
                 from /home/greg/tmp/reproduce/reflect-cpp/include/rfl/parsing/Parser.hpp:9:
/home/greg/tmp/reproduce/reflect-cpp/include/rfl/parsing/NamedTupleParser.hpp:214:27: error: ‘Memoization’ in namespace ‘rfl::internal’ does not name a template type
  214 |   static inline internal::Memoization<
      |                           ^~~~~~~~~~~
/home/greg/tmp/reproduce/reflect-cpp/include/rfl/parsing/NamedTupleParser.hpp: In static member function ‘static const auto& rfl::parsing::NamedTupleParser<R, W, _ignore_empty_containers, FieldTypes>::field_indices()’:
/home/greg/tmp/reproduce/reflect-cpp/include/rfl/parsing/NamedTupleParser.hpp:177:12: error: ‘field_indices_’ was not declared in this scope; did you mean ‘field_indices’?
  177 |     return field_indices_.value(set_field_indices<0>);
      |            ^~~~~~~~~~~~~~
      |            field_indices
make[2]: *** [CMakeFiles/main.dir/build.make:76: CMakeFiles/main.dir/main.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:100: CMakeFiles/main.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

The Reader and Writer have been designed with various JSON libraries in mind. Whereas the design for Reader seems fine, the Writer needs an update.

The problem is that many libraries for binary formats require you to directly assign children to their parents as opposed to first creating the children and then assigning them to their parents.

For instance, pugixml requires you to do this:

auto node_child = node_parent.append_child(_key.c_str());

Likewise, libbson (https://mongoc.org/libbson/current/creating.html):

bson_append_array_builder_begin (&parent, "foo", 3, &bab);
bson_array_builder_append_int32 (bab, 9);
bson_array_builder_append_int32 (bab, 8);
bson_array_builder_append_int32 (bab, 7);
bson_append_array_builder_end (&parent, bab);

The current workaround it to have structures like OutputArray and OutputObject (https://github.com/getml/reflect-cpp/blob/f/xml/include/rfl/flexbuf/OutputArray.hpp). This works well, but it adds an unnecessary extra layer and runtime overhead.

This could be avoided by a redesign of the writer that takes these limitations into account.

The library claims to support reflective programming in general, but found that I had to do a bit more work to really make it convenient to iterate over fields of a struct with type information and field names, e.g.

template <typename T, typename F>
void for_each_field(T&& value, F&& f) {
  std::apply(
      [&f]<typename... Fields>(Fields&&... fields) {
        ((f.template operator()<std::remove_cvref_t<Fields>::name_>(
             *std::forward<Fields>(fields)())),
         ...);
      },
      rfl::to_view(value).fields());
}

// example usage
auto print_ints = 
  []<auto FieldName, typename T>(const T& x) {
    if constexpr(std::is_same_v<T, int>) {
      std::cout << FieldName.string_view() << ": " << x << "\n";
    }
  };

struct foo { int x; float y; int z; };
for_each_field(foo{ 1, 2.0f, 3 }, print_ints);

Did I miss something that would have made this easier? Do you think helper functions like this would make sense to include in the library, or maybe in example documentation?

I'm also not sure if I missed a better way to get the name of an rfl::Field as a compile-time value, I had to use the internal name_ member above.

When rfl is being used on a struct in an anonymous namespace,
gcc complains at linkage stage with errors like:

/usr/bin/ld: src/.../tests.cpp.o: warning: relocation against `_ZN3rfl8internal11fake_objectIN12_GLOBAL__N_13ABCEEE' in read-only section `.text'
/usr/bin/ld: src/.../tests.cpp.o: in function `auto rfl::internal::get_field_names<(anonymous namespace)::DEF>()':
/home/...//include/rfl/internal/../internal/get_field_names.hpp:120:(.text+0x2950): undefined reference to `rfl::internal::fake_object<(anonymous namespace)::DEF>'
/usr/bin/ld: /home/.../include/rfl/internal/../internal/get_field_names.hpp:120:(.text+0x295b): undefined reference to `rfl::internal::fake_object<(anonymous namespace)::DEF>'
/usr/bin/ld: src/.../tests.cpp.o: in function `auto rfl::internal::get_field_names<(anonymous namespace)::ABC>()':
/home/.../include/rfl/internal/../internal/get_field_names.hpp:120:(.text+0x33fb): undefined reference to `rfl::internal::fake_object<(anonymous namespace)::ABC>'
/usr/bin/ld: /home/.../include/rfl/internal/../internal/get_field_names.hpp:120:(.text+0x3406): undefined reference to `rfl::internal::fake_object<(anonymous namespace)::ABC>'
/usr/bin/ld: /home/.../include/rfl/internal/../internal/get_field_names.hpp:120:(.text+0x3411): undefined reference to `rfl::internal::fake_object<(anonymous namespace)::ABC>'
/usr/bin/ld: /home/.../include/rfl/internal/../internal/get_field_names.hpp:120:(.text+0x341c): undefined reference to `rfl::internal::fake_object<(anonymous namespace)::ABC>'
/usr/bin/ld: /home/.../include/rfl/internal/../internal/get_field_names.hpp:120:(.text+0x3427): undefined reference to `rfl::internal::fake_object<(anonymous namespace)::ABC>'
/usr/bin/ld: src/.../tests.cpp.o:/home/.../include/rfl/internal/../internal/get_field_names.hpp:120: more undefined references to `rfl::internal::fake_object<(anonymous namespace)::ABC>' follow
/usr/bin/ld: warning: creating DT_TEXTREL in a PIE
collect2: error: ld returned 1 exit status

and clang throws erros at compilation time:

/home/.../include/rfl/internal/fake_object.hpp:9:10: error: variable 'rfl::internal::fake_object<(anonymous namespace)::DEF>' is used but not defined in this translation unit, and cannot be defined in any other translation unit because its type does not have linkage

It would be nice to add support for compile-time regular expressions for the following reasons as it would

• Allow for compile-time checking of validation regexes (pretty much impossible to push invalid regexes if the code doesn't compile)
• Quicker runtime validation of fields (since the regex is processed at compile-time)
• Allow for compile-time validation whenever we (eventually) support compile-time parsing of JSON

This last point seems like science-fiction, or like something that wouldn't ever be useful, but with the addition of #embed to C23 and its likely addition to C++ at some point (hopefully by C++26) it wouldn't be crazy to expect users to want to configure programs using inline / embedded JSON or other formats. There are already a few options for compile-time JSON parsing, so it's not unheard of, in which case, validation of JSON fields would be necessary too.

For all this, I propose (and will be working on a PR for) the addition of Hanickadot's CTRE as a dependency, as well as tweaks in the validator code to get rid of the extremely slow std::regex. If you want to see the speed differences, please check P1433R0, which received a very positive reaction from the C++ Standard's committee (hopefully we'll have a constexpr capable option for regex in the standard library soon).

Cap'n'proto is a C++ binary serialization format (https://github.com/capnproto/capnproto) similar to protobuf.

It also includes a format for dynamic reflection:

https://capnproto.org/cxx.html#dynamic-reflection

We can fill this gap by implementing a reflect-cpp interface to cap'n'proto.

If you are looking to a good first issue where you could easily add value to the library, you can always just add more Regex patterns:

https://github.com/getml/reflect-cpp/blob/main/include/rfl/patterns.hpp

One of the major goals of reflect-cpp is to make injection attacks harder by validating string patterns upfront using the Regex pattern validator (https://github.com/getml/reflect-cpp/blob/main/include/rfl/PatternValidator.hpp).

However, we don't have too many patterns yet.

If you can come up with good ideas for new patterns, write them in patterns.hpp, add some tests in https://github.com/getml/reflect-cpp/tree/main/tests/json and there is a very high likelihood that your PR will be merged.

MsgPack (https://msgpack.org/) is a JSON-like binary format that claims to be faster and smaller than JSON, because it is a binary.

The canonical implementation is in C and there is an official C++ interface (https://github.com/msgpack/msgpack-c/tree/c_master).

To me, this seems like a great contribution. It is a very popular serialization format that is supported by many languages. I think people would like to see a reflection-based interface for this.

Generally speaking, any serialization format can be integrated by implementing the IsReader and IsWriter concepts, which are documented here: https://github.com/getml/reflect-cpp/blob/main/docs/supporting_your_own_format.md. The way I see it, it should be fairly straightforward to implement these concepts for msgpack-c. (msgpack-cpp seems a bit harder.)

You should implement tests for this, in the same way the tests are set up for flexbuffers - by providing a Dockerfile that contains a reproducible environment in which the tests can be compiled and run:

https://github.com/getml/reflect-cpp/tree/main/tests/flexbuffers

So far we have only supported data types that every single serialization format can support as well. However, there are certain datatypes that can only be supported by certain formats that we should support as well.

For instance:

• Binary formats like CBOR, BSON, etc also support bytestrings. A common format for byte strings would be a good idea.
• Some binary formats also have specific types for datetime and timestamps.
• BSON also supports OID types.

Most includes seem to use an absolute path (starting with "rfl/...") with some exceptions I found:

• from_named_tuple.hpp
• fields.hpp
• Validator.hpp
• Timestamp.hpp
• PatternValidator.hpp
• Literal.hpp

which have some relative paths defined as well (might wanna unify that).

However I think this makes it a bit harder to include the library just for some quick testing as you have to specify an include directory (which you'd probably do for production anyways though ofc).

For me I like to try out new (header-only) libraries by just git init and git submodule add https://github.com/getml/reflect-cpp libs/reflect-cpp and using it in a simple main.cpp with #include "libs/reflect-cpp/include/rfl.hpp", running clang/gcc for compilation directly.
This doesn't work with these absolute paths though as:

rfl.hpp includes rfl/AllOf.hpp (fine so far) which then includes rfl/Result.hpp which is at the same level, as we are already in the rfl directory. So the include trying to go into rfl a level deeper doesn't work unless I add libs/reflect-cpp/include/ as an include directory (compiling with g++ -std=c++20 -I ./libs/reflect-cpp/include/ main.cpp), so it can always go "back" into the include directories root rfl-directory while with relative paths it should just work right out of the box without that.

also the described way to

Simply copy the contents of the folder include into your source repository OR add it to your include path.

doesn't quite work due to this (unless the 'OR' is meant to be an 'and' instead).

Not a big deal of course and relative paths have their cons while absolute paths have their pros, so you might wanna consider changing it but it's fine if you don't.

btw., that's a cool and useful library, definitely gonna consider using it for future projects. :)

As I see you use structured binding for getting a fields list, but I guess it doesn't work with inheritance. For example:

 struct Characteristics {
     rfl::Field<"name", std::string> name;
     rfl::Field<"surname", std::string> surname;
 };
 struct Bart : Characteristics {};

 const Bart bart = {"Bart", "Simpson"};
 auto& [f1, f2] = bart;

Is a correct code and should work with structured binding, but I can't compile it with the write_and_read function call:
rfl/internal/to_ptr_tuple.hpp(44): error C3448: the number of identifiers must match the number of array elements or members in a structured binding declaration

At the same time, this code doesn't work with structured binding from the box:

  struct Homer : Characteristics {
      rfl::Field<"work", std::string> company_name;
  };
  const Homer homer = {"Homer", "Simpson", "Station"};
  [[maybe_unused]] auto& [f1, f2, f3] = homer; // here is an error

Do you have any plans to fix it? Inheritane is not the best invention but it is useful.

While rfl::json::write(data) is super useful in many situations, if the type of the structs provided to this functions are important to the parser of the resulting json, the type names are lost. Same issue for other formats which do not have a type specified per aggregation.

Feature request:
Provide an option for this function and similar for other formats, to request to the implementation to inject an additional member to every "struct" being read, which will have a name like "type" (or "typename") and a value being the name of the type being reflected, and this recursively for all members. Ideally, such feature would also allow to specify what would be the name of that "type" member.

I am not sure if you are familiar with the magic_enum library, but this is the behavior I would expect. Some extension if magic_enum is detected would also be fine.

std::string color_name{"GREEN"};
auto color = magic_enum::enum_cast<Color>(color_name);
if (color.has_value()) {
  // color.value() -> Color::GREEN
}

The library could use XML support by interfacing libxml2 or pugixml (https://github.com/zeux/pugixml). There aren't many good reflection based XML parsers out there for and XML is still relevant in some domains, so there is high pay-off here. On the other hand, interfacing XML is a bit more difficult than most other formats just because of how it is designed.

Generally speaking, another format can be integrated by implementing the IsReader and IsWriter concepts, which are documented here: https://github.com/getml/reflect-cpp/blob/main/docs/supporting_your_own_format.md

There are a couple of things to keep in mind for XML in particular that you do not have to deal with when it comes to most other formats:

1. Unlike most other formats, XML objects require names. This can be accomplished either by extracting the type by combining get_type_name (https://github.com/getml/reflect-cpp/blob/main/include/rfl/internal/get_type_name.hpp) and remove_namespaces (https://github.com/getml/reflect-cpp/blob/main/include/rfl/internal/remove_namespaces.hpp). Alternatively, the user can define a literal Name on the struct. If the user defines a Name on the struct, that should take precedence. The equivalent should be the Tag in TaggedUnion which works exactly the same way (https://github.com/getml/reflect-cpp/blob/main/docs/variants_and_tagged_unions.md).

2. Behaviour should be equivalent to https://pydantic-xml.readthedocs.io/en/stable/pages/quickstart.html

To implement the requirements laid out in 2. it is probably necessary to have a custom implementation for rfl::parsing::Parser<XMLReader, XMLWriter, NamedTuple>. (For most other formats, you wouldn't have to do that.)

One feature we are currently missing is the automatic generation of a JSON schema. Please refer to the JSON schema specification for details:

https://json-schema.org/specification

The envisioned syntax is like this:

const std::string json_schema = rfl::json::to_schema<MyStruct>();

The function does not have to be consteval or constexpr. Runtime is fine.

Also, there doesn't have to be a reader. Maybe, in the future, we will be crazy enough to generate named tuples (https://github.com/getml/reflect-cpp/blob/main/docs/named_tuple.md) from JSON schema strings at compile time (which should be possible, in theory), but at the moment I would be happy if the syntax outlined above would work.

Please note that reflect-cpp contains validators like the Regex validators, numeric validators and AnyOf, AllOf, OneOf. It also supports Literals and Enums. These are deliberately designed with the JSON schema in mind and should be supported by the JSON schema as well. Please refer to the JSON schema specification for details. Also take a look at the reflect-cpp documentation to get a better understanding of these types (https://github.com/getml/reflect-cpp/tree/main/docs, particularly section 2)

Good morning!

Sorry for another thing to bother you over, but I was wondering how possible it would be to use the BSON converter to convert between a bsoncxx::value to a struct/class. In our project we are storing data inside of a Mongo database using the mongocxx driver, which depends upon libbson.

When we get a document via the driver we get a bsoncxx::view or bsoncxx::value back, and I noticed when reading through the docs that it's possible to implement a custom from_bson method. I understand that the BSON library here is using the C driver, but I was wondering if it would be possible somehow to convert between the bsoncxx:value we get and the reflect-cpp interface in order to serialize the class into our desired type.

There are somethings on the class that exist that I doubt reflect-cpp has any context over, like bsoncxx::oid _id (which most documents in the db have). I find the mongocxx docs are not the best so was hoping you might have some insight if this is possible or potential steps to try and implement it - I know given its a different library/dependency it cannot be implemented into this library itself, but maybe you have an idea of what needs to be done on our side. I imagine this is not a unique issue ^^

Hello! I have been converting a project that previously used nlohmann::json and its serialization to use this project. It's mostly been okay, but I've noticed there is no support for std::wstring. I know that wstrings are often a pain, so this isn't too unexpected, but previously, with nlohmann, there was a way to extend the parser to support wstring types by instructing the parser to automatically convert them to std::string.

Example:

namespace nlohmann
{
    template <>
    struct adl_serializer<std::wstring>
    {
            static void to_json(json& j, const std::wstring& str) { j = StringUtils::wstos(str); }

            static void from_json(const json& j, std::wstring& str) { str = StringUtils::stows(j.get<std::string>()); }
    };
}

In that example, we simply do the conversion from std::string to std::string and visa versa. I noticed while trawling through the docs that we can extend types in reflect-cpp by doing something similar:

namespace rfl {
namespace parsing {

template <class ReaderType, class WriterType>
struct Parser<ReaderType, WriterType, Person>
    : public CustomParser<ReaderType, WriterType, Person, PersonImpl> {};

}
} 

The main problem that I can see here, is that in the example of a string, if we setup a custom struct that mirrors a string and do the conversion, it's not going to transform it into a string correctly, but rather into a struct/object that has the internal fields and whatnot.

Is there a way to properly handle structs that have std::wstrings or other unsupported 'primative' types?

Hi
Great work!
Is it possible to use smart pointers on classes to be serialise?
Is it possible to serialise private and protected attributes?
Structs are public by default, is a public constructor necessary?
Thanks
Dom

In addition to the tests we currently run, it would be good idea to add extensive benchmarking using Google Benchmark.

https://github.com/google/benchmark

This would serve two purposes:

1. Quantify and continually monitor the overhead of the reflection API over manual solutions.
2. Compare the performance of the different serialization formats.

Recent benchmarks have indicated that the overhead mentioned in 1 is negligible, but a more formal and continuous monitoring would be very helpful.

Hey there! I was happy to learn about a library for reflection in C++. On C++ Developers LinkedIn group, you asked for constructive criticism, so here are a couple of bits of it:)

First of all, I think that for a shared library, one could make use of link-time optimisation [see: GCC Wiki on LTO]. For that, it's better to add yyjson as a dependency and then link it to the library.

Also, it is a slight disturbance and may seem picky, but why lowercase and uppercase starts in the names of the included files are mixed? It may make sense in languages like GoLang, but in C++, it's just confusing, IMHO.

I'll dive deeper in the following weeks to give more criticism on the code. But overall, it looks like an incredible tool!

I get the following error when trying to build the tests using clang 16.0.6 on MacOs 14.2.1. main, v0.6.0, v0.5.0, v0.4.0 and v0.3.0 all exhibit the same issue.

Am I missing something in the configuration or compiler? The details of the clang invocation are included below.

In file included from reflect-cpp/tests/json/test_all_of.cpp:4:
In file included from reflect-cpp/include/rfl/json.hpp:4:
In file included from reflect-cpp/include/rfl/json/Parser.hpp:6:
In file included from reflect-cpp/include/rfl/json/../parsing/Parser.hpp:8:
In file included from reflect-cpp/include/rfl/json/../parsing/Parser_default.hpp:10:
In file included from reflect-cpp/include/rfl/json/../parsing/../internal/enums/StringConverter.hpp:11:
reflect-cpp/include/rfl/json/../parsing/../internal/enums/get_enum_names.hpp:136:51: error: 
      non-type template argument is not a constant expression
  using EmptyNames = Names<EnumType, rfl::Literal<"">, 0>;

/opt/local/bin/clang++-mp-16  -Ireflect-cpp/include -std=c++20 -Wall -Wno-sign-compare -Wno-missing-braces -Wno-psabi -pthread -fno-strict-aliasing -fwrapv -O2 -ftemplate-backtrace-limit=0 -fsanitize=undefined -O3 -DNDEBUG -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk -std=gnu++20 -MD -MT tests/json/CMakeFiles/reflect-cpp-json-tests.dir/test_anonymous_fields.cpp.o -MF CMakeFiles/reflect-cpp-json-tests.dir/test_anonymous_fields.cpp.o.d -o CMakeFiles/reflect-cpp-json-tests.dir/test_anonymous_fields.cpp.o -c reflect-cpp/tests/json/test_anonymous_fields.cpp

Hi again! Just wondering if it's possible to have the JSON config be transformed into a pretty string or at least one that is more human readable. I wanted to generate some default configuration files, but them being minified is a bit of a problem.

Hi,

Is there a way to use reflect-cpp with projects that only c++17 compatible?

BSON is a binary format similar to JSON (as the name implies). The standard implementation is written in C: https://mongoc.org/libbson/current/index.html

BSON is used by many NoSQL databases, including MongoDB. It would be nice to be able to interface NoSQL-Databases in a type-safe way and this is why it could be very rewarding to add BSON support in reflect-cpp.

Generally speaking, any serialization format can be integrated by implementing the IsReader and IsWriter concepts, which are documented here: https://github.com/getml/reflect-cpp/blob/main/docs/supporting_your_own_format.md. The way I see it, it should be fairly straightforward to implement these concepts for libbson.

You should implement tests for this, in the same way the tests are set up for flexbuffers - by providing a Dockerfile that contains a reproducible environment in which the tests can be compiled and run:

https://github.com/getml/reflect-cpp/tree/main/tests/flexbuffers

Is this boilerplate actually necessary?

rfl::Field<"children", std::vector<s32>> mChildren = std::vector<s32>();

reflect-cpp could use YAML support. YAML is often used to write configurations files and it would be nice to read those in with reflect-cpp.

There is a Reddit user who has developed this YAML library: https://github.com/fktn-k/fkYAML. The code quality seems very high, the documentation is great, even though the project is in its early stages. He is very enthusiastic about the idea of us using this. Of course, there are other libraries out there, so the first task should be to find a good library to interface.

Generally speaking, any serialization format can be integrated by implementing the IsReader and IsWriter concepts, which are documented here: https://github.com/getml/reflect-cpp/blob/main/docs/supporting_your_own_format.md. Since there are several YAML libraries out there, you should only pick libraries for which it is easy to implement these interfaces. Other criteria are code quality and speed.

You should implement tests for this, in the same way the tests are set up for flexbuffers - by providing a Dockerfile that contains a reproducible environment in which the tests can be compiled and run:

https://github.com/getml/reflect-cpp/tree/main/tests/flexbuffers

Since string literals have static storage duration, is the copy here required? It should be enough o simply initialize a string view with pointer and size.

The following program fails to link on g++ 13.2.0 x86-64 Linux (Ubuntu):

#include <rfl/internal/get_field_names.hpp>
struct A { int i; };
int main() { rfl::internal::get_field_names<A>(); }

error:

/tmp/ccKMlHON.o:a.cpp:function auto get_field_names<A>():(.text._Z15get_field_namesI1AEDav+0x5): error: undefined reference to 'fake_object<A>'
collect2: error: ld returned 1 exit status

might be related to #15
reduced: https://godbolt.org/z/1jecWY947

possible workaround is to use [[gnu::no_sanitize_undefined]]

We should provide this library on vcpkg, so it can be installed from there.

The following snippet fails to compile under clang-cl version 17.0.6 and Visual Studio 17.8.5 with /std:c++latest.

#include <rfl.hpp>

struct test {
  bool x;
  bool y;
};

void f() {
  for (const auto& f : rfl::fields<test>()) {}
}

The same snippet compiles successfully when building with the microsoft compiler, or if I remove one of the fields in the struct.

The compilation error seems to be some sort of static_assert about duplicate string literals:

In file included from tools/test.cc:1:
In file included from external/_main~_repo_rules~reflect_cpp/include\rfl.hpp:12:
In file included from external/_main~_repo_rules~reflect_cpp/include/rfl/Attribute.hpp:10:
external/_main~_repo_rules~reflect_cpp/include/rfl/Literal.hpp(295,17): error: static assertion failed due to requirement '!has_duplicates()': Duplicate strings are not allowed in a Literal.
  295 |   static_assert(!has_duplicates(),
      |                 ^~~~~~~~~~~~~~~~~
external/_main~_repo_rules~reflect_cpp/include/rfl/internal/../internal/get_field_names.hpp(97,15): note: in instantiation of template class 'rfl::Literal<internal::StringLiteral<22>{{{103, 101, 116, 95, 102, 105, 101, 108, 100, 95, ...}}}, internal::StringLiteral<22>{{{103, 101, 116, 95, 102, 105, 101, 108, 100, 95, ...}}}>' requested here
   97 |   return rfl::Literal<_names1..., _names2...>::template from_value<0>();
      |               ^
external/_main~_repo_rules~reflect_cpp/include/rfl/internal/../internal/get_field_names.hpp(105,12): note: in instantiation of function template specialization 'rfl::internal::concat_two_literals<internal::StringLiteral<22>{{{103, 101, 116, 95, 102, 105, 101, 108, 100, 95, ...}}}, internal::StringLiteral<22>{{{103, 101, 116, 95, 102, 105, 101, 108, 100, 95, ...}}}>' requested here
  105 |     return concat_two_literals(_head, concat_literals(_tail...));
      |            ^
external/_main~_repo_rules~reflect_cpp/include/rfl/internal/../internal/get_field_names.hpp(127,14): note: in instantiation of function template specialization 'rfl::internal::concat_literals<rfl::Literal<internal::StringLiteral<22>{{{103, 101, 116, 95, 102, 105, 101, 108, 100, 95, ...}}}>, rfl::Literal<internal::StringLiteral<22>{{{103, 101, 116, 95, 102, 105, 101, 108, 100, 95, ...}}}>>' requested here
  127 |       return concat_literals(
      |              ^
external/_main~_repo_rules~reflect_cpp/include/rfl/internal/../internal/get_field_names.hpp(138,12): note: in instantiation of function template specialization 'rfl::internal::get_field_names()::(anonymous class)::operator()<0ULL, 1ULL>' requested here
  138 |     return get(std::make_index_sequence<num_fields<T>>());
      |            ^
external/_main~_repo_rules~reflect_cpp/include/rfl/internal/../field_names_t.hpp(14,24): note: in instantiation of function template specialization 'rfl::internal::get_field_names<test>' requested here
   14 |     decltype(internal::get_field_names<std::remove_cvref_t<T>>)>::type;
      |                        ^
external/_main~_repo_rules~reflect_cpp/include/rfl/internal/../internal/to_ptr_named_tuple.hpp(68,29): note: in instantiation of template type alias 'field_names_t' requested here
   68 |     using FieldNames = rfl::field_names_t<T>;
      |                             ^
external/_main~_repo_rules~reflect_cpp/include/rfl/internal/../internal/ptr_named_tuple_t.hpp(16,42): note: in instantiation of function template specialization 'rfl::internal::to_ptr_named_tuple<test>' requested here
   16 |     typename std::invoke_result<decltype(to_ptr_named_tuple<T>), T>::type;
      |                                          ^
external/_main~_repo_rules~reflect_cpp/include/rfl/internal/../named_tuple_t.hpp(39,1): note: in instantiation of template type alias 'ptr_named_tuple_t' requested here
   39 | using named_tuple_t =
      | ^
external/_main~_repo_rules~reflect_cpp/include/rfl/fields.hpp(12,36): note: in instantiation of template type alias 'named_tuple_t' requested here
   12 |   return internal::get_meta_fields<named_tuple_t<T>>();
      |                                    ^
tools/test.cc(9,29): note: in instantiation of function template specialization 'rfl::fields<test>' requested here
    9 |   for (const auto& f : rfl::fields<test>()) {}
      |

Currently C-style arrays are not supported. The following example will trigger compile errors:

#include <iostream>

#include "rfl.hpp"
#include "rfl/json.hpp"

struct A {
 int a[3];
};

int main() {
  A a = {1, 2, 3};

  std::cout << rfl::json::write(a) << "\n";  // Expect: {"a":[1,2,3]}
}

The reason is that rfl::internal::has_n_fields gives wrong results for struct A.

C-style arrays are not allowed to be initialized by copy or move. In such cases, Any{} will be converted as the element type of the C-style array due to brace elision of aggregate initialization (just like the example above, a can be initialized with single pair of braces). As a result, has_n_fields<A, 3> is true.

This can be solved by a little more work when calculating the fields of aggregate types. I've written a demo with randomly generated tests to validate my thoughts.

In addition to the more complex formats, we would also like to support tabular formats like parquet or CSV. But currently, we don't even have an interface and concepts for that. These will have to be simplified version of our current parsing module.

It appears adding additional constructors/casting operators is problematic?

struct DrawMatrix {
  struct MatrixWeight {
    u32 boneId;
    f32 weight;

    MatrixWeight() : boneId(-1), weight(0.0f) {}
    MatrixWeight(u32 b, f32 w) : boneId(b), weight(w) {}

    bool operator==(const MatrixWeight& rhs) const = default;
  };

  std::vector<MatrixWeight> mWeights; // 1 weight -> singlebound, no envelope

  bool operator==(const DrawMatrix& rhs) const = default;
};
struct JSONMatrixWeight {
  rfl::Field<"bone_id", u32> bone_id = 0;
  rfl::Field<"weight", f32> weight = 1.0f;

  JSONMatrixWeight() = default;
  JSONMatrixWeight(const DrawMatrix::MatrixWeight& original)
      : bone_id(original.boneId), weight(original.weight) {}

  operator DrawMatrix::MatrixWeight() const {
    return DrawMatrix::MatrixWeight(bone_id(), weight());
  }
};

struct JSONDrawMatrix {
  rfl::Field<"weights", std::vector<JSONMatrixWeight>> mWeights =
      std::vector<JSONMatrixWeight>();

  JSONDrawMatrix() = default;
  JSONDrawMatrix(const DrawMatrix& original) {
    for (const auto& weight : original.mWeights) {
      mWeights.value().emplace_back(weight);
    }
  }

  operator DrawMatrix() const {
    DrawMatrix result;
    for (const auto& jsonWeight : mWeights()) {
      result.mWeights.push_back(
          static_cast<DrawMatrix::MatrixWeight>(jsonWeight));
    }
    return result;
  }
};

Error log

  In file included from RiiStudio\source\librii\g3d\io\JSON.cpp:3:
  In file included from RiiStudio\source\librii\..\vendor\rfl.hpp:12:
  In file included from RiiStudio\source\librii\..\vendor/rfl/as.hpp:4:
  In file included from RiiStudio\source\librii\..\vendor\rfl/from_named_tuple.hpp:7:
  In file included from RiiStudio\source\librii\..\vendor\rfl/internal/field_tuple_t.hpp:8:
RiiStudio\source\vendor\rfl\internal\to_field_tuple.hpp(1089,9): error : static assertion failed due to requirement 'rfl::always_false_v<librii::g3d::JSONDrawMatrix>': Only up to 100 fields are supported.
          static_assert(rfl::always_false_v<T>,
          ^             ~~~~~~~~~~~~~~~~~~~~~~
  RiiStudio\source\vendor\rfl\to_named_tuple.hpp(45,38): note: in instantiation of function template specialization 'rfl::internal::to_field_tuple<librii::g3d::JSONDrawMatrix>' requested here
          auto field_tuple = internal::to_field_tuple(_t);
                                       ^
  RiiStudio\source\vendor\rfl\parsing\Parser.hpp(96,38): note: in instantiation of function template specialization 'rfl::to_named_tuple<librii::g3d::JSONDrawMatrix>' requested here
              const auto named_tuple = to_named_tuple(_var);
                                       ^
  RiiStudio\source\vendor\rfl\parsing\Parser.hpp(847,66): note: in instantiation of member function 'rfl::parsing::Parser<rfl::json::Reader, rfl::json::Writer, librii::g3d::JSONDrawMatrix>::write' requested here
                  Parser<ReaderType, WriterType, std::decay_t<T>>::write(_w, *it),
                                                                   ^
  RiiStudio\source\vendor\rfl\parsing\Parser.hpp(294,69): note: in instantiation of member function 'rfl::parsing::VectorParser<rfl::json::Reader, rfl::json::Writer, std::vector<librii::g3d::JSONDrawMatrix>>::write' requested here
              auto value = Parser<ReaderType, WriterType, ValueType>::write(

Clang 16.0.5 on Windows 11 x64

:\Users\rii\Documents\dev\RiiStudio\source\vendor\rfl\internal\to_field_tuple.hpp(58,15): error : type 'const librii::g3d::JSONModel' decomposes into 3 elements, but only 1 name was provided
          auto& [f1] = _t;
                ^
  C:\Users\rii\Documents\dev\RiiStudio\source\vendor\rfl\to_named_tuple.hpp(45,38): note: in instantiation of function template specialization 'rfl::internal::to_field_tuple<librii::g3d::JSONModel>' requested here
          auto field_tuple = internal::to_field_tuple(_t);
                                       ^
  C:\Users\rii\Documents\dev\RiiStudio\source\vendor\rfl\parsing\Parser.hpp(95,38): note: in instantiation of function template specialization 'rfl::to_named_tuple<librii::g3d::JSONModel>' requested here
              const auto named_tuple = to_named_tuple(_var);
                                       ^
  C:\Users\rii\Documents\dev\RiiStudio\source\vendor\rfl\json\write.hpp(18,38): note: in instantiation of member function 'rfl::parsing::Parser<rfl::json::Reader, rfl::json::Writer, librii::g3d::JSONModel>::write' requested here
      const auto json_obj = Parser<T>::write(w, _obj);
                                       ^
  C:\Users\rii\Documents\dev\RiiStudio\source\librii\g3d\io\JSON.cpp(111,30): note: in instantiation of function template specialization 'rfl::json::write<librii::g3d::JSONModel>' requested here
    std::string s = rfl::json::write(homer);

In this case, this was the code

struct JSONModel {
 rfl::Field<"name", std::string> name;
 rfl::Field<"info", JSONModelInfo> info = JSONModelInfo();
 // rfl::Field<"bones", std::vector<BoneData>> bones;
 // rfl::Field<"positions", std::vector<PositionBuffer>> positions;
 // rfl::Field<"normals", std::vector<NormalBuffer>> normals;
 // rfl::Field<"colors", std::vector<ColorBuffer>> colors;
 // rfl::Field<"texcoords", std::vector<TextureCoordinateBuffer>> texcoords;
 // rfl::Field<"materials", std::vector<G3dMaterialData>> materials;
 // rfl::Field<"meshes", std::vector<PolygonData>> meshes;
 rfl::Field<"matrices", std::vector<JSONDrawMatrix>> matrices =
     std::vector<JSONDrawMatrix>();

 static JSONModel make(const Model& original) {
   JSONModel model{.name = "bruh"};
   model.name = original.name;
   model.info = original.info;
   for (const auto& matrix : original.matrices) {
     model.matrices.value().emplace_back(matrix);
   }
   return model;
 }

 operator Model() const {
   Model result;
   result.name = name();
   result.info = info();
   for (const auto& jsonMatrix : matrices()) {
     result.matrices.push_back(static_cast<DrawMatrix>(jsonMatrix));
   }
   return result;
 }
};
void test() {
 JSONModel homer{.name = "bruh"};
 std::string s = rfl::json::write(homer);
}

Similarly having name be default set causes issues too:

struct JSONModel {
  rfl::Field<"name", std::string> name = std::string();
  rfl::Field<"info", JSONModelInfo> info = JSONModelInfo();
  // rfl::Field<"bones", std::vector<BoneData>> bones;
  // rfl::Field<"positions", std::vector<PositionBuffer>> positions;
  // rfl::Field<"normals", std::vector<NormalBuffer>> normals;
  // rfl::Field<"colors", std::vector<ColorBuffer>> colors;
  // rfl::Field<"texcoords", std::vector<TextureCoordinateBuffer>> texcoords;
  // rfl::Field<"materials", std::vector<G3dMaterialData>> materials;
  // rfl::Field<"meshes", std::vector<PolygonData>> meshes;
  rfl::Field<"matrices", std::vector<JSONDrawMatrix>> matrices =
      std::vector<JSONDrawMatrix>();

  static JSONModel make(const Model& original) {
    JSONModel model;
    model.name = original.name;
    model.info = original.info;
    for (const auto& matrix : original.matrices) {
      model.matrices.value().emplace_back(matrix);
    }
    return model;
  }

  operator Model() const {
    Model result;
    result.name = name();
    result.info = info();
    for (const auto& jsonMatrix : matrices()) {
      result.matrices.push_back(static_cast<DrawMatrix>(jsonMatrix));
    }
    return result;
  }
};

void test() {
  JSONModel homer;
  std::string s = rfl::json::write(homer);
}

s is written as "{}" since it is assumed to have zero fields given it is itself default-constructible.

Compiler: Clang 16.0.5
Target: x64 Windows
Host: Windows 11 x64

It should be really simple, sth like

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
    name = "refl_main",
    hdrs = glob([
        "include/**/*.h",
        "include/*.h",
    ]),
    srcs = [
      "src/yyjson.c",
    ],
    includes = ["include"],
    visibility = ["//visibility:public"],
)

The basic requirement is as follows. Suppose you had an enum like this:

enum class WindowsState: uint32
{
  maximised = uint32(1)<<0,
  borderless = uint32(1)<<1,
  topmost = uint32(1)<<2
};

People then expect to able to do this:

rfl::json::write(WindowsState::borderless | WindowsState::topmost);

And get the following string:

"borderless|topmost"

Likewise, we should be able to read the string using rfl::json::read and reproduce the original value.

There is an agreement that it is reasonable to restrict flag enums to values that are 1 or multiples of 2 (for technical reasons that are explained below, some restrictions are necessary).

For example:

enum class WindowsState: uint32
{
  maximised = uint32(1)<<0,
  borderless = uint32(1)<<1,
  topmost = uint32(1)<<2,
  borderless_and_topmost = (uint32(1)<<1) | (uint32(1)<<2)
};

rfl::json::write(WindowsState::borderless_and_topmost);

This would also result in the following string:

"borderless|topmost"

And not:

"borderless_and_topmost"

People are OK with that.

Open question

The remaining question is how to communicate to the compiler that something needs to be treated as a flag.

Possibility 1 is this syntax. The user would have to place that somewhere in the code:

rfl::is_flag_enum<WindowsState>: std::true_type{};

Possibility 2 is not to force the user to do anything and just have the library try both options.

magic_enum appears to be using possibility 1:

https://github.com/Neargye/magic_enum/blob/master/include/magic_enum/magic_enum.hpp

And much like the ideas proposed in here, they restrict themselves to 1 or multiples of 2.

Implementation details

In order to implement this, you would have to understand how serializing enums works in the first place.

The most important file is this one:

https://github.com/getml/reflect-cpp/blob/main/include/rfl/internal/enums/get_enum_names.hpp

You would also have to understand what rfl::Literal is:

https://github.com/getml/reflect-cpp/blob/main/docs/literals.md

There are two problems we have to solve:

1. Given an enum MyEnum{ option1, option2, ...}, how do we figure out how many and which options there are?

2. Given an enum value MyEnum::option1, how do we get the name "option1" as a string?

Problem 1 is solved by brute-force iteration. This is what is happening in get_enum_names. If the underlying type of the enum is fixed (like it is for all scoped enums), then you can always call static_cast<MyEnum>(some_integer) and this behaviour is defined. If some_integer matches option1 in the enum, then static_cast<MyEnum>(some_integer) is equivalent to having MyEnum::option1. This brute-force iteration takes place at compile time. This is the main reason there needs to be some kind of limit on what the enum values can be.

Basically it works like this: We iterate through the integers at compile time get the string representation of static_cast<MyEnum>(i). Based on that string representation, we can decide whether this is a proper enum option or not. If it is a proper enum option, it is added to rfl::Literal and our std::array which contains the enums.

This is what get_enum_names does.

Problem 2 is solved in get_enum_name_str_view, which returns a std::string_view of the enum_name. This works by employing std::source_location::current().function_name() and passing the enum value as a template parameter to the function. It will then show up in func_name and all we have to do is get it from func_name.

get_enum_name_str_lit just transforms the string view into our rfl::internal::StringLiteral, which we need to pass through rfl::Literal.

All we have to do is to rewrite get_enum_names() such that it doesn't iterate through 0,1,2,3,... but instead it iterates through 1,2,4,8,16,.... the ranges should be determined based on the bit size of the underlying fixed type.

How ChatGPT explains the code

Here is how ChatGPT explains the code in https://github.com/getml/reflect-cpp/blob/main/include/rfl/internal/enums/get_enum_names.hpp:

This code appears to be a part of a C++ library (RFL) focused on handling reflection for enums. Let's break it down:

• The code defines utilities within nested namespaces rfl::internal::enums.
• get_enum_name_str_view() generates a std::string_view of the name of an enum value based on the passed enum.
• get_enum_name() uses the previous function to generate a compile-time string literal representing the name of the enum value.
• start_value initializes an empty array of EnumType named enums_.
• get_enum_names() generates an array of enum values and their corresponding names for a given enum type.

The key points in the code involve compile-time reflection for enums. It iterates through possible enum values (_i) up to a maximum value. For each value, it tries to get the name using get_enum_name<static_cast<EnumType>(_i)>(). If the name starts with '(', it's considered an empty string and iteration continues; otherwise, it creates a new array of enum values and names.

The conditions in get_enum_names() are ensuring that the enum type is a scoped enum (using class or struct) and that its underlying type is integral. If these conditions aren't met, it will trigger static assertions.

The code essentially aims to create a compile-time list of enum values and their corresponding names, assuming the enums follow certain conventions (scoped, integral underlying type). This can be useful for various purposes like serialization, logging, or any situation where you'd need enum names alongside their values.

This issue is a long-standing TODO list towards enabling compile-time serialization and deserialization (the latter is of which particular importance).

Why do we need compile-time (de)serialization?

The idea is not new. json2cpp by Jason Turner (the guy from C++ Weekly, guess where I got the title for this issue from) already kind of provides this functionality.

The reasoning is simple. One might want to make compile-time decisions based on some kind of JSON configuration instead of having to set a bunch of #defines or whatnot. With C23 having added #embed and C++ probably doing so at some point, this would be even cooler as the JSON/whatever serial format wouldn't even have to live as crust in a header file, it could just be pulled from the filesystem from a normal .json file. The possibilities are endless.

Lastly, I propose the following counterpoint: why not?. For years people have been making cool constexpr capable projects (CTRE by Hanickadot being an excellent example). The work you can do at compile-time is the work that's guaranteed not to incur in undefined behavior (constexpr code is not allowed to have undefined behavior). If only for the assurance of safety of the library on its own, since we could have compile-time tests rather than runtime tests, this is excellent.

It is also generally quicker if you manage to precompute import parts of programs. Do we really need to read that JSON file that's supposed never to be modified by the user at the start of our program anyway? Doesn't it kind of suck that they can modify it too? I wish we could just embed this into our program in an easy way...

We might not know all of the uses this could have a priori, but that doesn't mean we shouldn't go ahead with this (especially given there is already the excellent use case described above). This is a no-brainer.

TODO

What follows is a list of tasks to be done in this regard. New points will be appended to it as time goes on and we figure out more stuff that needs to be done before the final objective is done. PRs or commits that check particular boxes should be added here too for tracking's sake.

• constexpr-friendly rfl::Result and rfl::Error
• Compile-time serialization and deserialization

Hi everyone, I'm working with classes like the following:

class Person {
public:
  int age() const;
private:
  int age_;
}

in which all fields are private but can be accessed by getter. This type of class is very common, e.g. Protobuf messages.

Is there any way to get the public method list from a class, so that I can access the fields in such classes?

Could any one help me? Thanks in advance!

TinyCBOR appears to be a well-documented and well-designed C library for CBOR written by Intel.

https://github.com/intel/tinycbor