Hi sir,

I am making a system that storage some structs in hash tables(with uthash) I'd like to put it into files - using tpl.

But, until now I can't make it work.

Do you have any sample to show me how to do this? I believe that are complementary projects and I really want to use them.

Regards,
Fabiano

Dear developer,

If I serialize an int on a 64 bit machine, an deserialize the tpl on a machine whose ints are 32 bits, what would happen ?

Another related question: if I serialize an int whose value is 5 on the 64 bit machine, does the tpl image allocates 8 bytes for it (plus any metadata it needs) or does it optimize it (e.g. by allocating only 1 byte). I'm asking this because we have big size restrictions on our embedded system.

Thanks

I have been used tpl in some projects before, and recently, I need(have) to dump 'float' type data. I checked the source codes of tpl, and found it was not hard to add 'float' support. Why was it not supported in the beginning ? Or I missed some reason ? Thanks very much!

Hi!
I have a problem when compiling the lib with MinGW in Windows (no cygwin):
I build it either as a static lib or a dll (with TPL_EXPORTS defined)
In both ones, I have linker errors:
undefined reference to '_imp__tpl_xxx' (xxx = all tpl functions)

I can't figure where is the problem. Thanks :)

Dear author,
What additions or changes should I make to your project if I need to modify it to implement the data serialization scheme in the Windows kernel?
I look forward to your suggestions.
Have a good time

Dear developer,

I need some help regarding the tpl_gather.

We would like to serialize structures of different types through the network. As I understand, the tpl_gather is able to get one tpl at a time (regardless of its type).

Now when the callback is called, the doc says "The callback can do anything with the tpl image but it must not free it.". That's where my understanding stops.

Does that mean that the next step will be a top_load() of the in-memory image argument img ? If so we have a serious problem since we don't know its format string: tpl_load expects a previous call to tpl_map() with the format string.

How do you handle that ?

Thanks

This is the runtime error msg.

DTS_MSG: Stensal C/C++ DTS detected a fatal program error!
DTS_MSG: Continuing the execution will cause unexpected behaviors, abort!
DTS_MSG: Reading 4 bytes at 0xffffced8 will read undefined values.
DTS_MSG: Diagnostic information:

• The object to-be-read (start:0xffffcecc, size:12 bytes) is allocated at

• file:/home/sbuilder/workspace/tpl/tests/test35.c::6, 18
  

• 0xffffcecc 0xffffced7
• +------------------------+
• | the object to-be-read |......
• +------------------------+

•                        ^~~~~~~~~~
  

•    the read starts at 0xffffced8 that is right after the object end.
  

• Stack trace (most recent call first):
  -[1] file:/home/sbuilder/workspace/tpl/tests/test35.c::10, 38
  -[2] file:/musl-1.1.10/src/env/__libc_start_main.c::168, 11

./bootstrap
./configure --prefix=/some/path

does not work it returns:
config.status: error: cannot find input file: `src/win/Makefile.in'

Now this file contains
libtpl_la_LDFLAGS = -no-undefined -version-info 0:0:0
which leads to the problem that built library file does not contain version info.
I would like to suggest to place here correct version numbers like
libtpl_la_LDFLAGS = -no-undefined -version-info 1:6:1
Thanks!

I use TPL to serialize structures between different processes. Both 32 and 64 bits are allowed. One of the structures contains a string (pointer) and a pointer to something else. That something else is irrelevant in other processes but has to remain in the structure definition. It is part of public API and cannot be changed.

Since that pointer was supposed to be NULL, I did a dirty trick: I used 's' for that field. Everything worked like a cham... until today, when it was not NULL.

I would like to have a 'p' that would just ignore and jump over a uintptr_t on serialization, and generates a NULL on deserialization.

Do you see any problems with this? Do you think this feature should be implemented? If this is straight forward to add, it would be great if you could help me with this. If you don't have the time, I think I will try to dig into the library and add the new type myself.

When comiling tpl for 64 bit environment, a lot of warnings c4242/c4267 are reported.

This should be fixed, due to pedantic CI will end in errors!

warning c4242: '...': conversion from 'uintptr_t' to 'int'  possible loss of data
warning c4267 '...' conversion from 'size_t' to 'unsigned int' possible loss of data

Affected tpl version: 1.61

Hello;

Thank you for documenting your source code enough so a neophyte could use it. And clear testing. I can't tell you how much time protobuf-c cost me. It is appreciated that you took the time.
I normally update things when I find them and learn them, I improved your autoconf to include library version and some other changes. There might be some tweaking needed for windows, I only tried it on ubuntu. And win/malloc.c and realloc.c might have been toasted in the dist changes. You can uprev the library now.
libtpl-1.4.tar.gz
I don't want to generate a change to your source unless you want the changes. Attached is your tpl-master with update autoconfigure options .

Thanks again,

Dave

TPL is used in a custom library As Source and is compiled along with TPL to .a archive using gcc in MSYS2/MinGW64 environment.

Now this .a file is linked in a Visual C++ Console Project (executable) and the compilation results in the following error

libcustom.a(tpl.o) : error LNK2019: unresolved external symbol __getreent referenced in function tpl_oops
libcustom.a(tpl.o) : error LNK2019: unresolved external symbol __errno referenced in function tpl_gather_blocking

Any idea which other external library should be linked ?

typedef struct DataGram {
    char* data;
    char header;
};

    struct DataGram temp;
    temp.header = 'm';
    temp.data = "ppp";

    char* buffert;
    size_t i, length;
    tpl_node *tn;
    struct DataGram s1;



   printf( "input map{%s}\n", temp.data);

    tn = tpl_map( "S(sc)", &temp );  /* pass structure address */
    tpl_pack( tn, 0 );
    tpl_dump( tn, TPL_MEM,&buffert, &length  );//&buffert, &length 
    tpl_free( tn );


    printf( "size = %lu\n", length );


     for(;;)
     {
        sendto(sockfd, buffert, length,0,(struct sockaddr *)&cliaddr,sizeof(cliaddr));
     }

length=recvfrom(sockfd,buffert,10000,0,NULL,NULL);
        printf("i got a message,client got %d bytes \n",length);
        tn = tpl_map( "S(sc)", &s1 );
        tpl_load( tn, TPL_MEM, buffert, length );
        tpl_unpack( tn, 0 );
        tpl_free( tn );
        //free( buffert );

        char header = 'p';
        printf("data %s", s1.data);

If we only have one value in our struct then we are able to unpack it at the client side.
But if we use two values in our struct then we get -1 bytes sent. why?

For us Windows users it would be useful to be able to serialize the wide char type and wide char strings. For example, the wchar_t and wchar_t* types. Thanks.

Hi Troy,
I have this linked list:

typedef struct list_t {
    char* directory;
    char *fsName;
    char* album;
    char* title;
    char* artist;
    struct list_t *next;
    struct list_t *prev;
} list_t;

and I succeded in serializing it (I think) but it fails to load every time.
This is the code:

void serialize (list_t *listDB)
{
    tpl_node *tn;
    list_t *i, tmp;

    tn = tpl_map ("A(S(sssss))", &tmp);
    for (i = listDB; i != NULL; i = i->next) {
        tmp = *i;
        tpl_pack (tn, 1);
    }
    tpl_dump (tn, TPL_FILE, "list.tpl");
    tpl_free (tn);
}

list_t *deserialize ()
{
    tpl_node *tn;
    list_t *listDB, tmp, *new;

    tn = tpl_map ("(A(S(sssss))", &tmp);
    tpl_load (tn, TPL_FILE, "list.tpl");
    /*tpl_unpack ( fin*/
    listDB = malloc (sizeof (list_t));
    *listDB = tmp;

    while (tpl_unpack (tn, 1) > 0) {
        new = malloc (sizeof (list_t));
        *new = tmp;
        listDB = link (listDB, new);
    }

    return listDB;
}

It gives me: failed to parse (A(S(sssss))

How can I solve this?

Code to reproduce:

struct Test tests[] = {{"first", 1}, {"second", 2},{"third", 3}};
tpl_node *tn;
tn = tpl_map("S(si)#", &tests, 3);
tpl_pack(tn, 0);
tpl_dump(tn, TPL_MEM|TPL_PREALLOCD, buffer, 5000);
tpl_free(tn);

This will leak 2 strings: second and third. Basically for cases like this, only the string from the first array is released.

The problem is that in tpl_free, for case TPL_TYPE_POUND, data is just removed, and number of structures in array is lost.

Is it possible to dynamical unpack data?
I have a struct like "id:unkown struct". Which means first i need to know the id, then i can now what type of data next and provide corresponding format string.
Is it possible?

Hello,

I got too much warnings: passing 'const char *' to parameter of type 'char *' discards qualifiers

when using "tpl_map"

I did this patch, but I'd rather use an official version of Tpl, with no local patch. it was tested: ok.

$ git diff

diff --git a/src/tpl.c b/src/tpl.c
index e4f9665..753bf0a 100644
--- a/src/tpl.c
+++ b/src/tpl.c
@@ -309,7 +309,7 @@ char *calc_field_addr(tpl_node *parent, int type,char *struct_addr, int ordinal)
     return struct_addr + offset;
 }

-TPL_API tpl_node *tpl_map(char *fmt,...) {
+TPL_API tpl_node *tpl_map(const char *fmt,...) {
   va_list ap;
   tpl_node *tn;

@@ -319,10 +319,11 @@ TPL_API tpl_node *tpl_map(char *fmt,...) {
   return tn;
 }

-TPL_API tpl_node *tpl_map_va(char *fmt, va_list ap) {
+TPL_API tpl_node *tpl_map_va(const char *fmt, va_list ap) {
     int lparen_level=0,expect_lparen=0,t=0,in_structure=0,ordinal=0;
     int in_nested_structure=0;
-    char *c, *peek, *struct_addr=NULL, *struct_next;
+    const char *c, *peek;
+    char *struct_addr=NULL, *struct_next;
     tpl_node *root,*parent,*n=NULL,*preceding,*iter_start_node=NULL,
              *struct_widest_node=NULL, *np; tpl_pidx *pidx;
     tpl_pound_data *pd;
diff --git a/src/tpl.h b/src/tpl.h
index 1da93c4..3067c73 100644
--- a/src/tpl.h
+++ b/src/tpl.h
@@ -116,7 +116,7 @@ typedef struct tpl_gather_t {
 typedef int (tpl_gather_cb)(void *img, size_t sz, void *data);

 /* Prototypes */
-TPL_API tpl_node *tpl_map(char *fmt,...);       /* define tpl using format */
+TPL_API tpl_node *tpl_map(const char *fmt,...);       /* define tpl using format */
 TPL_API void tpl_free(tpl_node *r);             /* free a tpl map */
 TPL_API int tpl_pack(tpl_node *r, int i);       /* pack the n'th packable */
 TPL_API int tpl_unpack(tpl_node *r, int i);     /* unpack the n'th packable */
@@ -127,7 +127,7 @@ TPL_API char* tpl_peek(int mode, ...);         /* sneak peek at format string */
 TPL_API int tpl_gather( int mode, ...);        /* non-blocking image gather */
 TPL_API int tpl_jot(int mode, ...);            /* quick write a simple tpl */

-TPL_API tpl_node *tpl_map_va(char *fmt, va_list ap);
+TPL_API tpl_node *tpl_map_va(const char *fmt, va_list ap);

 #if defined __cplusplus
     }

(END)

I'm trying to get the header length for a particular tpl format string. Here's my code.

#include <iostream>
#include <tpl.h>

size_t header_size(char *_buffer, int _buffer_size) {
	std::cout << "_buffer_size: " << _buffer_size << '\n';
	tpl_bin tb;

	std::cout << "before tpl_map\n";
	tpl_node *tn = tpl_map("B",&tb);

	tb.sz = _buffer_size;
	tb.addr = _buffer;

	std::cout << "_buffer_size: " << _buffer_size << '\n';

	std::cout << "before tpl_pack\n";
	tpl_pack(tn,0);

	char tmp[_buffer_size];
	ssize_t tpl_size = 0;

	std::cout << "_buffer_size: " << _buffer_size << '\n';
	std::cout << "before tpl_dump\n";
	int i = tpl_dump(tn,TPL_MEM,tmp,&tpl_size);

	std::cout << "tpl_size: " << tpl_size << '\n';
	std::cout << "_buffer_size: " << _buffer_size << '\n';

	std::cout << "before tpl_free\n";
	tpl_free(tn);
	std::cout << "before return\n";

	return tpl_size - _buffer_size;

}

int main() {
	char buffer[100] = "abcdefg";
	std::cout << header_size(&buffer[0],sizeof(buffer)) << '\n';
	std::cout << "--------------------------------------------------\n";
	std::cout << header_size(nullptr,0) << '\n';
	std::cout << "--------------------------------------------------\n";
	std::cout << header_size(&buffer[0],0) << '\n';
	std::cout << "--------------------------------------------------\n";
	std::cout << header_size(&buffer[0],1) << '\n';
	std::cout << "--------------------------------------------------\n";
	std::cout << header_size(&buffer[0],3) << '\n';
	std::cout << "--------------------------------------------------\n";
	std::cout << header_size(&buffer[0],9) << '\n';
}

For the case of buffer size 0, the _buffer_size variable in the function gets altered after the tpl_dump function is called but only when _buffer_size is 0. Here's the code output on my machine.

_buffer_size: 100
before tpl_map
_buffer_size: 100
before tpl_pack
_buffer_size: 100
before tpl_dump
tpl_size: 114
_buffer_size: 100
before tpl_free
before return
14
--------------------------------------------------
_buffer_size: 0
before tpl_map
_buffer_size: 0
before tpl_pack
_buffer_size: 0
before tpl_dump
tpl_size: 14
_buffer_size: 21858
before tpl_free
before return
18446744073709529772
--------------------------------------------------
_buffer_size: 0
before tpl_map
_buffer_size: 0
before tpl_pack
_buffer_size: 0
before tpl_dump
tpl_size: 14
_buffer_size: 21858
before tpl_free
before return
18446744073709529772
--------------------------------------------------
_buffer_size: 1
before tpl_map
_buffer_size: 1
before tpl_pack
_buffer_size: 1
before tpl_dump
tpl_size: 15
_buffer_size: 1
before tpl_free
before return
14
--------------------------------------------------
_buffer_size: 3
before tpl_map
_buffer_size: 3
before tpl_pack
_buffer_size: 3
before tpl_dump
tpl_size: 17
_buffer_size: 3
before tpl_free
before return
14
--------------------------------------------------
_buffer_size: 9
before tpl_map
_buffer_size: 9
before tpl_pack
_buffer_size: 9
before tpl_dump
tpl_size: 23
_buffer_size: 9
before tpl_free
before return
14

Thanks for the help.

when i first time use tpl,it build errors like:undefined reference to `_imp__tpl_pack',and then i found

ifdef _WIN32

#ifdef TPL_EXPORTS //(1)here is the probrom

define TPL_API __declspec(dllexport)

else /* */

ifdef TPL_NOLIB //(2)and here

define TPL_API

else

define TPL_API __declspec(dllimport)

endif /* TPL_NOLIB */

endif /* TPL_EXPORTS*/

else

define TPL_API

endif

and then i change the ifdef with ifndef,then the probrom solve,i do'n know why or that is true?

I'd be happy to write a Mac Homebrew formula (package) for tpl. To do this, I'd need some sort of release tag though. Would you mind tagging a v0.1.0 or v1.0.0 release or such?

How to serialize complex structures?
https://stackoverflow.com/questions/59463793/how-to-save-nested-c-struct-data-to-disk

https://github.com/php/php-src/

I pack std::string object to tpl_map like this:
std::string content = readFile();
char carray = const_cast<char>(content.c_str());
tpl_node *tn = tpl_map("s", carray);
tpl_pack(tn, 0);
tpl_dump(tn, TPL_FILE, "player.dat");
tpl_free(tn);

but it doesn't work.
throw this exception:

how can i do it?