Code Monkey home page Code Monkey logo

oz-sgl's Introduction

Standard Generic Library (SGL) for Pascal

Why this project appeared

In the process of porting code from C ++ to Delphi, very often have to port code using STL collections. The set of collections offered by Delphi is very ascetic and therefore sometimes
it is difficult to find a suitable replacement. Sometimes you come across the code where the object is placed in the stack or uses its own memory allocator.

I really dislike the suggested data refresh mode. To update the data, I need to retrieve the value placed in it from the collection, update the value and then put the changed value back. This requires at least two additional copy operations. We cannot pass a data item as a var parameter to a procedure.

There is no way for a collection to change how memory is allocated. Memory for objects and records is allocated from one shared heap. After use, the memory must be carefully returned to the heap. Freeing memory correctly is not always a trivial task, and it takes both processor time and the programmer's time to write this code. In STL, you can specify your own memory allocator for all kinds of collections.

This implementation relies on records and pointers. So far, I see no way to implement what I want using standard objects. The creation and destruction of objects uses a shared heap of memory. There is no way to place objects on the call stack. The good old "object" is declared deprecated and adding new features for this type is not supported.

Typed region-based memory management

This collection implementation relies on the mechanism memory management based on typed memory regions. The use of typed memory regions makes it possible to simplify the solution of a number of tasks:

  • Memory release code.

The task of freeing memory becomes easier and can be done much faster.

  • Parallel programming.

It is a well-known fact that a standard memory manager must be thread-safe. Only one thread can access the memory manager at any given time. Allocating and freeing memory uses mutual exclusion mechanisms and is not a fast operation, especially if the memory is heavily defragmented. When using a separate typed memory region, we refer to the standard memory manager only at the moment of increasing the required memory and deleting the structure after its use.

Standard data structures

Support for basic structures with the ability to specify a memory allocator. The elements of the list are accessed through pointers. As a rule, the memory for values is located in the so-called segmented memory region, which will not be moved during operation. If it is necessary to increase the memory of a region, an additional memory segment is allocated for it. This means that we can access data items located in such a region through a pointer.

For arrays, we use the so-called contiguous memory region. Data items are accessed through an index. If necessary, increase the memory of the region, a segment with a large size is allocated for it and data from the current memory segment is copied to the new segment. After copying the data, the old segment will be deleted.

Ultimately, working through pointers is very convenient and efficient. The code becomes much simpler and more concise. However, if you have no experience with pointers, it is easy to "shoot yourself in the foot". For fans of encapsulation, you can aggregate the desired structure as a private field. Next, we open only the necessary part of the interface by overriding the required methods and properties in the public section. If we put the inline option, we avoid additional costs. The Delphi compiler will not generate code for overridden methods. At the place of the method call, there will be a direct call to the aggregate structure method.

Generic lists and dictionaries

  • TsgTuple<T1, ...> Generic Tuples
  • TsgArray<T> Generic Array with memory allocation from a shared memory region
  • TsgList<T> Generic List of Values
  • TsgRecordList<T> Generic List of Values accessed by pointer
  • TsgLinkedList<T> Generic Bidirectional Linked List
  • TsgForwardList<T> Generic Unidirectional Linked List
  • TsgHashMap<Key, T> Generic Unordered dictionary
  • TsgMap<Key, T> Generic Ordered Dictionary based on 2-3 tree
  • TsgSet<Key> Generic Set based on 2-3 trees

Untyped data structures

  • TsgPointerArray Untyped List of Pointers
  • TsgPointerList Untyped List of Values accessed by pointer
  • TCustomLinkedList Untyped Bidirectional Linked List
  • TsgCustomTree Untyped Dictionary based on 2-3 trees

Iterators

We've started adding Delphi iterators. Now we can use the construction for p in List do; The most interesting thing is that we use record to implement the iterator and it works! Compared to using objects, the generated code is much more efficient and, which is nice, no calls to the heap, the variable for the iterator is located on the stack. This turned out to be quite a pleasant surprise for me!

Memory allocator

The ability to specify a memory allocator also means that we work mainly with records. Typically, some structure uses one or more memory regions, which is a simple memory manager. After using the structure, we have the opportunity return all the memory occupied by freeing up the memory region. We have limitations using inheritance. In some cases, we can replace inheritance with aggregation and helpers. Typically for implementing collections, this is not a problem. Using records allows collections to be stacked. This is sometimes very convenient.

Object pool

An object pool allows you to manage the reuse of structures when creating objects is memory intensive or when a limited number of objects of a certain type can be created.

If we stop using something, it is not always worth deleting the structure or object. Often times the object has to be re-created.

oz-sgl's People

Contributors

edwinyzh avatar marat1961 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

oz-sgl's Issues

Hashmap Iterator accesses the data element via address.

If we have access to the data element via a hashmap iterator via an address, then we can change the key or value, which means that the hashmap will be broken. Subsequent search may become incorrect.
Therefore the iterator should not allow to change the value.

Add functionality to a typed memory region.

  1. Flag Owned
  2. RangeCheck flag
  3. Action to remove an element from the collection:
  • clear the item value;
  • clear the item value and allow reuse;
  • hold the item value.
    Depending on the flag set, we use
    suitable removal method (FreeAndClear, Remove)
  1. Notification flag (when inserting, when deleting)
  2. Methods for an array
    Get the address of an object by its index
    GetPtr = function (Index: Integer): Pointer;

Write library documentation on wiki.

What should be in the documentation.

  1. Library structure.
  2. General description of the library.
  3. For each collection there must be
  • detailed description;
  • data organization structure;
  • in which case it is worth using;
  • examples of using.

Develop a memory manager test that covers all possible cases.

  1. Creates heap defragmentation.
  2. Return to the heap in which the blocks will be combined from below.
  3. Return to the heap in which the blocks will be combined from above.
  4. Return to the heap in which the blocks will be both above and below.
  5. A situation is created when one more block is required
  6. Tests for checking calls with invalid data:
  • The address is not from the heap;
  • The address is not aligned on the border of two pointers;
  • Invalid memory size specified;
  • Multiple attempts to return the memory block.

Implement a repository for string values.

String repository

Managed data types have a lot of overhead.
This also applies to string type.
I am already coming to the idea that in some places it would be useful to use some kind of repository for strings.
Then we can use PChar instead of a string.

Storing strings in a dictionary is probably a good idea.
Finding a string in a hashmap can be done efficiently.
Most commonly used strings are immutable values.

Typically, strings are used to describe metadata, such as field names, class names, and enumeration values.
When designing a user interface, we deal with a lot of labels.

For example, when we pass data to json, most of this data is field names.
If we use Google protocol buffer, integer encoding without leading zeros is used to encode the fields.

Compression algorithms come with a lot of overhead.

Consider, for example, passing tabular data to display a report.
String data is column names, display styles, width, alignment method, and the values ​​themselves,
which should be for each cell in the table.
It is possible to identify a repeated set of string data for a specific report type with each transmission.
So we can tell you the dataset for report # 9.
Some fields will have a very limited set of values ​​based on the enumerated type.

Permanent string repository.

The immutable part of the data, which may be in each transmission, must be declared as a part of a specific data format and transmitted to the client side once.

Variable string repository.

We pass the modified part as field values ​​or in a separate part of the message.
It is also possible to combine this piece of data with the applicable data and national language encoding.

Create an add-on over the memory region for storing tuples.

Then we could assume that we have a typed region that contains an array of data described by some tuple.
An easy interface for accessing an array of values would be extremely useful.

var 
  meta: TsgTupleMeta;
  rgn: PMemoryRegion;
  tuple1, tuple2: TsgTuple;
  ptr: Pointer;
  pv: PVector;
  ps: PChar;
  pi: PInteger;
begin  
  meta.MakeTrio<TVector, string, Integer>(nil, True);  
  rgn := meta.MakeTupleRegion;
  tuple1 := rgn[5];
  tuple2 := rgn[7];
  tuple1.Assign(rgn[7]);  
  ptr := tuple.tie(0);
  pi := tuple.tie<TVector>(0);
  ps := tuple.tie<string>(1);
  pv := tuple.tie<Integer>(2);
end;

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.