vtereshkov / xdpw Goto Github PK

XD Pascal: A small embeddable self-hosting Pascal compiler for Windows. Supports Go-style methods and interfaces

License: BSD 2-Clause "Simplified" License

Batchfile 0.26% Pascal 99.74%

pascal-language pascal-compiler delphi free-pascal xd-pascal self-hosting windows win32 winapi compiler-design compiler-construction compiler parser x86 x86-32 go golang programming-language game-development raylib

xdpw's Introduction

XD Pascal for Windows

Dedicated to my father Mikhail Tereshkov, who instilled in me a taste for engineering

Summary

XD Pascal is a small embeddable self-hosting compiler for a Pascal language dialect. Any comments, suggestions, or bug reports are appreciated. Feel free to contact the author on GitHub or by e-mail [email protected]. Enjoy.

Features

Go-style methods and interfaces
Native x86 code generation (32 bit Windows executables)
Support for both console and GUI applications
No external assembler or linker needed
Floating-point arithmetic using the x87 FPU
Integration with the Raylib game development library
Integration with Geany IDE
Compiler source for Delphi 6/7, Free Pascal and XD Pascal itself (Delphi 2009+ migration is straightforward)

Detailed description

Usage

Type in the command prompt:

xdpw <file.pas>

The source file should be specified with its extension (.pas).

Language

XD Pascal is similar to Delphi 6/7 and Free Pascal with the following changes:

Enhancements

The compiler is self-hosting
The compiler is extremely compact (~10000 lines) and can be easily embedded into larger systems
Go-style methods and interfaces are supported

Differences

Strings are null-terminated arrays of characters (C style), but indexed from 1 for Pascal compatibility
The Text type is equivalent to file. It can be used for both text and untyped files
Method calls and procedural variable calls require parentheses even for empty parameter lists

Limitations

No classical (C++/Delphi style) object-oriented programming
No visual components
Units cannot be compiled separately
Only peephole optimizations
Extended is equivalent to Double
No High and Low functions for open arrays. Open array length should be explicitly passed to a subroutine
Statement labels cannot be numerical

Formal grammar

ProgramOrUnit = [("program" | "unit") Ident ";"] 
                ["interface"] [UsesClause] Block "." .
                
UsesClause = "uses" Ident {"," Ident} ";" .                

Block = { Declarations } (CompoundStatement | "end") .

Declarations = DeclarationSection ["implementation" DeclarationSection] .

DeclarationSection = LabelDeclarations |
                     ConstDeclarations | 
                     TypeDeclarations |
                     VarDeclarations |
                     ProcFuncDeclarations .
                     
Initializer = ConstExpression |
              StringLiteral |
              "(" Initializer {"," Initializer} ")" |
              "(" Ident ":" Initializer {";" Ident ":" Initializer} ")" |
              SetConstructor .                     
               
LabelDeclarations = "label" Ident {"," Ident} ";"               
             
ConstDeclarations = (UntypedConstDeclaration | TypedConstDeclaration)
               {";" (UntypedConstDeclaration | TypedConstDeclaration)} .

UntypedConstDeclaration = "const" Ident "=" ConstExpression .
                                 
TypedConstDeclaration = "const" Ident ":" Type "=" Initializer .

TypeDeclarations = "type" Ident "=" Type ";" {Ident "=" Type ";"} .

VarDeclarations = "var" IdentList ":" Type ["=" Initializer] ";" 
                       {IdentList ":" Type ["=" Initializer] ";"} .

ProcFuncDeclarations = ("procedure" | "function") Ident 
                       [Receiver] [FormalParams] [":" TypeIdent] 
                       [CallModifier] ";" [(Directive | Block) ";"] .

Receiver = "for" Ident ":" TypeIdent .

CallModifier = "stdcall" | "cdecl" .

Directive = "forward" | "external" ConstExpression .         

ActualParams = "(" [ (Expression | Designator) |
                {"," (Expression | Designator)} ] ")" .

FormalParams = "(" FormalParamList {";" FormalParamList} ")" .
              
FormalParamList = ["const" | "var"] IdentList [":" ["array" "of"] TypeIdent] 
                                              ["=" ConstExpression] .             

IdentList = Ident {"," Ident} .

Type = "(" Ident {"," Ident} ")" |
       "^" TypeIdent |
       ["packed"] "array" "[" Type {"," Type} "]" "of" Type |
       ["packed"] "record" Fields "end" |
       ["packed"] "interface" FixedFields "end" |
       ["packed"] "set" "of" Type |
       ["packed"] "string" [ "[" ConstExpression "]" ] |
       ["packed"] "file" ["of" Type] |
       ConstExpression ".." ConstExpression |
       ("procedure" | "function") [FormalParams] [":" TypeIdent] [CallModifier] |
       Ident .
       
Fields = FixedFields 
           ["case" [Ident ":"] Type "of" 
               ConstExpression {"," ConstExpression} ":" "(" Fields ")"
          {";" ConstExpression {"," ConstExpression} ":" "(" Fields ")"}] [";"] .       
       
FixedFields = IdentList ":" Type {";" IdentList ":" Type} .       
       
TypeIdent = "string" | "file" | Ident .       

Designator = BasicDesignator {Selector} .

BasicDesignator = Ident |
                  Ident [ActualParams] |
                  Ident "(" Expression ")" .

Selector = "^" | 
           "[" Expression {"," Expression} "]" | 
           "." Ident | 
           "(" ActualParams ")".

Statement = [Label ":"] [ (Designator | Ident) ":=" Expression | 
                          (Designator | Ident) [ActualParams] {Selector} |
                          CompoundStatement |
                          IfStatement |
                          CaseStatement |
                          WhileStatement |
                          RepeatStatement | 
                          ForStatement |
                          GotoStatement |
                          WithStatement ] .
                          
Label = Ident .                          

StatementList = Statement {";" Statement} .

CompoundStatement = "begin" StatementList "end" .

IfStatement = "if" Expression "then" Statement ["else" Statement] .

CaseStatement = "case" Expression "of" CaseElement {";" CaseElement} 
                    [";"] ["else" StatementList] [";"] "end" .
                    
WhileStatement = "while" Expression "do" Statement .

RepeatStatement = "repeat" StatementList "until" Expression .

ForStatement = "for" Ident ":=" Expression ("to" | "downto") Expression "do" Statement.

GotoStatement = "goto" Label .

WithStatement = "with" Designator {"," Designator} "do" Statement .                    
 
CaseElement = CaseLabel {"," CaseLabel} ":" Statement .

CaseLabel = ConstExpression [".." ConstExpression] .

ConstExpression = Expression .

Expression = SimpleExpression [("="|"<>"|"<"|"<="|">"|">="|"in") SimpleExpression] .

SimpleExpression = ["+"|"-"] Term {("+"|"-"|"or"|"xor") Term}.

Term = Factor {("*"|"/"|"div"|"mod"|"shl"|"shr"|"and") Factor}.

Factor = (Designator | Ident) [ActualParams] {Selector} |
         Designator |
         "@" Designator | 
         Number | 
         CharLiteral |
         StringLiteral |  
         "(" Expression ")" | 
         "not" Factor |
         SetConstructor |
         "nil" |
         Ident "(" Expression ")" {Selector} .
         
SetConstructor = "[" [Expression [".." Expression] 
                     {"," Expression [".." Expression]}] "]" .         

Ident = (Letter | "_") {Letter | "_" | Digit}.

Number = "$" HexDigit {HexDigit} | 
         Digit {Digit} ["." {Digit}] ["e" ["+" | "-"] Digit {Digit}] .

CharLiteral = "'" (Character | "'" "'") "'" | 
              "#" Number .

StringLiteral = "'" {Character | "'" "'"} "'".

Compiler

The compiler is based on a recursive descent parser. It directly builds a Windows PE executable without using any external assembler or linker.

Directives

$APPTYPE - Set application type. Examples: {$APPTYPE GUI}, {$APPTYPE CONSOLE}
$UNITPATH - Set additional unit search path. Example: {$UNITPATH ..\units\}

Optimizations

Some simple peephole optimizations are performed:

Push/pop elimination
FPU push/pop elimination
Local variable loading optimizations
Array element access optimizations
Record field access optimizations
Assignment optimizations
Comparison optimizations
Condition testing optimizations

Inlined procedures and functions

The following identifiers are implemented as part of the compiler. Their names are not reserved words and can be locally redefined by the user.

procedure Inc(var x: Integer);
procedure Dec(var x: Integer);
procedure Read([var F: file;] var x1 {; var xi});
procedure Write([var F: file;] x1[:w[:d]] {; xi[:w[:d]]});
procedure ReadLn([var F: file;] var x1 {; var xi});
procedure WriteLn([var F: file;] x1[:w[:d]] {; xi[:w[:d]]});
procedure New(var P: Pointer);
procedure Dispose(var P: Pointer);
procedure Break;
procedure Continue;
procedure Exit;
procedure Halt[(const error: Integer)];
function SizeOf(var x | T): Integer;
function Ord(x: T): Integer;
function Chr(x: Integer): Char;
function Low(var x: T | T): T;
function High(var x: T | T): T;
function Pred(x: T): T;
function Succ(x: T): T;
function Round(x: Real): Integer;
function Abs(x: T): T;
function Sqr(x: T): T;
function Sin(x: Real): Real;  
function Cos(x: Real): Real;  
function Arctan(x: Real): Real;  
function Exp(x: Real): Real;
function Ln(x: Real): Real;
function SqRt(x: Real): Real;

Standard library

System unit

function Timer: Integer;
procedure GetMem(var P: Pointer; Size: Integer);
procedure FreeMem(var P: Pointer);
procedure Randomize;
function Random: Real;
procedure Assign(var F: file; const Name: string);
procedure Rewrite(var F: file[; BlockSize: Integer]);
procedure Reset(var F: file[; BlockSize: Integer]);
procedure Close(var F: file);
procedure BlockRead(var F: file; var Buf; Len: Integer; var LenRead: Integer);
procedure BlockWrite(var F: file; var Buf; Len: Integer);
procedure Seek(var F: file; Pos: Integer);
function FileSize(var F: file): Integer;
function FilePos(var F: file): Integer;
function EOF(var F: file): Boolean;
function IOResult: Integer;
function Length(const s: string): Integer;
procedure Move(var Source; var Dest; Count: Integer);
function Copy(const S: string; Index, Count: Integer): string;
procedure FillChar(var Data; Count: Integer; Value: Char);
function ParamCount: Integer;
function ParamStr(Index: Integer): string;
procedure Val(const s: string; var Number: Real; var Code: Integer);
procedure Str(Number: Real; var s: string[; DecPlaces: Integer]);
procedure IVal(const s: string; var Number: Integer; var Code: Integer);
procedure IStr(Number: Integer; var s: string);
function UpCase(ch: Char): Char;

SysUtils unit

function IntToStr(n: Integer): string;
function StrToInt(const s: string): Integer;
function FloatToStr(x: Real): string;
function FloatToStrF(x: Real; Format: TFloatFormat; Precision, Digits: Integer): string;
function StrToFloat(const s: string): Real;
function StrToPChar(const s: string): PChar;
function PCharToStr(p: PChar): string;
function StrToPWideChar(const s: string): PWideChar;
function PWideCharToStr(p: PWideChar): string;

Samples

factor.pas - Integer factorization demo
lineq.pas - Linear equation solver. Uses gauss.pas unit. Requires eq.txt, eqerr.txt, or similar data file
life.pas - The Game of Life
sort.pas - Array sorting demo
fft.pas - Fast Fourier Transform demo
inserr.pas - Inertial navigation system error estimation demo. Uses kalman.pas unit
list.pas - Linked list operations demo
map.pas - Heterogenous list operations and Map function demo. Demonstrates XD Pascal methods and interfaces
gui.pas - GUI application demo. Uses windows.pas unit
raytracer.pas- Raytracer demo. Demonstrates XD Pascal methods and interfaces. Equivalent to raytracer.go

Known issues

Windows Defender antivirus is known to give false positive results on some programs compiled with XD Pascal.

xdpw's People

Contributors

Stargazers

Watchers

xdpw's Issues

does not work in delphi 10.3

does not work in delphi 10.3 community :(

Add 64-bit Windows support

Using a 64-bit code generator would make it very easy and 'natural' to implement Double and Int64 types.

Nevertheless, besides obvious changes in instruction prefixes, pointer sizes and PE header flags, the following difficulties are to be expected:

No push instruction for 64-bit immediate values even in the 64-bit mode
The stdcall calling convention has been replaced with fastcall in 64-bit Windows API, with the first four arguments passed via registers, all others via the stack.
Compiler source compatibility with Delphi/Free Pascal is questionable.

Request to transfer maintenance

In my previous message you said you no longer maintain this project. Would you be willing to transfer it to me? I've been wanting to find a Pascal compiler I could use for a project and this one is seriously accessible. I was considering FPC but at 250,000 lines (I recompiled it once) it's too big for one person. I was considering the P5 compiler (the one I mentioned for ISO 7185 support), but it's old. Yours looks interesting and reasonsable for one person to work on. I would like to take it over of you're no longer interested.

Thank you for your consideration.
Paul

Add ISO 7185 support

Hi, my name is Paul.

I think it would be nice if XDPW was ISO 7185 (Standard Pascal) compliant. In the Github repository tangentstorm / pascal in the directory P5 is the file pcom.pas which is a(n old) Pascal compiler from 1974 that was co-written by Nicklaus Worth himself. It generates instructions for the fictional Pascal-P machine (an early predecessor to Java's JavaVM virtual machine). It is also self-hosting, compiles perfectly with FPC or with itself, but on XDPW it faults on line 1 because it has file parameters in the &&Program** statement, which was allowed then and is allowed under ISO 7185. These parameters are usually ignored, but can be used as file names for input or output files (a relic of the CDC mainframe Pascal was invented on. This program, or its companion PINT.PAS use no advanced features like objects or even strings and would be a good testbed to use to check compatibility with the standard.

Thank you very much for your attention.

Paul Robinson [email protected]
"The lessons of history teach us - if they teach us anything - that no one learns the lessons that history teaches us."

Procedural variables or methods returning structures cannot be called as procedures

This program causes a compilation error:

program test;

type
  tvec = array [1..2] of integer;
  tfunc = function: tvec;
  
function f: tvec;
begin
result[1] := 5;
result[2] := 7;
end;

var
  func: tfunc;

begin
func := @f;
func();
end.

ошибка при компиляции проекта с Delphi7

файл common.inc
строка 544
INTEGERTYPE: Result := -2147483648;
целочисленное переполнение

Для Delphi надо написать
INTEGERTYPE: Result := -2147483647-1;

Конечно, мало кто сейчас пользует Delphi7 (я пользуюсь им только для бенчмарков).
Но если есть командный файл для сборки, то должно работать (или удалите командный файл).

Add Linux support

XDPWLinker.inc: add support for ELF output format
add glibc.inc for importing functions from libc.so
add Makefile for automated build on Linux

Implement DLL building

Currently XD Pascal can produce EXEs, but not DLLs. To support DLL building, we need to:

Fill the .edata and .reloc (?) sections in the PE
Introduce the library and exports keywords. There is no reliable information on the exports syntax

Easy to build under Delphi Rio

Just add an alias TChar = AnsiChar and replace any Char type by TChar like you did for TString :)

Implement abstract syntax tree

Currently XD Pascal generates machine instructions 'on-the-fly', like almost all early Pascal compilers, including Turbo Pascal. But building an AST can help perform larger-scale optimizations than those (peephole) optimizations available now.

Moreover, using an AST will eliminate the need for the clumsy MakeCStack procedure designed to deal with stdcall and cdecl functions.

Last but not least, it would probably help correctly compile expressions like a.f(x)^.b. Now it's a syntax error because the parser does not know when parsing f(x) whether its result is dereferenced (and so becomes a 'designator') or not (and so becomes a 'factor').

Implement Double type

Currently XD Pascal supports only one (32-bit) floating-point type, Real, which is equivalent to Single in Delphi. All other types, Single, Double, and Extended, are formally equivalent to this Real. Thus, the actual computation precision is always limited to 32-bit representations.

While the Extended type is not very practical (as it is too large and not portable to non-x87 platforms), the Double type could be really useful. The difficulty is that any procedure parameter or temporary value in XD Pascal is stored in a single 32-bit stack cell or register, and all function results are returned via eax. The introduction of the Double type may destroy this homogeneity.

The alternative approach is to switch to 64-bit code generation (#9) and get the Double type 'out-of-the-box'.

The road to Unicode support?

Hi, it would be nice to have support for unicode strings. The library that I want to use with it (around 700+ exported routines at this point) expect unicode strings.

I did manage to make the changes so that it compiles under the most recent version of Delphi, which is unicode and which my game library is made with.

Maybe create a special "experimental" branch for things such as this and this way your standard branch can remain unaffected?

In any case, great the see the steady stream of progress being made. It has come a long way since I first discovered it.

Automatic "uses" of referenced units?

Hi, great work on compiler since I last visited!

What would need to be changed/added so you simply do this:

program test;
uses
  unit1, unit2;
begin
end.

xdpw test.pas

So that you can eliminate the need to have to specify the included units on the command line? If I can figure out how to get his working, I should be able to start using xdpw in my projects. All the basic features are there and you are updating on a regular basis, which is fantastic to see.

Implement RTTI and the CASE statement over types

Comparing to Go, the current implementation of interfaces in XD Pascal is limited, since it lacks 'type assertions' and 'type switches' (i.e., the case statement over types, in Pascal terms). This omission, for example, makes empty interfaces meaningless, while they are widely used in Go, even for implementing Println.

Type switches require run-time type information (RTTI).

XD Pascal in KolibriOS

Hello,

Firstly I would like to thanks a lot the author,
and notify you that XD Pascal was ported into KolibriOS, as well as some apps.

http://forum.cantorsys.com/viewtopic.php?pid=794#p794
https://maxcodehack.github.io/hll-kos/pas/xdpas/

Thanks again for a nice compiler and have a nice day!

Best Regards,
Kiril aka Leency from KolibriOS Team

Add Position on line in error messages

Knowing exactly where on a line an error was discovered is extra helpful, and I recommend adding it. I'll even tell how. These changes are given in reverse order, bottom to top so a reference to a line number will remain the same after a change was made.

In scanner.pas, six changes.

Insert the Following procedure of 4 lines starting at line 697:

function ScannerPos: Integer;
begin
      Result := ScannerState.Position;
end;

After line 186 insert one line:
inc(ScannerState.Position);
Replace line 179
if ScannerState.ch = #10 then Inc(ScannerState.Line);
With the following five lines:

if ScannerState.ch = #10 then   // End of line found         '
  begin 
     Inc(ScannerState.Line);  
     Scannerstate.Position  := 0; 
  end;

After line 98 insert one line:
Position := 0;
After line 46 insert one line:
Position,
After line 29 insert one line:
function ScannerPos: Integer;

In file XDPW._PAS two changes:

Line 72, replace one line:
Notice(ScannerFileName + ' (' + IntToStr(ScannerLine) + ') Error: ' + Msg)
with the following split line:

  Notice(ScannerFileName + ' (' + IntToStr(ScannerLine) + 
              ':' + IntToStr(ScannerPos)  + ') Error: ' + Msg)

Line 61 replace one line:
Notice(ScannerFileName + ' (' + IntToStr(ScannerLine) + ') Warning: ' + Msg)
with the following split line:

  Notice(ScannerFileName + ' (' + IntToStr(ScannerLine) + ' 
             ':' + IntToStr(ScannerPos) +  ') Warning: ' + Msg)

Optional Change to Common.Pas, change line 14 from:
VERSION = '0.12';
to
VERSION = '0.13';

To confirm the change for anyone using it. I've compiled it and tried it and it's sweet, if you feed it a file with an error it now shows both line number and position on the line where the error was detected. I think it would be useful and I recommend its inclusion.

Paul.