ubiquitydotnet / llvm.net Goto Github PK
View Code? Open in Web Editor NEWLLVM Bindings for .NET
Home Page: https://ubiquitydotnet.github.io/Llvm.NET/
License: Apache License 2.0
LLVM Bindings for .NET
Home Page: https://ubiquitydotnet.github.io/Llvm.NET/
License: Apache License 2.0
Please provide the following information when submitting an issue.
Where appropriate replace the
[ ]
with a[X]
to mark an item as 'checked'
.NET Framework Used:
OS Environment:
OS Version: Windows 10 Pro Version 2004
I have already...
Description:
Building from src\Ubiquity.NET.Llvm.sln produces the following output
1>------ Build started: Project: Ubiquity.NET.Llvm, Configuration: Debug Any CPU ------
Error occurred while restoring NuGet packages: The local source 'C:\Files\Code\LLVM\Llvm.NET\BuildOutput\Nuget' doesn't exist.
1>C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(241,5): error NETSDK1004: Assets file 'C:\Files\Code\LLVM\Llvm.NET\BuildOutput\Intermediate\Ubiquity.NET.Llvm\project.assets.json' not found. Run a NuGet package restore to generate this file.
1>Done building project "Ubiquity.NET.Llvm.csproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
Manually creating the folder BuildOutput\Nuget
allows Llvm.NET to successfully build.
Steps to reproduce the problem:
Expected Behavior
Expected to build without errors.
Actual Behavior
Received build error.
Additional context
Is your feature request related to a problem? Please describe.
The only way to create a BitcodeModule from memory appears to be to write the memory to a file, create a MemoryBuffer from a file, and then load the BitcodeModule from the newly created MemoryBuffer. Ideally the memory could be fed directly to the MemoryBuffer.
Describe the solution you'd like
A new constructor added to MemoryBuffer that takes a byte array (byte[]
) and a buffer name (string
) and internally calls LLVMCreateMemoryBufferWithMemoryRangeCopy
Describe alternatives you've considered
Writing data to a file and then loading from a file.
Additional context
The signature for LLVMCreateMemoryBufferWithMemoryRangeCopy doesn't seem right, I would have expected either a byte array or a pointer, not a string for raw data.
The code generator is using the out ElementType, uint count pattern instead of elementType[], uint count) with marshaling attribute for an arry with IntPtr sub type.
The NuGet package linked in the readme 404s as of right now. Has it been renamed?
Description:
PsModules\CommonBuild\CommonBuild.psd1 contains a UTF-16 BLE BOM, causing GIT to think it is a binary file. This should be changed, and a git attributes added to specify the default encoding. Additionally, all files in the repo should be scanned for a BOM to ensure git treats everything as a proper text file.
The official builds for develop branch failed.
The tests failed due to a crash.
The active test run was aborted. Reason: Test host process crashed : Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Repeat 2 times:
--------------------------------
at Ubiquity.NET.Llvm.Interop.LLVMDIBuilderRef.LLVMDisposeDIBuilder(IntPtr)
--------------------------------
at Ubiquity.NET.Llvm.Interop.LLVMDIBuilderRef.ReleaseHandle()
at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean)
at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean)
at System.Runtime.InteropServices.SafeHandle.Finalize()
This is from a .NET finalizer attempting to call the dispser for a DIBuilder reference. However, the item is already disposed (Or it's container was) So, this needs investigation on how it could happen. The fact that this passed local runs AND the automated PR runs indicates it is, indeed a race condition, but nevertheless a real issue.
I get the following error when I try and build the solution.
Error MSB4030 "25/10/2017 10:20:57" is an invalid value for the "TimeStamp" parameter of the "GetBuildIndexFromTime" task. The "TimeStamp" parameter is of type "System.DateTime". MultiPlatformBuild C:\VS2017\Projects\Llvm.NET-master\BuildOutput\packages\CSemVer.Build.Tasks.1.0.0-alpha.2--ci-REL.194F4984\build\CSemVer.Build.Tasks.targets 117
Any thoughts ?
Andy
https://stackoverflow.com/questions/209133/exception-messages-in-english
Though not a big problem if restricted to Appveyor environment. May cause trouble to local development.
Found while testing the full integration and support of OrcJit:
Repro:
Run Kaleidoscope 6 or 7 with the sample Kaleidoscope code from the LLVM Tutorial.
# Logical unary not.
def unary!(v)
if v then
0
else
1;
# Unary negate.
def unary-(v)
0-v;
# Define > with the same precedence as <.
def binary> 10 (LHS RHS)
RHS < LHS;
# Binary logical or, which does not short circuit.
def binary| 5 (LHS RHS)
if LHS then
1
else if RHS then
1
else
0;
# Binary logical and, which does not short circuit.
def binary& 6 (LHS RHS)
if !LHS then
0
else
!!RHS;
# Define = with slightly lower precedence than relationals.
def binary = 9 (LHS RHS)
!(LHS < RHS | LHS > RHS);
# Define ':' for sequencing: as a low-precedence operator that ignores operands
# and just returns the RHS.
def binary : 1 (x y) y;
Root Cause:
The internal value caching doesn't track Replace All Uses With and deletion operations for Values. If the context is shared by a number of modules that come and go or the functions within them are transformed, then it is highly likely that any number of them will become invalid because some transform replaced or removed an instruction etc.. This ends up with a dangling pointer from managed code side. This becomes a blatant fail when LLVM has re-used the memory for some other type, which may not be a Value at all causing a crash, or if it is a value an InvalidCastException because the resolving to a specific wrapped type doesn't match the new reality.
Hi,
I was looking at the changes made in #185 which now allows for the creation of anonymous structures. However, when looking at DebugStructType
I couldn't find a constructor which didn't require it to be named [1][2]. Is this an oversight or am I simply going about it the wrong way?
the Kaleidoscope parse tree visitor article still references the older ParseTree is the AST implementation - this should be updated to reflect the transform from parset ree into simpler AST.
InstructionBuilder.SDiv creates UDiv instead of SDiv.
Here's some example code to demonstrate the issue.
public void SDivBug()
{
Context context = new Context();
BitcodeModule module = context.CreateBitcodeModule();
// We're going to create the LLVM equivalent of this C function:
// int test(int x)
// {
// return x / -3;
// }
IFunctionType function_type = context.GetFunctionType(context.Int32Type, new ITypeRef[]{context.Int32Type});
IrFunction function = module.CreateFunction("test", function_type);
BasicBlock block = function.AppendBasicBlock("entry");
InstructionBuilder builder = new InstructionBuilder(block);
Value lhs = function.Parameters[0];
Value rhs = context.CreateConstant(-3);
// vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Value sdiv = builder.SDiv(lhs, rhs); // <== Creating SDiv
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
builder.Return(sdiv);
if (!module.Verify(out string msg))
{
Console.WriteLine("module.Verify gave error: {0}", msg);
}
else
{
Console.WriteLine("module.Verify was okay");
}
Console.WriteLine(module.WriteToString());
}
Running this gives the following output:
module.Verify was okay
define i32 @test(i32 %0) {
entry:
%1 = udiv i32 %0, -3
ret i32 %1
}
Note that we have udiv instead of sdiv.
It looks like the following part of the InstructionBuilder code is selecting the wrong op builder:
Llvm.NET/src/Ubiquity.NET.Llvm/Instructions/InstructionBuilder.cs
Lines 219 to 229 in aabdd51
OS Environment:
Windows 10
I have already...
Reproduced the problem using the latest stable release.
Reviewed the documentation.
Reviewed the current issues to check that the issue isn't already known.
Description:
The NUGET package for Ubiquity.NET.Llvm includes a props file that only sets a property to the path . MSBuild now has built-in support for that and therefore this isn't needed. Additionally there are no uses of the property anywhere in the repository, so this is a prime candidate for removal.
Steps to reproduce the problem:
Expected Behavior
No redundant files in the pacakge
Actual Behavior
props file is redundant
I don't have VS2017 and cannot open the project. It would be really helpful if the project included a link on the main page for downloading the latest version of all of this. You have a link to something called appveyor, but you don't describe what that is, or how I can get the latest binaries from there.
Tuples can be represented with DebugStructType
, which works fine with one exception: 0-tuples. When attempting to create an empty struct, I'm greeted with the argument exception 'Type must be sized to get target size information'
. This leaves me with the question of how I would best go about representing this structure. Any advice?
Can't find 7-zip command line executable
At C:\Users\local.dev\source\repos\nanoFrameworkCore\Llvm.NET\buildutils.ps1:297 char:9
throw "Can't find 7-zip command line executable"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
DebugFunctionType
uses DebugType<ITypeRef, DIType>
instead of IDebugType<ITypeRef, DIType>
in its signature which causes invariance issues when trying to use it together with the other debug types. The other composite debug types uses IDebugType<ITypeRef, DIType>
which leads me to think that this might be an oversight.
Is your feature request related to a problem? Please describe.
LLVM JIT is undergoing another major re-work (LLVM 13 has settled down and exposed more of ORCJitv2 in the "stable" C API, but there have been others as well and the stability is a matter of interpretation. Additionally not all of the consumers of LLVM.NET need JIT support. Thus, this proposal is to split the JIT managed layer to a distinct NUGET package so that it can evolve independently and track the progress of ORCJitv2 without impacting consumers of Llvm.NET in general.
Describe the solution you'd like
Distinct package for the managed code layer of the JJIT support (Already a distinct namespace so this shouldn't be a large scale effort.
Describe alternatives you've considered
Despite using a ConcurrentDictionary under the hood the ContextCache implementation of GetContextFor isn't thread safe, it uses tryGetValue and then if that fails calls the constructor, which calls the context add method to add itself to the cache. There is a TOCTU problem there where another thread could add a different instance bound to the same underlying handle violating the reference equality guarantees.
ContextCache internally hides the ConcurentDictionary via an IDictionary interface, it shouldn't do that. It should leverage the members of the Concurrent dictionary like GetOrAdd() etc... to ensure proper thread safety and ref equality guarantees.
A known issue with the Kaleidoscope samples but until now not documented/tracked anywhere.
Repro:
def foo(x y) x+y;
foo(2, 3);
Evaluated to 5
def foo(x y) x*y;
foo(2,3);
Evaluated to 5
Obviously it still resolves the foo
symbol to the original definition. The official LLVM tutorial indicates this should work automatically, but doesn't. (Might work for the interpreter and might have been true for the original legacy JIT but doesn't work for the current MCJIT implmentation nor the OrcJIT. What is needed is to remove the module from the JIT engine before adding the new one. This will fully replace the module and remove any global symbol mappings for the module in the JIT. A fix for this is available and pending resolution to #43.
Line 29 in 54789d0
Hello, was just reading through the readme and noticed that the line above is currently linking to netmf.com
which passes through a 301 redirect chain that ends up at https://www.prfmnews.com/
which doesn't seem to be related.
The docs for the internal string marshaling support are not accurate anymore. WIth 8.0.0 the library uses the bindings generator to create marshaling using a specific custom marshaling type, instead of the overloaded general marshaller with cookie parameter..
LInks from the index and it's TOC are missing the "samples" segment of the path and therefore trigger a 404 error.
Load (and others (like GEP) supporting the new opaue pointer concept with back-compat for typed pointers are incorrectly using the pointer type as the type pointer, however, it turns out, they should acully use the pointee type.
Hi,
I was looking at DebugStructType
and and noticed several constructors which seemingly can't be called outside of the assembly, or at least not without creating custom concretions of some interfaces. This left me thinking that maybe these constructors (and others like it) should be marked as internal
as they're probably not intended to be called by the consumer anyway?
Just a thought.
Please provide the following information when submitting an issue.
I am trying to link a function to be 'external' yet on the output ll it's not doing anything. This does work for internal, and all other linkage types.
function.Linkage = Linkage.External;
// Doesnt work
//output
define void @run()
//end of output
//expected output
define external void @run()
//end of expected output
function.Linkage = Linkage.Internal;
//Does work
//output
define internal void @run()
//end of output
This is a pretty minor issue, but nevertheless might be something to take note of. IntPredicate
and RealPredicate
have different naming schemes from each other. The latter uses Than
while the former does not. For example: IntPredicate.SignedLess
& RealPredicate.OrderedAndLessThan
.
Additionally, the documentation for RealPredicate
seem to be suffering from a copy-paste error:
"Predicate enumeration for integer comparison".
Please provide the following information when submitting an issue.
Where appropriate replace the
[ ]
with a[X]
to mark an item as 'checked'
.NET Framework Used:
OS Environment:
OS Version: Windows 10 Pro Version 2004
I have already...
Description:
Running Build-Interop.ps1 results in the following build error.
InteropLog1_CppSharp10_1.txt
This appears to be an issue with an issue with CppSharp using an older version of clang then VS2019 expects. Updating CppSharp to 10.2 results in a different build error:
InteropLog2_CppSharp10_2.txt
and upgrading CppSharp to version 10.3, 10.4 or 10.5 results in the following build error:
InteropLog3_CppSharp10_3.txt
Steps to reproduce the problem:
Expected Behavior
Successful build without errors
Actual Behavior
Build fails because of errors.
Additional context
Running Build-All.ps1 shows the following environment details
PS C:\Files\Code\Misc\Llvm.NET> .\Build-All.ps1
Build Info:
Name Value
---- -----
TestResultsPath C:\Files\Code\Misc\Llvm.NET\BuildOutput\Test-Results
BinLogsPath C:\Files\Code\Misc\Llvm.NET\BuildOutput\BinLogs
BuildOutputPath C:\Files\Code\Misc\Llvm.NET\BuildOutput\
RepoRootPath C:\Files\Code\Misc\Llvm.NET
LlvmVersion 10.0.0
DownloadsPath C:\Files\Code\Misc\Llvm.NET\Downloads
ToolsPath C:\Files\Code\Misc\Llvm.NET\Tools
MsBuildLoggerArgs {/clp:Verbosity=Minimal}
DocsOutputPath C:\Files\Code\Misc\Llvm.NET\BuildOutput\docs
SrcRootPath C:\Files\Code\Misc\Llvm.NET\src
LlvmLibsPackageReleaseName 10.0.0-msvc-16.5
LlvmLibsRoot C:\Files\Code\Misc\Llvm.NET\llvm
NuGetOutputPath C:\Files\Code\Misc\Llvm.NET\BuildOutput\NuGet
DocsRepoPath C:\Files\Code\Misc\Llvm.NET\BuildOutput\docs
NuGetRepositoryPath C:\Files\Code\Misc\Llvm.NET\BuildOutput\packages
MSBUILD:
Name Value
---- -----
BinPath C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin
FullPath C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\MSBuild.exe
FoundOnPath True
Version 16.8.2.56705
Name Value
---- -----
IsReleaseBuild False
IsPullRequestBuild False
IsAutomatedBuild False
BuildKind: LocalBuild
CiBuildName: ZZZ
Is your feature request related to a problem? Please describe.
Currently projects based on Llvm.NET only work on Windows. It would be great if they could be used with .NET Core on Mac and Linux!
In particular, I'm using Llvm.NET in a project that is cross-platform, and I can't merge my new feature to master until I have a cross-platform solution.
Describe the solution you'd like
The optimal solution would be for there to be Mac and Linux NuGet packages for Llvm.NET, as well as Windows.
Describe alternatives you've considered
None
Additional context
I'm starting to work on this now. As I understand it from this project's maintainer, it's mostly a build issue. The .NET libraries are all cross-platform by default; the problem is the generated C++ library that wraps the actual LLVM libraries.
The LLVM signature of memcpy, memset, and memmove instrinsics was changed to remove the align parmater (in favor of placing an alignment attribute on the actual parameters since it could be that they are not both the same)
Is your feature request related to a problem? Please describe.
ConstantDataArray are sometimes used to store binary blobs and there is no built-in way to access the raw data in those cases.
Describe the solution you'd like
A built in ConstantDataArray.getRawDataValues
method
Describe alternatives you've considered
I have been using this method to access the data
[SuppressUnmanagedCodeSecurity]
[DllImport("Ubiquity.NET.LibLLVM", CallingConvention = CallingConvention.Cdecl)]
public static extern IntPtr LLVMGetAsString(LLVMValueRef c, out size_t Length);
public static byte[] GetRawBytes(this ConstantDataSequential value)
{
var valueHandle = (LLVMValueRef)typeof(Value)
.GetProperty("ValueHandle", BindingFlags.Instance | BindingFlags.NonPublic)
.GetValue(value);
var charPtr = LLVMGetAsString(valueHandle, out var length);
var result = new byte[length];
Marshal.Copy(charPtr, result, 0, length);
return result;
}
I've also considered something like this that would prevent a second copy of the byte array to a different typed array, but I did not require it for my use case.
public static unsafe T[] GetData<T>(this ConstantDataSequential value) where T : unmanaged
{
var valueHandle = (LLVMValueRef)typeof(Value)
.GetProperty("ValueHandle", BindingFlags.Instance | BindingFlags.NonPublic)
.GetValue(value);
var charPtr = LLVMGetAsString(valueHandle, out var length);
var result = new T[length / Marshal.SizeOf<T>()];
fixed(T* ptr = result)
{
Buffer.MemoryCopy(charPtr.ToPointer(), ptr, length, length);
}
return result;
}
Additional context
The default LLVMGetAsString returns a managed string and ConstantDataSequential.getRawDataValues returns a StringRef, but LLVM seems to use char pointers and StringRefs for both strings and non-string binary data.
The memberpage plugin isn't being applied in the appveyor builds, or potentially on some local machine builds depending on the version of the dotnet SDK on the system.
This appears to be due to a change in the dotnet SDK between 2.1.202 and 2.1.302 where dotnet\sdk\2.1.302\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets
no longer produces the PackageDefinitions items that the Llv.Net.Docfx project custom GetDocfxPackagePaths
depended on to locate the memberpage and MSDN xref nuget packages. Either the GetDocfxPackagePaths needs to be updated to trigger the target that will produce those MSBuildItems or a new mechanism for reliably locating those needs to be found.
Is your feature request related to a problem? Please describe.
Can't create a DIFile with any Checksum Information or source text
Describe the solution you'd like
Add optional parameters (default to "none" semantics) to DiBuilder.CreateFile to allow adding the checksum information. This will include creating a new value type for the checksum info, and enum for checksum kind, as well as new LibLLVM* APIs to create the DIFile as the existing LLVM-C API does not provide support for these parameters.
Describe alternatives you've considered
Wait around for the official LLVM-C API to catch up to changes in the underlying LLVM... [Not something that's worked well over time.]
Is your feature request related to a problem? Please describe.
NOTE:
Feature requests are intended to request a new feature or a change in the intended/designed
functionality (a.k.a. DCR). They should not be used to report bugs, which are incorrect
implementations of the designed functionality.
Editing and maintaining the bindingsConfig.yml FunctionBindings section is tedious and error prone, While there are a lot of validations built into the LlvmBindingsGenerator it's a completely foreign language to generate C#, an arguably non-foreign language). It is likely simpler to just express those manual marshaling constructs in actual C#. The functionBindings section is basically just P/Invoke signatures in a weird language... which makes it harder to maintain.
Describe the solution you'd like
Tooling can generate a single p/invoke file from current FunctionBindings to eliminate the tedious creation of such a file, which is then maintained manually after that. Then, once that is created from the FunctionBindingsTable, then most of that part of the configuration file is removed/replaced with a simpler list of "manually managed interop" functions. With the rest auto generated. This allows for auto generation on many of the APIs, and only the parts with ambiguities needing manually declarations is actually maintained (and in a familiar language to start with).
Now that the bindings generator works in .NET Core, this entire process could move to a compiler source generator. This would allow the tool to perform validation, including the manually edited parts to ensure nothing is missed as it does now, while only generating for things that are non-ambiguous. The HandleMap, AnonymousEnums, and IgnoredHeaders sections can remain in the configuration file as they are pretty straight forward and easy to understand.
Describe alternatives you've considered
Additional context
Building things into the compilation process gives greater options for validation AND helps set the stage for doing this part of things for multiple platforms.
NOTE
This isn't a total reversal to doing it all manually. It's a realization that some of it is generatable but other parts require intervention and that the best way to do that is in the target language C#. So the tool will need to understand that code to detect redundant P/Invokes of things removed, new APIs, not yet covered, as well as validate signatures of the methods still match the latest LLVM versions. (It does this now based on the YAML config information so that needs extension to Roslyn syntax trees instead). Thus it's a move towards a hybrid approach with the tooling analyzing the manually maintained code to report any issues with it. (Thus, hopefully, eliminating the error prone part)
Please provide the following information when submitting an issue.
Where appropriate replace the
[ ]
with a[X]
to mark an item as 'checked'
.NET Framework Used:
OS Environment:
OS Version: 19042.630
I have already...
Description:
Llvm.NET.Context.CreateConstantString fails with
"Unable to cast object of type 'Llvm.NET.Values.ConstantAggregateZero' to type 'Llvm.NET.Values.ConstantDataArray'"
when given an empty string.
Expected Behavior
It should properly handle the case and successfully construct the corresponding ConstantDataArray.
Actual Behavior
It throws and exception.
Hi,
I switched to alpha.3 after I noticed that you had published it and promptly made some progress on the code generation. I can create anonymous structs just fine now, however, when trying to nest structs I'm faced with the following exception:
Incompatible types: destination pointer must be of the same type as the value stored.
I'm using the visitor pattern to traverse my AST, generating IR as I go. When I visit a node, I translate that node's type to its corresponding LLVM type. It seems like Llvm.NET is not satisfied with that, requiring the exact type. Is there any way around this?
This is my code:
public override Expression VisitTupleExpression(TupleExpression node)
{
var type = GetType(node.Type);
var tuplePtr = builder.Alloca(type);
for (var i = 0; i < node.Elements.Count(); i++)
{
var element = node.Elements.ElementAt(i);
Visit(element);
var value = valueStack.Pop();
var valuePtr = builder.GetStructElementPointer(type, tuplePtr, (uint)i);
builder.Store(value, valuePtr);
}
valueStack.Push(tuplePtr);
return node;
}
Hi,
Is this project still maintained?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.