microsoft / prose Goto Github PK

Microsoft Program Synthesis using Examples SDK is a framework of technologies for the automatic generation of programs from input-output examples. This repo includes samples and sample data for the Microsoft Program Synthesis using Example SDK.

Home Page: https://microsoft.github.io/prose/

License: Other

C# 59.49% HTML 0.61% Dockerfile 0.02% Jupyter Notebook 18.77% Python 21.11%

synthesis sdk prose microsoft dotnet csharp data-transformation data-wrangling program-synthesis examples

prose's Introduction

Microsoft Program Synthesis using Examples SDK

The Program Synthesis using Examples (PROSE) SDK includes a set of technologies for the automatic generation of programs from input-output examples. This repo includes samples, release notes, and some other miscellaneous projects related to the Microsoft PROSE SDK.

The samples are split into two categories:

Samples for using existing PROSE DSL APIs to accomplish tasks in api-samples/api-samples.sln.
Samples for creating program synthesis solutions using the PROSE SDK by authoring a DSL in the dsl-samples directory:

You can find guides for some of these sample projects and other information about the PROSE project here: https://microsoft.github.io/prose/

Optionally, you can get started quickly using Docker:

git clone https://github.com/microsoft/prose.git
cd prose
docker build -t prose-samples .
docker run -it --rm -v "$(pwd):/opt/prose-samples" -w "/opt/prose-samples" prose-samples bash
# Inside the Docker container
cd dsl-samples/ProseSample/ProseSample  # ... or the directory for any other sample project
dotnet run  # run the sample in the current directory

Contributing

The source of truth for all the information in this repository is actually an internal Microsoft repository, and any changes made here are at risk of being overwritten by future public releases from the PROSE team. If you detect issues with any of the samples or other things in this repo, please open an issue, and someone from the PROSE team will work with you to see that the problem is addressed.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

prose's People

Contributors

Stargazers

Watchers

Forkers

gustavoasoares xornand derzorngottes cephdon mwidmeyer alexpolozov eglassman kleopatra999 gulwani e050110 penguincwarrior destenson gsakkas shuyindeng orkun1675 bids power10dan arjunradhakrishna karenho912 patricklss1106 vrnithinkumar spencerx artificiaiintelligence nanzf vikramadhav kethku muhammadiqbal tmc vakaras shyamalschandra jreuben11 kevinwkc opbarbo indigos33k3r rajivyadavlw xurongchen kunalspathak bluelight1324 namin fraser-greenlee dohki chunlei saurkt danpere ktsoares programmiererphp rexwangyi nasa03 lserafin blakepancin sanyam07 dwainew rednickle bhaskers-blu-org2 daraoladapo taffywrinkle rmoin claudiusgonzo csim winnerineast jcotten global-localhost global19 global19-atlassian-net pengking dhvanilsanghvi dolphingarlic amogh-saxena qpc-database wslee maxvonhippel sissel-wu jimfhahn yetingli radekdoulik gaybro8777 wangsihan95 leemgs bharatr21 ysahil97 kwg126 petertran14 test-mass-forker-org-1 isabella232 internetofdata amrmuhammad linuxsuren mtg-kimc orby-will m4rm0k jadthehuman josepablocam romaninv pletain ananyas168 ivanliu-microsoft

prose's Issues

PowerShell Extract JSON

Great work on prose. I explored and wrapped the extract json in PowerShell. I need to find time to do more investigation of the rest of prose.

Is there an ignore case option?

If I pass a lower case pe5 as the last argument, Pprose throws the error below. Is there an ignore case option that can be specified?

splitSession.Constraints.Add(new NthExampleConstraint(inputs[0].Value, 0, "PE5"));

"The example value is not a substring of the input"

Failed to run the tutorial on Mac M1

I installed .NET 6 GA Arm64 SDK and ran the build command which worked successfully but when I try running the dotnet test command, it fails and throws an error.

Steps to reproduce:

After cloning the repository, cd prose/dsl-samples/tutorial/part1a.
Run dotnet build.
Run dotnet test. It will throw the following error:

Testhost process exited with error: It was not possible to find any compatible framework version
The framework 'Microsoft.NETCore.App', version '2.1.0' (arm64) was not found.
  - The following frameworks were found:
      6.0.0 at [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
You can resolve the problem by installing the specified framework and/or SDK.
The specified framework can be found at:
  - https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=2.1.0&arch=arm64&rid=osx.11.1-arm64
. Please check the diagnostic logs for more information.

Test Run Aborted.

ReadFlatFileLearner is not able to detect file delimiter

I have Text file
smallNER.txt
ReadFlatFileLearner.Learn(stream, options) is detecting an empty delimiter

Note: Columns count in each row are not the same .

System.NullRefenceException in FlashFill Example

When I'm running the FlashFill example in the SDK, I encountered an "System.NullRefenceException" for a learning example.

Code snippet:

var examples = new[]
   {
        new FlashFillExample(new FlashFillInput("c", "a", "k"), "c a. k"),
        new FlashFillExample(new FlashFillInput("a", "", "c"), "a c"),
        new FlashFillExample(new FlashFillInput("a", "j", "i"), "a j. i"),
        new FlashFillExample(new FlashFillInput("a", "", "i"), "a i"),
        new FlashFillExample(new FlashFillInput("x", "y", "i"), "x y. i")
    };
FlashFillProgram program = FlashFillProgram.Learn(examples);
string output = program.Run("a", "x", "i");

I run the code and there is a System.NullRefenceException on the last line of the code. Besides, if I only set examples containing only one instance (e.g. first instance of the example list), I won't get the Exception.

What is the cause and how can I handle it?

Thanks!

additional constraints with look_ahead_regex, look_behind_regex don't work anymore

Was playing with the new update in a hackathon. The below code results into learning failed if I pass a proper regex for lookBehindRegex, matchingRegex, and lookAheadRegex.

Microsoft.ProgramSynthesis.Extraction.Text.RegionLearner.Instance.LearnTopK(
    exampleConstraints, null, 1, lookBehindRegex, matchingRegex, lookAheadRegex);

I had played with the APIs a few months earlier, and in the previous version (0.1.1-preview-1), the below code worked fine and the resulting program would be consistent with lookBehindRegex, matchingRegex, and lookAheadRegex:

Microsoft.ProgramSynthesis.Extraction.Text.Learner.Instance.LearnTopKRegion(                
    positiveExamples, negativeExamples, null, 1, lookBehindRegex, matchingRegex, lookAheadRegex);

Semantic Transformation with Multiple Lookups

Hello,
First of all, thank you for creating PROSE and making it publicly available.
I am aware that PROSE is able to perform semantic transformation for parsing the dates and rounding the numbers, and I assume that in the background these dates and numbers are defined explicitly to the framework in order to find the match in the relevant table. However, my question is if my transformation requires multiple lookups to other tables, how can I add these related tables directly to PROSE? According to my understanding, FlashFill DSL is powerful enough to enable this kind of semantic transformation, but I'm not quite sure that how I can explicitly add these relevant tables to learning session in PROSE.
Thank you!

Net Core Build Fail

I get the following error when I try to build the ProseSamples project.

OS: Linux 5.6.6-arch1-1

~/.nuget/packages/microsoft.programsynthesis.dslctargets/7.9.1/build/Microsoft.ProgramSynthesis.DslcTargets.targets(82,9): error MSB3073: The command "dotnet dslc "@~/Documents/prose/ProgramSynthesis/ProseSample.Substrings/obj/Debug/netcoreapp2.1/ProseSample.Substrings.rsp"" exited with code 150. [~/Documents/prose/ProgramSynthesis/ProseSample.Substrings/ProseSample.Substrings.csproj]

Disjunctive spec clarification question

I'm having trouble understanding exactly how disjunctive specs and conditioned witness functions (for multiple parameters) are supposed to interact.

I made a minimal example here. It's a language with only (recursive) string append, i.e. for inputs x it can generate x, Append(x, x), Append(Append(x,x),x), etc.

The master branch and rev-cond branch show the same approach but condition the witness functions in the opposite order. Based on the conditioning, it can only synthesize programs with Append recursively along one branch. The problem seems to be that the second witness function is never dispatched, and it should take a disjunctive spec as an argument instead, since that's what the first returns. But intuitively, I would think that a witness function on an example spec could automatically be mapped over a disjunctive spec; is there something unsound about this or is it just not supported?

On the disjunctified branch, I tried to use DisjunctiveExamplesSpecs instead, but with even a reduced version for debugging, PROSE invokes the first witness function in an infinite loop on the same input (and never invokes the second). I would expect this not to be possible because (as I understand it) the results of these calls are memoized. I've tried many type signatures for the second witness function, and it never seems to be invoked.

But even if it did work, I'm not sure how you would maintain dependencies across the conditioned functions. For example, if you're trying to synthesize a program for "ff" -> "fff" (which can't be represented in this language), the prefix witness function would say that the left input could be "" | "f" | "ff" | "fff", and given that disjunctive spec, the suffix witness function would say that the right input could be "fff" | "ff" | "f" | "". But now the dependencies between particular terms across those disjunctions seem to be lost. What am I missing?

Thanks

(also tagging @namin to follow this)

Parse humanReadable program string

Hello,
I was following the tutorial, but at this step
var ast = ProgramNode.Parse("Id(simpleInput)", grammar, ASTSerializationFormat.HumanReadable);
I got Unhandled exception. System.NotSupportedException: Deserializing human-readable ASTs is not supported.
In which Id is my identity function that does nothing.

If I use the equivalent XML as the input
program_string = "<NonterminalNode symbol=\"simpleOutput\" rule=\"Id\"><VariableNode symbol=\"simpleInput\" /></NonterminalNode>";
and parse with ASTSerializationFormat.XML, then it works.

I look into the code and it seems like only the XML format is currently supported. So given a more complicated function, like "Substring(x, PositionPair(AbsolutePosition(x, 0), AbsolutePosition(x, 5)))", how do I manually construct the XML to test?
Thanks.

Error in Building ProseSample.sln

I am trying to build ProseSample.sln, but I get the following error

The type or namespace name 'Language' does not exist in the namespace 'ProseSample.Substrings'

I had downloaded the master branch of the repository from GitHub. Then Opened the ProseSample.sln file using Visual Studio 2019. Then I changed the Target framework to .NET Core 3.1 for 'ProseSample', 'ProseSample.Substrings', 'ProseSample.Substrings.Semantics' and 'ProseSample.TextExtraction'.

I also verified that the following NuGet packages are installed for the project:
Microsoft.ProgramSynthesis, Microsoft.ProgramSynthesis.Common, Microsoft.ProgramSynthesis.Compiler and Microsoft.ProgramSynthesis.Dslctargets

Use framework

For me to use the framework I need to install it in VS Code or Visual Studio, correct? But I'm trying to install via nuget and I can't. I would like an orientation, an installation tutorial or something. Thanks for listening!

Tutorial error

Can you tell me how to solve this? (image attached)

Solution file of PROSE not getting loaded is solution explorer in new version of Visual Studio i.e. 17.8.2

when I try to open any solution file in Visual studio ,it is not getting loaded saying unloaded in new version of Visual Studio i.e. 17.8.2 . can you please specify which version of visual Studio will work fine for it.

Package Microsoft.ProgramSynthesis 2.3.0 is not compatible with uap10.0.10586

Hello,
I tried to install Prose on my freshly installed Visual Studion using NUGET console. After executing:
Install-Package Microsoft.ProgramSynthesis
I get:

Install-Package : Package Microsoft.ProgramSynthesis 2.3.0 is not compatible with uap10.0.10586 (UAP,Version=v10.0.10586). Package Microsoft.ProgramSynthesis 2.3.0 supports:
  - net45 (.NETFramework,Version=v4.5)
  - netstandard1.6 (.NETStandard,Version=v1.6)
At line:1 char:1
+ Install-Package Microsoft.ProgramSynthesis
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Install-Package], Exception
    + FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PackageManagement.PowerShellCmdlets.InstallPackageCommand

Please help to resolve this issue.

Setting recursion limits of grammars

Hello and thank you very much for the great framework!

I have defined a recursive DSL grammar, and want to make the search space finite by limiting depth and "unrolling" the grammar finite times.
Perhaps could I do that by annotating production rules of the grammar? Then, what kinds of keywords can I use when using the DSL compiler?

Opaque failure - What to do when 'Learning fails'?

I see in the samples this code snippet. When I try and create my own dataset and try to learn from it from some not very nice real world examples, my program tends to output "Error: Learning fails!"

What does it really mean that learning failed? Does it mean that the grammar is too incomplete to build a suitable generalization so the grammar needs to be extended!? Could it also mean that generalization was too hard for the learning system and it gave up, in which case maybe it will work with more examples? How can I determine the correct path forward?

This Learn() api really needs to throw a specific exception instead of just returning null!

Data Sets shared on google drive are missing

The readme has a link to datasets that can't be accessed:

https://drive.google.com/drive/folders/1gBZuhV42VHhrwsiKpEA4dLYVdiiXbUGp?usp=sharing

Several issues with the tutorials

Hello and thank you for creating the PROSE SDK! I was very excited to find out about it.

I tried to work through the "Getting started" tutorial and the example on the "Framework" page, and found several typos and other little bugs. I will list them all here and I hope some of those will be helpful for improving the tutorials.

Framework https://microsoft.github.io/prose/documentation/prose/usage/
- The language definition has an x missing at line 9: in its semantics definition, CPos depends on two variables - a string s and an int k, so I suppose line 9 should be int P := CPos(x,k) | Pos(x, r, r, k);
- In the grammar, SubStr takes a string a two integers, while in the semantics - a string and an int tuple. I changed the semantics function to be public static string SubStr(string s, int left, int right){...}
- Similarly, in the grammar Pos takes a string, two Regex, and and int, and in the semantics - a string, a Regex tuple and an int. I changed the semantics function to be public static int? Pos(string s, Regex leftRegex, Regex rightRegex, int occurrence){...}
- Some minor things
  - in Pos, .FormatWith() doesn't work for me, so I used String.Format()
  - in SubStr, .Slice() doesn't work for me, so I used .Substring()
Getting started https://microsoft.github.io/prose/documentation/prose/tutorial/
- In Semantics: Match doesn't have a Right property. m.Right should be m.Index ?
- In WitnessFunctions:
  - In several places ExampleSpecification (or DisjunctiveExamplesSpecification) should be ExampleSpec (or DisjunctiveExamplesSpec).
  - There are a few casts missing (that can't be implicit). E.g. in AbsolutePosition, result[inputState] = ks; should be result[inputState] = ((IEnumerable<int?>)ks).Cast<object>();, but I guess for the tutorial is better to be simpler in syntax.
  - As in Semantics, Match doesn't have a Right property. m.Right should be m.Index ?
  - I'm not entire surely what's going on the the WitnessRegexPair function, but the values of leftMatches and rightMatches are list of Matches, while the regex list we add to the result is of Regex tuples, rather than Match tuples. In BuildStringMatches, shouldn't we keep track of the regexes we've used?
  - WitnessKForRegexPair should take a GrammarRule rather than a Grammar?
  - Minor: BuildStringMatches() should be static.

Extracting one (or more) regions of the same type from each new file

Hi!

I want to extract one (or more) regions of the same type from each new file. Below, I have code that demonstrates how the first two examples contain one region to extract and the third example contains two. However, I'm getting a System.ArgumentException when calling the RegionLearner.Instance.Learn() on the examples because there are two items with the same key (probably the two regions from the same input file).

I'm using the library incorrectly; the docstring for LearnRegion() confirms that it can only learn a single region per file. None of the other example function calls appear to model what I want.

What do you recommend?

Cheers,
Elena

string text0 = File.ReadAllText("training_samples/training_sample_0.html");
			string text1 = File.ReadAllText("training_samples/training_sample_1.html");
			string text2 = File.ReadAllText("training_samples/training_sample_2.html");

			var input0 = RegionLearner.CreateStringRegion(text0);
			var input1 = RegionLearner.CreateStringRegion(text1);
			var input2 = RegionLearner.CreateStringRegion(text2);

			var extractedRegion0_b = input0.Slice(107, 170);
			var extractedRegion1_b = input1.Slice(228, 305);
			var extractedRegion2_b = input2.Slice(225, 293);
			var extractedRegion3_b = input2.Slice(294, 366);

			//Console.WriteLine(extractedRegion0_b.ToString());
			//Console.WriteLine(extractedRegion1_b.ToString());

			//return;

			var examples_b = new[] {
				new CorrespondingMemberEquals<StringRegion, StringRegion>(input0, extractedRegion0_b), // "Carrie Dodson 100" => "Dodson"
				new CorrespondingMemberEquals<StringRegion, StringRegion>(input1, extractedRegion1_b), // "Leonard Robledo 75" => "Robledo"
				new CorrespondingMemberEquals<StringRegion, StringRegion>(input2, extractedRegion2_b), // "Leonard Robledo 75" => "Robledo"
				new CorrespondingMemberEquals<StringRegion, StringRegion>(input2, extractedRegion3_b) // "Leonard Robledo 75" => "Robledo"
            };


			RegionProgram topRankedProg_b = RegionLearner.Instance.Learn(examples_b);

Failed to add reference to 'libz3' while updating to Prose 2.0.0

It was not possible to update to update to Prose 2.0.0. When I try to do it using Nuget, I got the following error:

Severity Code Description Project File Line Suppression State
Error Failed to add reference to 'libz3'. 0

Steps to reproduce:

Open Nuget
Update Prose from 1.1.0 to 2.0.0

How to make grammar recursive

HI PROSE Team! My group and I are using PROSE for a synthesis project for image transformations. We are running into an issue where we can't make the grammar work recursively. We initially started to generate programs using this DSL:

@input Image input_image;

@start Image program := single;

Image single := FilterColor(input_image, color)
             | Recolor(input_image, color);

int color;

This works as expected and we are able to synthesize correct programs. However, the obvious limitation is that the grammar doesn't allow recursive operations, so we can only synthesize very simple programs. We then modified the grammar to be in the format:

@input Image input_image;

@start Image program := single;

Image single := FilterColor(single, color)
             | Recolor(single, color)
             | input_image;
	

int color;

However, with the same input / output test cases we weren't able to synthesize any program. From what I have been able to tell by googling around a bit is that we need another witness function for the 'single 'arguments for each of these grammar rules, however, I am not sure what those witness functions are meant to do. It was very clear what the witness functions for the color argument for the operators should be. Our repo is available here: https://github.com/RyanLuu/ArChEs/tree/recursive/demo/arches_v1/synthesis

Any help or suggestions on how to get our grammar working recursively would be greatly appreciated!

Test case not run

Test cases not running on Visual Studio.
System Spec:
Windows 10.
Visual Studio: 2022
.NET: 7.0.304

A few metadata files are missing

When I run 'Program.cs' in '/ProseSample' directory, I got the following errors:

Building: ProseSample (Debug)
Build started 7/2/2017 5:01:59 PM.
__________________________________________________
Project "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample/ProseSample.csproj" (Build target(s)):

Target _GetProjectReferenceTargetFrameworkProperties:
    __________________________________________________
    Project "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample/ProseSample.csproj" is building "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings.Semantics/ProseSample.Substrings.Semantics.csproj" (GetTargetFrameworkProperties target(s)):

Target _GetProjectReferenceTargetFrameworkProperties:
    __________________________________________________
    Project "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample/ProseSample.csproj" is building "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings/ProseSample.Substrings.csproj" (GetTargetFrameworkProperties target(s)):

Target _GetProjectReferenceTargetFrameworkProperties:
    __________________________________________________
    Project "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample/ProseSample.csproj" is building "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.TextExtraction/ProseSample.TextExtraction.csproj" (GetTargetFrameworkProperties target(s)):

Target ResolveProjectReferences:
    __________________________________________________
    Project "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample/ProseSample.csproj" is building "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings.Semantics/ProseSample.Substrings.Semantics.csproj" (GetTargetPath target(s)):

    __________________________________________________
    Project "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample/ProseSample.csproj" is building "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings/ProseSample.Substrings.csproj" (GetTargetPath target(s)):

    __________________________________________________
    Project "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample/ProseSample.csproj" is building "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.TextExtraction/ProseSample.TextExtraction.csproj" (GetTargetPath target(s)):

    __________________________________________________
    Project "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample/ProseSample.csproj" is building "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings.Semantics/ProseSample.Substrings.Semantics.csproj" (GetNativeManifest target(s)):

    __________________________________________________
    Project "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample/ProseSample.csproj" is building "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings/ProseSample.Substrings.csproj" (GetNativeManifest target(s)):

    __________________________________________________
    Project "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample/ProseSample.csproj" is building "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.TextExtraction/ProseSample.TextExtraction.csproj" (GetNativeManifest target(s)):

Target ResolveAssemblyReferences:
    Primary reference "ProseSample.Substrings.Semantics".
        Could not find dependent files. Expected file "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings.Semantics/bin/Debug/ProseSample.Substrings.Semantics.dll" does not exist.
        Could not find dependent files. Expected file "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings.Semantics/bin/Debug/ProseSample.Substrings.Semantics.dll" does not exist.
        Resolved file path is "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings.Semantics/bin/Debug/ProseSample.Substrings.Semantics.dll".
        Reference found at search path location "".
        The ImageRuntimeVersion for this reference is "".
    Primary reference "ProseSample.Substrings".
        Could not find dependent files. Expected file "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings/bin/Debug/ProseSample.Substrings.dll" does not exist.
        Could not find dependent files. Expected file "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings/bin/Debug/ProseSample.Substrings.dll" does not exist.
        Resolved file path is "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings/bin/Debug/ProseSample.Substrings.dll".
        Reference found at search path location "".
        The ImageRuntimeVersion for this reference is "".
    Primary reference "ProseSample.TextExtraction".
        Could not find dependent files. Expected file "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.TextExtraction/bin/Debug/ProseSample.TextExtraction.dll" does not exist.
        Could not find dependent files. Expected file "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.TextExtraction/bin/Debug/ProseSample.TextExtraction.dll" does not exist.
        Resolved file path is "/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.TextExtraction/bin/Debug/ProseSample.TextExtraction.dll".
        Reference found at search path location "".
        The ImageRuntimeVersion for this reference is "".
Target GenerateTargetFrameworkMonikerAttribute:
  Skipping target "GenerateTargetFrameworkMonikerAttribute" because all output files are up-to-date with respect to the input files.
Target CoreCompile:
    /Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5/csc.exe /noconfig /nowarn:1701,1702,2008 /nostdlib+ /platform:anycpu32bitpreferred /errorreport:prompt /warn:4 /define:DEBUG;TRACE /errorendlocation /preferreduilang:en-US /highentropyva+ /reference:/Users/yanchenm/dev/prose/packages/Antlr4.Runtime.4.6.1/lib/net45/Antlr4.Runtime.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.CodeAnalysis.CSharp.1.3.0/lib/net45/Microsoft.CodeAnalysis.CSharp.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.CodeAnalysis.Common.1.3.0/lib/net45/Microsoft.CodeAnalysis.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.Compiler.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Compiler.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Compound.Extraction.Field.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Compound.Split.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Conditionals.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Conditionals.Learning.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Conditionals.Semantics.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.CoreFxLab.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Detection.DataType.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Extraction.Json.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Extraction.Json.Learning.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Extraction.Json.Semantics.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Extraction.Text.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Extraction.Text.Learning.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Extraction.Text.Semantics.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Learning.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Matching.Text.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Matching.Text.Learning.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Matching.Text.Semantics.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Split.File.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Split.File.Learning.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Split.File.Semantics.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Split.Text.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Split.Text.Learning.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Split.Text.Semantics.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Transformation.Json.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Transformation.Json.Learning.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Transformation.Json.Semantics.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Transformation.Text.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Transformation.Text.Semantics.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Utils.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Wrangling.dll /reference:/Users/yanchenm/dev/prose/packages/Microsoft.ProgramSynthesis.3.0.0/lib/net45/Microsoft.ProgramSynthesis.Wrangling.Json.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/mscorlib.dll /reference:/Users/yanchenm/dev/prose/packages/Newtonsoft.Json.8.0.3/lib/net45/Newtonsoft.Json.dll /reference:/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings/bin/Debug/ProseSample.Substrings.dll /reference:/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings.Semantics/bin/Debug/ProseSample.Substrings.Semantics.dll /reference:/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.TextExtraction/bin/Debug/ProseSample.TextExtraction.dll /reference:/Users/yanchenm/dev/prose/packages/System.Collections.Immutable.1.3.0/lib/portable-net45+win8+wp8+wpa81/System.Collections.Immutable.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/System.Core.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/System.dll /reference:/Users/yanchenm/dev/prose/packages/System.Interactive.3.0.0/lib/net45/System.Interactive.dll /reference:/Users/yanchenm/dev/prose/packages/System.Reflection.Metadata.1.3.0/lib/portable-net45+win8/System.Reflection.Metadata.dll /reference:/Users/yanchenm/dev/prose/packages/System.ValueTuple.4.3.1/lib/netstandard1.0/System.ValueTuple.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Collections.Concurrent.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Collections.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.ComponentModel.Annotations.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.ComponentModel.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.ComponentModel.EventBasedAsync.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Diagnostics.Contracts.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Diagnostics.Debug.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Diagnostics.Tools.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Diagnostics.Tracing.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Dynamic.Runtime.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Globalization.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.IO.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Linq.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Linq.Expressions.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Linq.Parallel.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Linq.Queryable.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Net.NetworkInformation.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Net.Primitives.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Net.Requests.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.ObjectModel.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Reflection.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Reflection.Emit.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Reflection.Emit.ILGeneration.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Reflection.Emit.Lightweight.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Reflection.Extensions.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Reflection.Primitives.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Resources.ResourceManager.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Runtime.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Runtime.Extensions.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Runtime.InteropServices.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Runtime.InteropServices.WindowsRuntime.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Runtime.Numerics.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Runtime.Serialization.Json.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Runtime.Serialization.Primitives.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Runtime.Serialization.Xml.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Security.Principal.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.ServiceModel.Duplex.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.ServiceModel.Http.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.ServiceModel.NetTcp.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.ServiceModel.Primitives.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.ServiceModel.Security.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Text.Encoding.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Text.Encoding.Extensions.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Text.RegularExpressions.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Threading.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Threading.Tasks.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Threading.Tasks.Parallel.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Xml.ReaderWriter.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Xml.XDocument.dll /reference:/Library/Frameworks/Mono.framework/Versions/5.0.1/lib/mono/4.5-api/Facades/System.Xml.XmlSerializer.dll /debug+ /debug:portable /filealign:512 /optimize- /out:obj/Debug/ProseSample.exe /subsystemversion:6.00 /target:exe /utf8output /analyzer:../../packages/Microsoft.CodeAnalysis.Analyzers.1.1.0/analyzers/dotnet/cs/Microsoft.CodeAnalysis.Analyzers.dll /analyzer:../../packages/Microsoft.CodeAnalysis.Analyzers.1.1.0/analyzers/dotnet/cs/Microsoft.CodeAnalysis.CSharp.Analyzers.dll Program.cs Properties/AssemblyInfo.cs Utils.cs "/var/folders/6_/kybr84dd5jggnn10dj3t_gcr2yv533/T/.NETFramework,Version=v4.5.AssemblyAttributes.cs"
    CSC : error CS0006: Metadata file '/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings/bin/Debug/ProseSample.Substrings.dll' could not be found
    CSC : error CS0006: Metadata file '/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings.Semantics/bin/Debug/ProseSample.Substrings.Semantics.dll' could not be found
    CSC : error CS0006: Metadata file '/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.TextExtraction/bin/Debug/ProseSample.TextExtraction.dll' could not be found
Done building target "CoreCompile" in project "ProseSample.csproj" -- FAILED.

Done building project "ProseSample.csproj" -- FAILED.

Build FAILED.

CSC : error CS0006: Metadata file '/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings/bin/Debug/ProseSample.Substrings.dll' could not be found
CSC : error CS0006: Metadata file '/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.Substrings.Semantics/bin/Debug/ProseSample.Substrings.Semantics.dll' could not be found
CSC : error CS0006: Metadata file '/Users/yanchenm/dev/prose/ProgramSynthesis/ProseSample.TextExtraction/bin/Debug/ProseSample.TextExtraction.dll' could not be found
    0 Warning(s)
    3 Error(s)

Time Elapsed 00:00:01.47

---------------------- Done ----------------------

Build: 3 errors, 0 warnings

I did not make any changes, not sure how to fix it. Please help! Thank you!
I'm using Mac 10.12.5, and VS 7.0.1.

Unhandled Exception: Unable to cast object

Hi! I have encounter an unhandled exception which I suppose is due to an explicit cast instead of .ToArray()/.ToList() cast. The problem occurs as follows:

I have a grammar that uses the Map standard concept. So something like:

string[] a := A(f, b) = Map(\z:MapArgument => f, b);
MapArgument[] b:= B(....);

Now when I try to synthesise a program, PROSE throws the following exception (stack trace copied at the end): "Unable to cast object of type 'WhereSelectArrayIterator`2[System.Object,System.Object]' to type 'System.String[]'.".

This looks like there is an explicit cast such as (string[]) arr somewhere where it should be arr.ToArray(). Indeed, when I change all string[] to IEnumerable<string> in my grammar and semantics, everything works fine.

Attached is the exception trace:

"Unable to cast object of type 'WhereSelectArrayIterator`2[System.Object,System.Object]' to type 'System.String[]'."

   at lambda_method(Closure , Object[] )
   at Microsoft.ProgramSynthesis.Rules.BlackBoxRule.Evaluate(Object[] args)
   at Microsoft.ProgramSynthesis.AST.NonterminalNode.Evaluate(State state)
   at Microsoft.ProgramSynthesis.AST.ProgramNode.Invoke(State state)
   at Microsoft.ProgramSynthesis.AST.LetNode.Evaluate(State state)
   at Microsoft.ProgramSynthesis.AST.ProgramNode.Invoke(State state)
   at Microsoft.ProgramSynthesis.AST.NonterminalNode.Evaluate(State state)
   at Microsoft.ProgramSynthesis.AST.ProgramNode.Invoke(State state)
   at Microsoft.ProgramSynthesis.VersionSpace.DirectProgramSet.<>c__DisplayClass7_0.<CalculateClusters>b__0(ProgramNode p)
   at System.Linq.Lookup`2.Create[TSource](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at System.Linq.GroupedEnumerable`3.GetEnumerator()
   at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at Microsoft.ProgramSynthesis.VersionSpace.DirectProgramSet.CalculateClusters(State inputState)
   at Microsoft.ProgramSynthesis.VersionSpace.ProgramSet.ClusterOnInput(State inputState)
   at Microsoft.ProgramSynthesis.VersionSpace.ProgramSet.CalculateClusters(State[] inputStates)
   at Microsoft.ProgramSynthesis.VersionSpace.ProgramSet.ClusterOnInputTuple(IEnumerable`1 inputStates)
   at Microsoft.ProgramSynthesis.Learning.Strategies.DeductiveSynthesis.BuildFromDependencies(WitnessingPlan plan, List`1 order, Int32 current, IDictionary`2 graph, ImmutableDictionary`2 prereqsBindings, SynthesisEngine engine, CancellationToken cancel, NonterminalRule rule, LearningTask`1 task)
   at Microsoft.ProgramSynthesis.Learning.Strategies.DeductiveSynthesis.BuildFromDependencies(WitnessingPlan plan, List`1 order, Int32 current, IDictionary`2 graph, ImmutableDictionary`2 prereqsBindings, SynthesisEngine engine, CancellationToken cancel, NonterminalRule rule, LearningTask`1 task)
   at Microsoft.ProgramSynthesis.Learning.Strategies.DeductiveSynthesis.LearnRuleFromWitnesses(SynthesisEngine engine, NonterminalRule rule, LearningTask`1 task, CancellationToken cancel)
   at Microsoft.ProgramSynthesis.Learning.Strategies.DeductiveSynthesis.LearnSymbol(SynthesisEngine engine, LearningTask`1 task, CancellationToken cancel)
   at Microsoft.ProgramSynthesis.Learning.Strategies.SynthesisStrategy`2.Learn(SynthesisEngine engine, LearningTask task, CancellationToken cancel)
   at Microsoft.ProgramSynthesis.Learning.SynthesisEngine.<>c__DisplayClass25_2.<LearnSymbol>b__0()
   at Microsoft.ProgramSynthesis.Learning.SynthesisEngine.InvokeStrategy[TResult](Func`1 learner)
   at Microsoft.ProgramSynthesis.Learning.SynthesisEngine.LearnSymbol(LearningTask task, Nullable`1 cancel)
   at Microsoft.ProgramSynthesis.Learning.Strategies.DeductiveSynthesis.LearnRuleFromWitnesses(SynthesisEngine engine, NonterminalRule rule, LearningTask`1 task, CancellationToken cancel)
   at Microsoft.ProgramSynthesis.Learning.Strategies.DeductiveSynthesis.LearnSymbol(SynthesisEngine engine, LearningTask`1 task, CancellationToken cancel)
   at Microsoft.ProgramSynthesis.Learning.Strategies.SynthesisStrategy`2.Learn(SynthesisEngine engine, LearningTask task, CancellationToken cancel)
   at Microsoft.ProgramSynthesis.Learning.SynthesisEngine.<>c__DisplayClass25_2.<LearnSymbol>b__0()
   at Microsoft.ProgramSynthesis.Learning.SynthesisEngine.InvokeStrategy[TResult](Func`1 learner)
   at Microsoft.ProgramSynthesis.Learning.SynthesisEngine.LearnSymbol(LearningTask task, Nullable`1 cancel)
   at Microsoft.ProgramSynthesis.Learning.SynthesisEngine.LearnSymbol(Symbol symbol, Spec spec, Nullable`1 cancel)
   at Microsoft.ProgramSynthesis.Learning.SynthesisEngine.LearnGrammar(Spec spec, Nullable`1 cancel)
   at Synthesis.Program.Main(String[] args) in z:\Repos\...
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

Silent fail of LearnSequenceReferencingSibling example in WranglingSamples

The LearnSequenceReferencingSibling example in WranglingSamples does not print anything. In the following code (linked here: https://github.com/Microsoft/prose/blob/master/Extraction.Text/LearningSamples.cs#L388-L393), instead of iterating over the regions as in prior documentation examples found in https://microsoft.github.io/prose/documentation/extraction-text/usage/#extracting-a-sequence-of-stringregions
like this:

foreach (var r in topRankedProg.Run(countries))
{
    var output = r.Output != null ? r.Output.Value : "null";
    Console.WriteLine("\"{0}\" => \"{1}\"", r.Reference, output);
}

it iterates over a much more complicated expression, which returns an empty list:

foreach (var a in areas
                .SelectMany(area => topRankedProg.Run(area)
                                                    .Select(output => new { Input = area, Output = output }))) {
                var output = a.Output != null ? a.Output.Value : "null";
                Console.WriteLine("\"{0}\" => \"{1}\"", a.Input, output);
            }

I tried changing this example to iterate over topRankedProg.Run(areas) so it's more like the example in the documentation, but the object returned by topRankedProgram.Run( ) does not out an output field.
Help?

Using Filter with Kth

Hi PROSE team! I've been working on creating a DSL for web scraping and it currently supports various tree/list operations to find data in a DOM tree. For selecting/filtering elements in my "node lists" I'm currently using the built-in functions Kth and Filter. My current DSL is shown below:

@input ProseHtmlNode tree;

@start IReadOnlyList<ProseHtmlNode> program := rule;

IReadOnlyList<ProseHtmlNode> rule :=
	:= Concat(rule, rule)
	 | MatchNodes(match, nodes) = Filter(\x: ProseHtmlNode => match, nodes)
	 | nodes

IReadOnlyList<ProseHtmlNode> nodes 
	:= Children(subTree)
         | Descendants(subTree)
	 | Single(subTree)

ProseHtmlNode subTree 
	:= tree
	 | SelectChild(rule, k) = Kth(rule, k)

bool match 
	:= MatchTag(x, tag)
	 | MatchAttribute(x, attr)
	 | True()

This is then able to generate programs like Single(SelectChild(Descendents(tree), 1)) and MatchNodes(MatchTag(x, "div"), Descendants(tree)) but it fails when trying to select the Kth element out of a filtered list. For something like "Give the first node with the tag of 'div'" I'm hoping to generate something like:
Single(SelectChild(MatchNodes(MatchTag(x, "div"), Descendants(tree)), 0))

I currently don't have witness functions for either SelectChild or MatchNodes as I wasn't sure if I needed to write them for built-in functions (there was some mention that the framework could more aggressively optimize the built-ins so I didn't want to interfere with that). I have noticed that it gets close when calling the witness function for MatchTag but the incoming args are never really satisfiable by the predicate. For example, if I was searching for the first "div" node, Filter will clearly be attempting to filter the correct list that contains that "div" node, but MatchNode will get examples like:

("div", true),
("exclude", false),
("div", false),
("div", false)

Which, with the way I have MatchTag implemented, won't contain a single tag that satisfies all the examples. It seems like for this to work, MatchTag would need to receive the examples:

("div", true),
("exclude", false),
("div", true),
("div", true)

and have SelectChild select only the first of the filtered list, but I'm not sure what I need to change to make that happen.

The tree synthesis portion of the repo can be found here if you'd like to take a look: https://github.com/zachwood0s/dom_program_synthesis/tree/main/ProseTutorial/tree_synthesis

Any help or suggestions on how to get this interaction to work would be greatly appreciated!

Error in creating my own DSL

Hello,
I try to rewrite tutorial to create my own DSL, and when I run Program.cs, I got a NullReferenceException in
var prose = new SynthesisEngine(Grammar, synthesisConfig);
and I find Grammar is null.
I found I didn't provide score functions for my DSL, but is it neccessary to provide score function for every components in DSL? Can I use some default score function?
Thank you!

microsoft / prose Goto Github PK

prose's Introduction

Microsoft Program Synthesis using Examples SDK

Contributing

prose's People

Contributors

Stargazers

Watchers

Forkers

prose's Issues

Steps to reproduce:

Recommend Projects

Recommend Topics

Recommend Org