microsoftresearch / naiad Goto Github PK

There is a bug in LegacyLocalMailbox.Flush(RemoteAddress) that allows two worker threads to execute vertex code (in particular, the VertexInput.Flush() method). This causes unsafe access to the contents of the various message buffers in downstream mailboxes, which can lead to data corruption and crashes.

Storage account keys should be fetched automatically for accounts attached to the user's default subscription. The submission client is running as the user, and can fetch them automatically. Getting them to the user executable running on the cluster is trickier. Perhaps the user should store desired storage account names in the app.config, and the submission client could populate them with keys as part of submission.

It seems that Naiad jobs run with only a single thread in Azure, even if a larger number of threads is requested.

Crashes with an exception after uploading files:

Exception The cluster version 3.1.0.227 is not supported by this version of the tools. Please upgrade your tools to the newer version. The compatibility range for these tools is 1.6 - 3.0.
System.NotSupportedException: The cluster version 3.1.0.227 is not supported by this version of the tools. Please upgrade your tools to the newer version. The compatibility range for these tools is 1.6 - 3.0.

Server stack trace:
   at Microsoft.WindowsAzure.Management.HDInsight.JobSubmission.HDInsightHadoopClient.AssertSupportedVersion(Version hdinsightClusterVersion, IVersionFinderClient versionFinderClient)
   at Microsoft.WindowsAzure.Management.HDInsight.JobSubmission.HDInsightHadoopClient.GetJobSubmissionClusterDetails(Boolean ignoreCache)
   at Microsoft.WindowsAzure.Management.HDInsight.JobSubmission.HDInsightHadoopClient.<CreateStreamingJobAsync>d__e.MoveNext()

Exception rethrown at [0]:
   at Microsoft.WindowsAzure.Management.HDInsight.Framework.Core.Library.TaskExtensions.WaitForResult[T](Task`1 task, TimeSpan timeout)
   at Microsoft.WindowsAzure.Management.HDInsight.Framework.Core.Library.TaskExtensions.WaitForResult[T](Task`1 task)
   at Microsoft.WindowsAzure.Management.HDInsight.JobSubmission.HDInsightHadoopClient.CreateStreamingJob(StreamingMapReduceJobCreateParameters streamingMapReduceJobCreateParameters)
   at Microsoft.Research.Peloponnese.ClusterUtils.AzureYarnClient.Submit(XDocument config, String errorLocation) in d:\Users\misard\src\peloponnese\Utils\ClusterClient.cs:line 386
   at Microsoft.Research.Naiad.Cluster.ClusterSubmission.Submit() in c:\gits\naiad\ClusterSubmission\NaiadPeloponneseSupport\ClusterSubmission.cs:line 157
   at Microsoft.Research.Naiad.Cluster.Azure.Program.Run(String[] args) in c:\gits\naiad\ClusterSubmission\AzureSubmission\Program.cs:line 153
   at Microsoft.Research.Naiad.Cluster.Azure.Program.Main(String[] args) in c:\gits\naiad\ClusterSubmission\AzureSubmission\Program.cs:line 161

The current inline serializer uses .NET serialization for strings, collections, tuples and other framework classes. (The default serializer specializes strings and other classes, but is less efficient for simple structs.) It would be worthwhile to specialize some common cases, in advance of deciding on a more pluggable serializer.

When running on Mono with 3 or more processes, a Naiad computation will deadlock in the controller initialization phase.

To reproduce:

$ ./Examples.exe connectedcomponents 1000000 2000000 -t 1 -n 3 --local -p 0 &
$ ./Examples.exe connectedcomponents 1000000 2000000 -t 1 -n 3 --local -p 1 &
$ ./Examples.exe connectedcomponents 1000000 2000000 -t 1 -n 3 --local -p 2 &

The same program works for -n 2 -p {0, 1}. Thanks to the folks at ETH Zurich for reporting this issue.

Currently, the scheduler catches any exceptions in OnRecv or OnNotify callbacks, and invokes InternalComputation.Cancel(), which should terminate execution, by signaling the FrontierEmpty event. However, BaseComputation.Join() now blocks on a CountdownEvent that is signaled by each Scheduler on successful completion of Shutdown work, and this is not signaled in the exception case.

Just wondering why you have this rather than using Tuple?

Hello Naiad Team!
I download the code and try to build the solution. But following errors occurs. Can you please give me some guidance how to configure the build setting? I follow the readme in the project but it doesn't work for me.

Severity Code Description Project File Line
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found Storage E:\info\naiad\Naiad-release_0.5\frameworks\Storage\CSC
Error CS1629 Unsafe code may not appear in iterators Naiad E:\info\naiad\Naiad-release_0.5\Naiad\CodeGeneration\AutoSerialization.cs 1242
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found Lindi E:\info\naiad\Naiad-release_0.5\Frameworks\Lindi\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found WorkGenerator E:\info\naiad\Naiad-release_0.5\Frameworks\WorkGenerator\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\Lindi\bin\Debug\Microsoft.Research.Naiad.Lindi.dll' could not be found Storage E:\info\naiad\Naiad-release_0.5\frameworks\Storage\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\WorkGenerator\bin\Debug\Microsoft.Research.Naiad.WorkGenerator.dll' could not be found Storage E:\info\naiad\Naiad-release_0.5\frameworks\Storage\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found AzureSupport E:\info\naiad\Naiad-release_0.5\Frameworks\AzureSupport\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\Lindi\bin\Debug\Microsoft.Research.Naiad.Lindi.dll' could not be found AzureSupport E:\info\naiad\Naiad-release_0.5\Frameworks\AzureSupport\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\frameworks\Storage\bin\Debug\Microsoft.Research.Naiad.Storage.dll' could not be found AzureSupport E:\info\naiad\Naiad-release_0.5\Frameworks\AzureSupport\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found DifferentialDataflow E:\info\naiad\Naiad-release_0.5\Frameworks\DifferentialDataflow\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\Lindi\bin\Debug\Microsoft.Research.Naiad.Lindi.dll' could not be found DifferentialDataflow E:\info\naiad\Naiad-release_0.5\Frameworks\DifferentialDataflow\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found GraphLINQ E:\info\naiad\Naiad-release_0.5\Frameworks\GraphLINQ\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\Lindi\bin\Debug\Microsoft.Research.Naiad.Lindi.dll' could not be found GraphLINQ E:\info\naiad\Naiad-release_0.5\Frameworks\GraphLINQ\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\AzureSupport\bin\Debug\Microsoft.Research.Naiad.AzureSupport.dll' could not be found AzureExamples E:\info\naiad\Naiad-release_0.5\AzureExamples\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found AzureExamples E:\info\naiad\Naiad-release_0.5\AzureExamples\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\GraphLINQ\bin\Debug\Microsoft.Research.Naiad.GraphLINQ.dll' could not be found AzureExamples E:\info\naiad\Naiad-release_0.5\AzureExamples\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\Lindi\bin\Debug\Microsoft.Research.Naiad.Lindi.dll' could not be found AzureExamples E:\info\naiad\Naiad-release_0.5\AzureExamples\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\DifferentialDataflow\bin\Debug\Microsoft.Research.Naiad.DifferentialDataflow.dll' could not be found Examples E:\info\naiad\Naiad-release_0.5\Examples\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found Examples E:\info\naiad\Naiad-release_0.5\Examples\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\GraphLINQ\bin\Debug\Microsoft.Research.Naiad.GraphLINQ.dll' could not be found Examples E:\info\naiad\Naiad-release_0.5\Examples\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\Lindi\bin\Debug\Microsoft.Research.Naiad.Lindi.dll' could not be found Examples E:\info\naiad\Naiad-release_0.5\Examples\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found HdfsSupport E:\info\naiad\Naiad-release_0.5\frameworks\HdfsSupport\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\Lindi\bin\Debug\Microsoft.Research.Naiad.Lindi.dll' could not be found HdfsSupport E:\info\naiad\Naiad-release_0.5\frameworks\HdfsSupport\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\frameworks\Storage\bin\Debug\Microsoft.Research.Naiad.Storage.dll' could not be found HdfsSupport E:\info\naiad\Naiad-release_0.5\frameworks\HdfsSupport\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\WorkGenerator\bin\Debug\Microsoft.Research.Naiad.WorkGenerator.dll' could not be found HdfsSupport E:\info\naiad\Naiad-release_0.5\frameworks\HdfsSupport\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\AzureSupport\bin\Debug\Microsoft.Research.Naiad.AzureSupport.dll' could not be found NugetSample E:\info\naiad\Naiad-release_0.5\NugetSample\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\DifferentialDataflow\bin\Debug\Microsoft.Research.Naiad.DifferentialDataflow.dll' could not be found NugetSample E:\info\naiad\Naiad-release_0.5\NugetSample\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found NugetSample E:\info\naiad\Naiad-release_0.5\NugetSample\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Naiad\bin\Debug\Microsoft.Research.Naiad.dll' could not be found WebHdfsSupport E:\info\naiad\Naiad-release_0.5\Frameworks\WebHdfsSupport\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\Lindi\bin\Debug\Microsoft.Research.Naiad.Lindi.dll' could not be found WebHdfsSupport E:\info\naiad\Naiad-release_0.5\Frameworks\WebHdfsSupport\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\frameworks\Storage\bin\Debug\Microsoft.Research.Naiad.Storage.dll' could not be found WebHdfsSupport E:\info\naiad\Naiad-release_0.5\Frameworks\WebHdfsSupport\CSC
Error Metadata file 'E:\info\naiad\Naiad-release_0.5\Frameworks\WorkGenerator\bin\Debug\Microsoft.Research.Naiad.WorkGenerator.dll' could not be found WebHdfsSupport E:\info\naiad\Naiad-release_0.5\Frameworks\WebHdfsSupport\CSC

Hello,

We are running calculations on Naiad and computation fails on large datasets. For single thread and small datasets, there is no problem. We running Naiad on virtual machine in Mac.
Can you help to find out reason ?

Thank you.

Error :
Logging initialized to console
00:07:57.5913840, Graph 0 failed on scheduler 1 with exception:
System.NullReferenceException: Object reference not set to an instance of an obj
ect.
at Microsoft.Research.Naiad.Dataflow.VertexOutputBufferPerTime2.Send(TRecord record) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Frameworks\StandardVertic es.cs:line 313 at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.Conside rFlushingBufferedUpdates() in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime \Progress\UpdateAggregator.cs:line 157 at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.OnRecv( Dictionary2 deltas) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Progr
ess\UpdateAggregator.cs:line 77
at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateProducer.Start() i
n c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Progress\UpdateProducer.cs:
line 92
at Microsoft.Research.Naiad.Scheduling.Scheduler.DrainMessagesForComputation(
Int32 computationIndex) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Sc
heduling\Scheduler.cs:line 343
00:07:57.5941910, Cancelling execution of graph 0, due to exception:
System.NullReferenceException: Object reference not set to an instance of an obj
ect.
at Microsoft.Research.Naiad.Dataflow.VertexOutputBufferPerTime2.Send(TRecord record) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Frameworks\StandardVertic es.cs:line 313 at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.Conside rFlushingBufferedUpdates() in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime \Progress\UpdateAggregator.cs:line 157 at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.OnRecv( Dictionary2 deltas) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Progr
ess\UpdateAggregator.cs:line 77
at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateProducer.Start() i
n c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Progress\UpdateProducer.cs:
line 92
at Microsoft.Research.Naiad.Scheduling.Scheduler.DrainMessagesForComputation(
Int32 computationIndex) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Sc
heduling\Scheduler.cs:line 343

Logging initialized to console
00:07:57.5913840, Graph 0 failed on scheduler 1 with exception:
System.NullReferenceException: Object reference not set to an instance of an obj
ect.
at Microsoft.Research.Naiad.Dataflow.VertexOutputBufferPerTime2.Send(TRecord record) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Frameworks\StandardVertic es.cs:line 313 at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.Conside rFlushingBufferedUpdates() in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime \Progress\UpdateAggregator.cs:line 157 at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.OnRecv( Dictionary2 deltas) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Progr
ess\UpdateAggregator.cs:line 77
at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateProducer.Start() i
n c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Progress\UpdateProducer.cs:
line 92
at Microsoft.Research.Naiad.Scheduling.Scheduler.DrainMessagesForComputation(
Int32 computationIndex) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Sc
heduling\Scheduler.cs:line 343
00:07:57.5941910, Cancelling execution of graph 0, due to exception:
System.NullReferenceException: Object reference not set to an instance of an obj
ect.
at Microsoft.Research.Naiad.Dataflow.VertexOutputBufferPerTime2.Send(TRecord record) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Frameworks\StandardVertic es.cs:line 313 at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.Conside rFlushingBufferedUpdates() in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime \Progress\UpdateAggregator.cs:line 157 at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.OnRecv( Dictionary2 deltas) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Progr
ess\UpdateAggregator.cs:line 77
at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateProducer.Start() i
n c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Progress\UpdateProducer.cs:
line 92
at Microsoft.Research.Naiad.Scheduling.Scheduler.DrainMessagesForComputation(
Int32 computationIndex) in c:\Users\khgl\Desktop\codebase\Naiad\Naiad\Runtime\Sc
heduling\Scheduler.cs:line 343

I just want to run a pagerank program(Naiad/Examples/GraphLINQ/PageRank.cs) for graph analysis.When I run for a 1GB-Size graph,everything goes well; but when I run for a 3.6G-size graph(about 260,000,000 edges),the system prompted an exception followed:
Reading file C:\Naiad-release_0.5\rmat24.el
00:02:56.2408081, Graph 0 failed on scheduler 0 with exception:
System.OutOfMemoryException: Array dimension beyond the scope of support.
在 System.Collections.Generic.List1.set_Capacity(Int32 value) 在 System.Collections.Generic.List1.EnsureCapacity(Int32 min)
在 System.Collections.Generic.List1.Add(T item) 在 Microsoft.Research.Naiad.Frameworks.GraphLINQ.GraphCompactor1.OnReceive(Message2 message) 位置 C:\Naiad-release_0.5\Frameworks\GraphLINQ\GraphLINQ.cs:行号 1199 在 Microsoft.Research.Naiad.Dataflow.StandardVertices.UnaryVertex3.<>c.b__2_0(Message2 message, UnaryVertex3 vertex) 位置 C:\Naiad-release_0.5\Naiad\Frameworks\StandardVertices.cs:行号 446
在 Microsoft.Research.Naiad.Dataflow.Stage2.<>c__DisplayClass11_11.b__1(Message2 m) 位置 C:\Naiad-release_0.5\Naiad\Dataflow\Stage.cs:行号 383 在 Microsoft.Research.Naiad.Dataflow.ActionReceiver2.<>c__DisplayClass3_0.<.ctor>b__0(Message2 m, ReturnAddress u) 位置 C:\Naiad-release_0.5\Naiad\Dataflow\Endpoints.cs:行号 327 在 Microsoft.Research.Naiad.Dataflow.ActionReceiver2.OnReceive(Message`2 message, ReturnAddress from) 位置 C:\Naiad-release_0.5\Naiad\Dataflow\Endpoints.cs:行号 317
..........
how could I fixed this problem.I have tried to build the program on X86 and X64 platform,but could not fix the problem.
Thanks!

Due to a difference in the type inference implementation between .NET and Mono, the Mono build crashes with the following error:

        Naiad/ConnectedComponents.cs(77,49): error CS0315: The type `Microsoft.Research.Naiad.Dataflow.IterationIn<TTime>' cannot be used as type parameter `TTime' in the generic type or method `Microsoft.Research.Naiad.Dataflow.StandardVertices.UnaryVertex<TInput,TOutput,TTime>'. There is no boxing conversion from `Microsoft.Research.Naiad.Dataflow.IterationIn<TTime>' to `Microsoft.Research.Naiad.Time<Microsoft.Research.Naiad.Dataflow.IterationIn<TTime>>'

The corresponding code looks like this:

        public static Stream<Pair<TKey, TValue>, IterationIn<TTime>> StreamingAggregate<TKey, TValue, TTime>(this Stream<Pair<TKey, TValue>, IterationIn<TTime>> input, Func<TValu
e, TValue, TValue> aggregate)
            where TTime : Time<TTime>
            where TValue : IEquatable<TValue>
        {
            return Foundry.NewUnaryStage(input, (i, s) => new StreamingAggregateVertex<TKey, TValue, IterationIn<TTime>>(i, s, aggregate), x => x.First.GetHashCode(), x => x.First.GetHashCode(), "StreamingAggregate");
        }

Although this (somewhat exotic) type constraint is appropriately specific, it can be relaxed in the interests of compatibility (either to working on streams of any time type, or streams of the specific IterationIn<Epoch> time type).

Hi Naiad team,

I'm a bit late to the party, but after finally watching the presentation video from SOSP, I was very excited to learn that Naiad is available open source. I'm excited to try it out.

My main development environment is not a windows machine, but I noticed there is a script called "build_mono.sh". I was wondering if you've had any success building it? It might be something else I am doing wrong, but when I try to build using this script, it fails because it cannot find "EventTask", which should be under System.Diagnostics.Tracing. I'm using xbuild version 12.0.0, and Mono 3.10.0.0. As further evidence, the xamarin API doesn't seem to have this class.

Finally, in light of the somewhat recent and very shocking announcement from Microsoft about MSR-SV, do you have any idea what impact this might have on future support for Naiad? I'm very sorry to all of the folks who were impacted.

Thanks,
Alvin

Invoking Examples.exe wordcount -p 2 -n 2 --local with the latest commit (360ff09) causes an ArgumentOutOfRangeException.

The issue was already occurring in release_0.4.

Exception Details:

System.IndexOutOfRangeException was unhandled
  HResult=-2146233080
  Message=Index was outside the bounds of the array.
  Source=Microsoft.Research.Naiad
  StackTrace:
       at Microsoft.Research.Naiad.BaseController..ctor(Configuration config) in d:\Schuldateien\SS 14\Sources\naiad\Naiad\Runtime\Controlling\Controller.cs:line 771
       at Microsoft.Research.Naiad.NewController.FromConfig(Configuration conf) in d:\Schuldateien\SS 14\Sources\naiad\Naiad\Runtime\Controlling\Controller.cs:line 283
       at Microsoft.Research.Naiad.InternalOneOffComputation..ctor(Configuration configuration) in d:\Schuldateien\SS 14\Sources\naiad\Naiad\Runtime\SubgraphManager.cs:line 67
       at Microsoft.Research.Naiad.NewComputation.FromArgs(String[]& args) in d:\Schuldateien\SS 14\Sources\naiad\Naiad\Runtime\Controlling\Controller.cs:line 220
       at Microsoft.Research.Naiad.Examples.WordCount.WordCount.Execute(String[] args) in d:\Schuldateien\SS 14\Sources\naiad\Examples\Naiad\WordCount.cs:line 98
       at Microsoft.Research.Naiad.Examples.Program.Main(String[] args) in d:\Schuldateien\SS 14\Sources\naiad\Examples\Program.cs:line 105
       at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
  InnerException: 

Currently, these projects carry a dependency on the WindowsAzure.Storage NuGet package, which is not fetched automatically by the build process.

StreamingInputVertex.PerformAction holds a lock on the object while it processes all of its input records (calls to output.Send()). This blocks any threads trying to enqueue work items. It seems that the lock should be able to held for less time, only related to variables/state that might be modified by other threads.

I am looking to do Tumbling Windows similar to what String Insight dose.

http://blogs.msdn.com/b/streaminsight/archive/2010/11/23/windows-part-1-the-basics.aspx

I do not currently see the operator in Naiad source.

Is it planned?

If not can you give guidance on how best to build it.

Hello Naiad developers,

We're using Naiad for a university project and we don't seem to be able to figure out how to pass non-primitive types between vertices. As advised in the readme we're using structs but it seems like messages are sent and never received.

We're still working with the release_0.4 branch since release_0.5 does not work with mono.

Here's the code to reproduce the issue https://gist.github.com/utaal/c10e450e1227f7ef5462 .
Please drop it in /Examples/Naiad, add examples.Add ("serialization", new Serialization.Serialization ()); to Program::Main. TestVertex2 is not receiving messages.

Is there any other known issue with the serialization framework? Are we using it incorrectly?

Thank you.

It appears that the default behavior of Mono's CodeDOM is to treat warnings as errors, even when the TreatWarningsAsErrors option is set to false. Running Naiad in distributed mode yields the following type of error:

Fatal: errors occurred during the generation of a serializer for type Microsoft.Research.Naiad.Serialization.MessageHeader:
/tmp/4jebzyce.gpz/ReleaseObjects/5b99a60e.0.cs(70,11) : warning CS0219: The variable `bytesRemaining' is assigned but its value is never used
/tmp/4jebzyce.gpz/ReleaseObjects/5b99a60e.0.cs(145,11) : warning CS0219: The variable `bytesRemaining' is assigned but its value is never used
/tmp/4jebzyce.gpz/ReleaseObjects/5b99a60e.0.cs(200,11) : warning CS0219: The variable `bytesRemaining' is assigned but its value is never used


Unhandled Exception:
System.Exception: Code generation for Microsoft.Research.Naiad.Serialization.MessageHeader failed
  at Microsoft.Research.Naiad.Serialization.AutoSerialization+SerializationCodeGeneratorForType.GenerateSerializer[MessageHeader] () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Serialization.BaseSerializationCodeGenerator.GetSerializer[MessageHeader] () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Runtime.Networking.TcpNetworkChannel.get_HeaderSerializer () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Runtime.Networking.TcpNetworkChannel.AnnounceStartup (Int32 barrierId) [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Runtime.Networking.TcpNetworkChannel.DoStartupBarrier () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.BaseController.DoStartupBarrier () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.BaseComputation.Activate () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.InternalOneOffComputation.Activate () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Examples.Throughput.Throughput.Execute (System.String[] args) [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Examples.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0

The culprit is a stray declaration of a "bytesRemaining" variable in the TrySerializeMany method of each generated serializer.

If a program decides to create a Naiad computation (either from Controller.NewComputation() or using a OneOffComputation), there is no way to cancel that computation, and no way to release the resources consumed by the underlying Controller.

Hi Team Naiad,

I am fairly new to Naiad (and must say that it sounds really cool!). I was trying to write a simple Console Application in Visual Studio, where given a set of words, I can find the length of the smallest word. With each epoch of data, I want to calculate and return the length of the smallest word. To do so in Naiad,I wrote the following snippet:

var words = computation.NewInputCollection();
words.SelectMany(x => x.Split(' '))
.Min(w => w, w => w.Length )
.Subscribe(l => { foreach (var element in l) Console.WriteLine(element); });

With this snippet, I am not getting any word lengths at all, and I wanted to ask if I am using the functions right. Any advise/ help would be greatly appreciated!
Thanks in advance!

Best Regards,
Tanya

Hi,

The following exception occasionally takes down our computation. We don't really know how to reproduce it in isolation as it happens only when running on large input datasets and after a few minutes of execution. I am not yet sure that our computation is fully deterministic when run multiple times on the same input (we're verifying this now) so the fact that this happens occasionally may be due to non-deterministic behaviour in our code.
"ParsedInputPono" (renamed) is a plain-old-NET-object with a few public fields (of type long, int, string) and a constructor that only copies the arguments to the respective fields.
We're running a debug build on linux with mono (I'll try to rerun with --debug to get line numbers) on about 16 machines with 4 workers each (-t 4).
Can you help us out?

Thank you.

00:07:58.2570129, Graph 0 failed on scheduler 3 with exception:
System.NullReferenceException: Object reference not set to an instance of an object
  at Microsoft.Research.Naiad.Serialization.AutoGenerated.ParsedInputPono.TrySerializeMany (Microsoft.Research.Naiad.Serialization.SubArray`1& destination, ArraySegment`1 values) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Serialization.SendBufferPage.WriteElements[ParsedInputPono] (NaiadSerialization`1 serializer, ArraySegment`1 elements) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Serialization.AutoSerializedMessageEncoder`2[ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].WriteElements (ArraySegment`1 records, Int32 srcVertexId) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Serialization.AutoSerializedMessageEncoder`2[ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].Write (ArraySegment`1 records, Int32 srcVertexId) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.Channels.RemoteMailbox`2[ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].Send (Message`2 message, ReturnAddress from) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.Channels.BufferingPostbox`2[ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].FlushBuffer (Int32 index) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.Channels.BufferingPostbox`2[ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].InternalSend (Message`2 records) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.Channels.BufferingPostbox`2[ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].Send (Message`2 records) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.VertexOutputBuffer`2[ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].Send (Message`2 message) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.VertexOutputBufferPerTime`2[ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].SendBuffer () [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.VertexOutputBufferPerTime`2[ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].Send (ParsedInputPono record) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Frameworks.Lindi.ExtensionMethods+SelectManyVertex`3[System.String,ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].OnReceive (Message`2 message) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.StandardVertices.UnaryVertex`3[System.String,ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch].<MakeStage>m__0 (Message`2 message, Microsoft.Research.Naiad.Dataflow.StandardVertices.UnaryVertex`3 vertex) [0x00000] in <filename unknown>:0 
  at (wrapper delegate-invoke) System.Action`2<Microsoft.Research.Naiad.Dataflow.Message`2<string, Microsoft.Research.Naiad.Dataflow.Epoch>, Microsoft.Research.Naiad.Dataflow.StandardVertices.UnaryVertex`3<string, ParsedInputPono, Microsoft.Research.Naiad.Dataflow.Epoch>>:invoke_void_T1_T2 (Microsoft.Research.Naiad.Dataflow.Message`2<string, Microsoft.Research.Naiad.Dataflow.Epoch>,Microsoft.Research.Naiad.Dataflow.StandardVertices.UnaryVertex`3<string, ParsedInputPono, Microsoft.Research.Naiad.Dataflow.Epoch>)
  at Microsoft.Research.Naiad.Dataflow.Stage`2+<NewInput>c__AnonStorey2`1+<NewInput>c__AnonStorey3[Microsoft.Research.Naiad.Dataflow.StandardVertices.UnaryVertex`3[System.String,ParsedInputPono,Microsoft.Research.Naiad.Dataflow.Epoch],Microsoft.Research.Naiad.Dataflow.Epoch,System.String].<>m__0 (Message`2 m) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.ActionReceiver`2+<ActionReceiver>c__AnonStorey0[System.String,Microsoft.Research.Naiad.Dataflow.Epoch].<>m__0 (Message`2 m, ReturnAddress u) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.ActionReceiver`2[System.String,Microsoft.Research.Naiad.Dataflow.Epoch].OnReceive (Message`2 message, ReturnAddress from) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.Channels.PipelineChannel`2+Fiber[System.String,Microsoft.Research.Naiad.Dataflow.Epoch].Send (Message`2 records) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.VertexOutputBuffer`2[System.String,Microsoft.Research.Naiad.Dataflow.Epoch].Send (Message`2 message) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.VertexOutputBufferPerTime`2[System.String,Microsoft.Research.Naiad.Dataflow.Epoch].SendBuffer () [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.VertexOutputBufferPerTime`2[System.String,Microsoft.Research.Naiad.Dataflow.Epoch].Send (System.String record) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Dataflow.StreamingInputVertex`1[System.String].PerformAction (WorkItem workItem) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Scheduling.Scheduler+WorkItem.Run () [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Scheduling.Scheduler.Schedule (WorkItem workItem) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Scheduling.Scheduler.RunWorkItem (Int32 graphId) [0x00000] in <filename unknown>:0 
  at Microsoft.Research.Naiad.Scheduling.Scheduler.RunNotification (Int32 computationIndex) [0x00000] in <filename unknown>:0 

Hi Naiad developers, the iterative model in Naiad is quite attractive. But I am wondering what if the previous round of computation keeps going but the new event comes in? For example, in connected component case, let's say edge <i, j> comes in and triggers some computation. Before the computation ends, edge <m, n> arrives. So will the edge <m, n> be buffered? Or the two rounds of computations can be performed simultaneously? Thanks!

Hi again,

I've encountered another couple of issues when running https://gist.github.com/utaal/96c4328fbc48a07d69b4 on mono.
First, in distributed mode with two processes, process 1 does not seem to actually block on Sync. The following is the output of process 1.

Logging initialized to console
Pushing data
Syncing 0
00:00:02.7292913, Syncing at epoch (0) that is in the future of __Input0
Synced 0
Syncing 1
00:00:02.7400550, Syncing at epoch (1) that is in the future of __Input0
Synced 1
Syncing 2
00:00:02.7401329, Syncing at epoch (2) that is in the future of __Input0
Synced 2
Syncing 3
00:00:02.7401982, Syncing at epoch (3) that is in the future of __Input0
Synced 3
Syncing 4
00:00:02.7407855, Syncing at epoch (4) that is in the future of __Input0
Synced 4
Syncing 5
00:00:02.7408682, Syncing at epoch (5) that is in the future of __Input0
Synced 5
Syncing 6
00:00:02.7409424, Syncing at epoch (6) that is in the future of __Input0
Synced 6
Syncing 7
00:00:02.7410222, Syncing at epoch (7) that is in the future of __Input0
Synced 7
Syncing 8
00:00:02.7410966, Syncing at epoch (8) that is in the future of __Input0
Synced 8
Syncing 9
00:00:02.7411738, Syncing at epoch (9) that is in the future of __Input0
Synced 9
(...)

I was watching the output of process 0 and by this time the subscription did not even finish outputting data for epoch 1.
Is this behavior somehow expected? Do you believe it may be mono-specific?

Moreover, occasionally, when running with -t > 1 we either get the following exception mid-computation - all the processes stop working immediately after this - or the computation just gets stuck.

00:00:03.2077718, Graph 0 failed on scheduler 0 with exception:
System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
  at System.Collections.Generic.Dictionary`2[Microsoft.Research.Naiad.Runtime.Progress.Pointstamp,System.Int64].get_Item (Pointstamp key) [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.ConsiderFlushingBufferedUpdates () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.OnRecv (System.Collections.Generic.Dictionary`2 deltas) [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateProducer.Start () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Scheduling.Scheduler.RunWorkItem (Int32 graphId) [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Scheduling.Scheduler.InternalStart () [0x00000] in <filename unknown>:0
00:00:03.2084448, Cancelling execution of graph 0, due to exception:
System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
  at System.Collections.Generic.Dictionary`2[Microsoft.Research.Naiad.Runtime.Progress.Pointstamp,System.Int64].get_Item (Pointstamp key) [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.ConsiderFlushingBufferedUpdates () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateAggregator.OnRecv (System.Collections.Generic.Dictionary`2 deltas) [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Runtime.Progress.ProgressUpdateProducer.Start () [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Scheduling.Scheduler.RunWorkItem (Int32 graphId) [0x00000] in <filename unknown>:0
  at Microsoft.Research.Naiad.Scheduling.Scheduler.InternalStart () [0x00000] in <filename unknown>:0 

I apologize for the missing file names / line numbers, I was not able to reproduce the issue when running with --debug.

Thank you.
Andrea