Linqpad driver is missing some of the event classes in the connection view:

In this case, all the 4 classes have Info and Associate events (some Enter/Exit). For some reason they only appear in the Device_Client class.

The query though works as expected and returns the same counts as PerfView:

playback.GetAll(
    typeof(Tx.Windows.Microsoft_Azure_Devices_Provisioning_Client.Associate),
    typeof(Tx.Windows.Microsoft_Azure_Devices_Provisioning_Client.Info),
    typeof(Tx.Windows.Microsoft_Azure_Devices_Provisioning_Client.RegisterAsync)
    ).Count().Dump();

Getting PDH_CSTATUS_NO_OBJECT error when attempting to parse blg files

Unhandled exception. System.Exception: PDH_CSTATUS_NO_OBJECT
   at Tx.Windows.PdhUtils.CheckStatus(PdhStatus actualStatus, PdhStatus[] expectedStatus)
   at Tx.Windows.PdhUtils.GetCounterAndInstanceList(String logFilename, String machineName, String objectName, List`1& counterList, List`1& instanceList)
   at Tx.Windows.PdhUtils.GetCounterPaths(String dataSource)
   at Tx.Windows.PerfCounterFileReader..ctor(IObserver`1 observer, String file)
   at Tx.Windows.PerfCounterObservable.<>c__DisplayClass0_0.<FromFile>b__0(IObserver`1 o)
   at System.Reactive.AnonymousObservable`1.SubscribeCore(IObserver`1 observer) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\AnonymousObservable.cs:line 37
   at System.Reactive.ObservableBase`1.ScheduledSubscribe(IScheduler _, AutoDetachObserver`1 autoDetachObserver) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\ObservableBase.cs:line 82
--- End of stack trace from previous location ---
   at System.Reactive.PlatformServices.ExceptionServicesImpl.Rethrow(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServicesImpl.cs:line 16
   at System.Reactive.ExceptionHelpers.ThrowIfNotNull(Exception exception) in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Internal\ExceptionServices.cs:line 20
   at System.Reactive.Linq.ObservableImpl.GetEnumerator`1.MoveNext() in D:\a\1\s\Rx.NET\Source\src\System.Reactive\Linq\Observable\GetEnumerator.cs:line 70

Code that causes the error:

var rawcounters = PerfCounterObservable.FromFile(filename);
int count = rawcounters.Count().Wait();

Custom parser for syslog was kludgy and realistically custom parsing and enrichment is easier to do downstream.

I need to query several performance counters at the same time. For that reason I need to use my own timer and explicitly read out the realtime performance counter. Would be great to have that possibility.

update PerfCounterReader.cs AddCounter throws exception on status PdhStatus.PDH_CSTATUS_NO_INSTANCE

current functionality fails playback.Run() for entire .blg if any counter fails due to status PDH_CSTATUS_NO_INSTANCE with no option to handle

code calling into Tx.Windows:
Playback playback = new Playback();
playback.AddPerfCounterTraces(outputFile);
IObservable o = playback.GetObservable();
o.Subscribe(p => records.Add(p), x => Log.Warning($"{x.Message}"));

failing function:
protected void AddCounter(string counterPath, int index)
{
PdhCounterHandle counter;
PdhStatus status = PdhNativeMethods.PdhAddCounter(_query, counterPath, IntPtr.Zero, out counter);
if (status == PdhStatus.PDH_ENTRY_NOT_IN_LOG_FILE)
return;

        PdhUtils.CheckStatus(status, PdhStatus.PDH_CSTATUS_VALID_DATA);

        var counterInfo = new PerfCounterInfo(counterPath, counter, index);
        _counters.Add(counterInfo);
    }

Can you please give an example on how to do this?

 string[] arr = new string[1];
 arr[0] = "\\Processor(*)\\% processor time";
 PerfCounterRealTimeProbe pcrtp = new PerfCounterRealTimeProbe(this, TimeSpan.FromSeconds(1), arr);

However, that causes a PDH_CSTATUS_NO_OBJECT exception.

Playback object throws an exception when merging a windowed observable that doesn't have inputs with an empty observable on the playback scheduler. The following code snippet throws an InvalidOperationException:

using (var playback = new Playback())
{
    playback.AddInput(new[] { new Timestamped<object>(4L, DateTimeOffset.UtcNow), });
    playback.GetObservable<string>()
            .Window(TimeSpan.FromSeconds(40), playback.Scheduler)
            .Merge(Observable.Empty<object>(), playback.Scheduler)
            .Subscribe(Console.Out.WriteLine);
    playback.Run();
}

"Integer" type is "long". Since TrapTypeMap is using int for converting from Integer to Enum, the exception is encountered.

Fix:
TrapTypeMap.cs

Change convertedValue = Expression.Convert(convertedValue, typeof(int));
to
convertedValue = Expression.Convert(convertedValue, typeof(long));

Throws this exception when installed from LinqPad drivers page

Error instantiating type Tx.LinqPad.TxDataContextDriver in C:\Users\xyz\AppData\Local\LINQPad\Drivers\DataContext\4.6\Tx.LinqPad (3d3a4b0768c9178e)\Tx.LinqPad.dll

• Could not load file or assembly 'file:///C:\Users\xyz\AppData\Local\LINQPad\Drivers\DataContext\4.6\Tx.LinqPad (3d3a4b0768c9178e)\Tx.SqlServer.dll' or one of its dependencies. An attempt was made to load a program with an incorrect format.

When trying to install this into a new instance of LINQPad, I see this in the "Choose a Driver" screen for the "LINQ to Traces" driver:

I have a situation in which the Dispose gets called while the Timer is null so the call to _timer.Dispose(); fails in PerfCounterRealTimeProbe.cs . I suggest to check the _timer for a null value before disposing. Shall I create a PR for that?

Is it possible to read FailedReqLogFiles with TX?
FailedReqLogFiles logs are the logs made by IIS with the "Failed Request Tracing Rules" and generate xml files with name frXXXXXX.xml in ..\inetpub\logs\FailedReqLogFiles\W3SVCX

I opened a BLG file and read to the end. Every Timestamp have the same value, the same as the first sample.

Repro:

        var perfs = PerfCounterObservable.FromFile("MyPcPerfMon.blg");


            perfs.Subscribe(x =>
            {
                Console.WriteLine($"{x.Timestamp} - {x.CounterSet}/{x.CounterName}/{x.Instance}: Value: {x.Value}"); // All Timestramps are the same
                Console.WriteLine();

                if (x.Index % 3 == 2)
                {                        
                    Console.WriteLine("Enter to continue");
                    Console.ReadLine();
                }
            });

Hello, i don't know if this is the proper site to propose this or not (sorry if not), but i go ..

Simplify this syntax:
list.Where(x => x.IsValid)
by this:
list.Where(_.IsValid)

Like in Scala language:
https://stackoverflow.com/questions/7673545/usage-of-in-scala-lambda-functions

¿It's possible?
Thanks :)

fix typo in name \Tx.Windows\PerfCounters\PerfCounteReader.cs -> PerfCounterReader.cs

On commit 49369b4 when the TImeUtil.cs was removed, the returned timestamp appears to be incorrect. The dates are local date/times to based on the time zone the machine is running on, however, the Timestamp.Kind == DateTimeKind.Utc

PerfCounterFileReader.cs

DateTime timestamp = DateTime.FromFileTimeUtc(time);
                                          ^^^

The docs for PdhCollectQueryDataWithTime says:

pllTimeStamp [out]
Time stamp when the first counter value in the query was retrieved. The time is specified as FILETIME.

This is also a problem with: PerfCounterRealTimeProbe.cs.

EtwNativeEvent.cs and BondTypeMap.cs also call DateTime.FromFileTimeUtc() but I haven't looked into what those FILETIMEs are supposed to be.

After looking at the source for DateTime.FromFileTime(long) and DateTime.FromFileTImeUtc(long), it seems nether is correct. The FILETIME returned is a local time, and you want a the kind to be DateTimeKind.Local.

appveyor failing on issue #50 pr #52 due to missing nuget dependencies
https://ci.appveyor.com/project/SergeyBaranchenkov/tx-6doyh/builds/22763689

error NETSDK1004: Assets file 'C:\tx\Source\Tx.Core\obj\project.assets.json' not found. Run a NuGet package restore to generate this file. [C:\tx\Source\Tx.Core\Tx.Core.csproj]

https://github.com/Microsoft/Tx/blob/b2b6a69b2d2b7ce20dfd058911c179d722b279e1/Source/Build.cmd#L9

LINQpad v4 AnyCPU edition cannot load Tx (LINQ to Logs and Traces) LINQpad driver.

It will show the following error:

Error instantiating type Tx.LinqPad.TxDataContextDriver in C:\ProgramData\LINQPad\Drivers\DataContext\4.0\Tx.LinqPad (3d3a4b0768c9178e)\Tx.LinqPad.dll

• Could not load file or assembly 'file:///C:\ProgramData\LINQPad\Drivers\DataContext\4.0\Tx.LinqPad (3d3a4b0768c9178e)\Tx.SqlServer.dll' or one of its dependencies. An attempt was made to load a program with an incorrect format.

That is due to mismatch between the processor architecture of the Tx.SqlServer assembly and the processor architecture of the reference "LINQPad", the only workaround is to use x86 version of LINQpad v4 (default one).

Dependency to Reactive Extensions is changed to use version 3.0.0.
The support of .NET 4.0 is dropped.

Also specifically for each component the additional breaking changes are:

• Tx.Core, Tx.Windows, Tx.Windows.TypeGeneration, Tx.SqlServer
  • No Changes
• Tx.Bond
  • Changed namespaces for all classes to Tx.Bond.
  • Removed deprecated classes BinaryTypeMap and BondTypeMap, use EnvelopeTypeMap instead.
  • GeneralPartitionableTypeMap was renamed to EnvelopeTypeMap.
  • BondProtocol was renamed to Protocol.
  • BondEtwWriter was removed, use BinaryEtwObserver instead.
  • BondEtwObserver was renamed to BinaryEtwObserver.
• Tx.Network
  • Removed deprecated Syslog classes, use new ones from Tx.Network.Syslogs namespace.
  • IpPacket class: removed the following properties - IpVersion, ProtocolNumber, DataBuffer.
  • IpPacket class: the following properties moved to PacketHeader property - SourceIpAddress, DestinationIpAddress, InternetHeaderLength, DscpValue, ExplicitCongestionNotice, IpPacketLength, FragmentGroupId, IpHeaderFlags, FragmentOffset, TimeToLive, ProtocolNumber, PacketHeaderChecksum.
  • IpPacket class: Type of PacketData changed to ArraySegment<byte> from byte[].
  • UdpDatagram class: the following properties moved to UdpDatagramHeader property - SourcePort, DestinationPort, UdpLength, UdpCheckSum.
  • UdpDatagram class: Type of UdpData changed to ArraySegment<byte> from byte[].
  • UdpDatagram class: It does not inherit from IpPacket anymore.
  • UdpReceiver now implements IObservable<IUdpDatagram> not IObservable<IpPacket>.
  • TrapTypeMap now implements IPartitionableTypeMap<IUdpDatagram, ObjectIdentifier> not IPartitionableTypeMap<IpPacket, ObjectIdentifier>.
  • SnmpCapture.ParseUdp now returns IEnumerable<IUdpDatagram> instead of IEnumerable<UdpDatagram>.
  • SnmpSimpleDatagram constructor replaced to accept ArraySegment<byte> instead of byte[].
  • SnmpTrapV1: constructor replaced to accept ArraySegment<byte> instead of byte[], removed constructor that accepts UdpDatagram.
  • SnmpTrapV2: constructor replaced to accept ArraySegment<byte> instead of byte[], removed constructor that accepts UdpDatagram.
  • SnmpEncodeDecoder class was removed.
  • SnmpSimpleTrapV2C class was removed.
  • VarBindExtensions class was removed.
  • ByteArrayExtentions class was removed.
  • NetworkTransformExtentions: removed PacketHeaderToWireBytes, UdpHeaderToWireBytes, ToWireBytes, UdpPseudoHeader, GetInternetChecksum, NetworkOrderUshort methods.

Nuget shows last version ad 3.0.4, Github releases shows v2.1.1 as the last one, it would be good to understand what is new in the recent versions.

At least for Tx.Core and Tx.Network

PerfCounterSample's instance field is truncated if contains '#' symbol.

Repro steps:

• Start multiple instances of LINQpad for example
• Collect any process counters for process with LINQpad.UserQuery#1 instance name. Use the following code snippet: PerfCounterObservable.FromRealTime(TimeSpan.FromSeconds(1), @"\Process(LINQPad.UserQuery#1)\Private Bytes").Dump()
• See that instances in the result set will be truncated to LI

Some SNMPTraps have OctetString OIDs. UTF8 encoding does not work for all these OctetStrings thus non printable chars are seen in the result. This is because the value of the OIDs are hexadecimal.

Replace links to tx.codeplex.com with links to https://github.com/Microsoft/Tx

When I attempt to load the current (1.0.51230) version, I get the following exception. Any ideas?

I started getting this because the assembly must have been auto-updated. I tried removing everything and starting from a clean install and I still get the exception.

This is one of the attributes required for defining a Trap.
TimestampAttribute.cs

Playback.GetObservable() throws a NotImplementedException when the Delay() operator is used.

    using (var playback = new Playback())
    {
        var evtTime = new DateTimeOffset(2018, 1, 1, 12, 0, 0);

        playback.AddInput(new[] { new Timestamped<object>(evtTime, evtTime) });
        playback.GetObservable<DateTimeOffset>()
            .Delay(TimeSpan.FromSeconds(10), playback.Scheduler)
            .Subscribe(x => Debug.WriteLine($"Data Time: {x}    SchedulerTime: {playback.Scheduler.Now}"));
        playback.Run();
    }

Currently it is internal due to not be fully compliant with PcapNg format.

Cannot generate types for ServiceFabric ETW provider - ManifestParser fails with a null ref exception.

CS0433 The type '[EVENT_NAME]' exists in both 'LogFile1, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' and 'LogFile2, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'

If the event exists in both with the same schema then one should be chosen. This would increase the debugging experience by a lot since we could actually import multiple files in the same session.

This is in reference to the LinqPad Tx driver

I'm trying to build from source.
Have tried the following:

• Opened Source\Tx.sln in VS 2015, but getting a bunch of CS0234 and CS0246 errors
• Tried running Source\build.cmd, but it fails with 'error MSB4126: The specified solution configuration "Release40|X64" is invalid.'

Any suggestions?

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

Adding the Tx provider to LinqPad 5 (32-bit) I get 'Could not load file or assembly 'Tx.LinqPad.resources, Version=1.0l.60318.0.... or one of its dependencies. The system cannot find the path specified.'