Code Monkey home page Code Monkey logo

qwiq's Introduction

QWIQ

License: MIT Build status: DEVELOP

MyGet MyGet

QWIQ is a Quick Work Item Query library for Team Foundation Server / Visual Studio Online. If you do a lot of reading or writing of work items, this package is for you!

What can it be used for?

Querying Team Foundation Server, of course! Instead of directly using the TFS Client OM, you could use QWIQ! It it made of packages designed to make working with Tfs/Vso a pleasure. Qwiq.Core is the no-frills base package, exposinng the raw types needed to read and write work items. Qwiq.Identity adds methods to simplify converting between your preferred method of identity (display names, user names) and TFS's identity classes. Qwiq.Linq provides a Linq query provider to be able to write Linq to query tfs. Qwiq.Mapper enables converting from IWorkItem, the raw Qwiq.Core type, to your own classes to enable strongly typed access to your WorkItems. Qwiq.Relatives extends Qwiq.Linq to enable slightly more complicated queries allowing for basic queries of related workitems. Qwiq.Mocks provides default implementations for commonly mocked classes within Qwiq, and should allow for getting up and unit testing quickly. Why use this over the Client OM? Glad you asked!

1. Easier to consume

Let's be honest, the TFS libraries are a pain to use. There are a lot of them, several are dynamically loaded, and a few are native. While we can't avoid it, you can! Just install the Qwiq.Core package and everything will be in your \bin folder when you need it.

2. Easier to test

Qwiq makes testing your apps a breeze. Everything has an interface. Everything uses factories (or factory methods) instead of constructors. Install our Qwiq.Mocks package for easy to use mocks, or mock what you need from out interfaces for your tests and go. No more messy, temperamental fakes, or adapters cluttering your code.

3. Easier to understand

How often do you update a work item? How often do you create a new security group? We stripped out the rarely used stuff to make interfaces cleaner and the relationships between types simpler. Missing something you can't live without? Send us a pull request!

How to install it

We have two ways you can install our packages: through NuGet.org which contains our stable packages only, and MyGet.org, which contains vNext and stable packages.

If you want our vNext packages, add our MyGet feed to your NuGet clients:

  • v3 (VS 2015+ / NuGet 3.x): https://www.myget.org/F/qwiq/api/v3/index.json
  • v2 (VS 2013 / NuGet 2.x): https://www.myget.org/F/qwiq/api/v2

Once the feed is configured, install via the nuget UI or via the nuget package manager console

Install Core

From the NuGet package manager console NuGet

PM> Install-Package Qwiq.Core

Or via the UI Qwiq.Core

MyGet

PM> Install-Package Qwiq.Core -Source https://www.myget.org/F/qwiq/api/v3/index.json

Or via the UI Qwiq.Core,

Install Client

We now have two clients: one for SOAP, and one for REST NuGet

PM> Install-Package Qwiq.Client.Soap

Or via the UI Qwiq.Client.Soap,

From the NuGet package manager console MyGet

PM> Install-Package Qwiq.Client.Soap -Source https://www.myget.org/F/qwiq/api/v3/index.json

Or via the UI Qwiq.Client.Soap,

Basic Usage

For .NET

using Qwiq;
using Qwiq.Credentials;

using Microsoft.VisualStudio.Services.Client;
...

// We support
//  - OAuth2
//  - Personal Access Token (PAT)
//  - Username and password (BASIC)
//  - Windows credentials (NTLM or Federated with Azure Active Directory)
//  - Anonymous

// Use the full URI, including the collection. Example: https://QWIQ.VisualStudio.com/DefaultCollection
var uri = new Uri("[Tfs Tenant Uri]");
var options = new AuthenticationOptions(uri, AuthenticationTypes.Windows);
var store = WorkItemStoreFactory
                .Default
                .Create(options);

// Execute WIQL
var items = store.Query(@"
    SELECT [System.Id] 
    FROM WorkItems 
    WHERE [System.WorkItemType] = 'Bug' AND State = 'Active'");

For PowerShell

[Reflection.Assembly]::LoadFrom("E:\Path\To\Qwiq.Core.dll")
# Can use SOAP or REST clients here
[Reflection.Assembly]::LoadFrom("E:\Path\To\Qwiq.Client.Soap.dll")

$uri = [Uri]"[Tfs Tenant Uri]"
$options = New-Object Qwiq.Credentials.AuthenticationOptions $uri,Windows
$store = [Qwiq.Client.Soap.WorkItemStoreFactory]::Default.Create($options)

$items = $store.Query(@"
    SELECT [System.Id] 
    FROM WorkItems 
    WHERE [System.WorkItemType] = 'Bug' AND State = 'Active'", $false)

Contributing

Getting started with Git and GitHub

Once you're familiar with Git and GitHub, clone the repository and start contributing!

qwiq's People

Contributors

jobeland avatar khansalman avatar lyra1337 avatar mattkotsenas avatar pelavall avatar rjmurillo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

qwiq's Issues

An attribute defined on a type for mapping that does not exist on the WIT throws an unhandled FieldDefinitionNotExistException

@rjmurillo commented on Wed Feb 24 2016

During mapping an exception may be encountered

Microsoft.TeamFoundation.WorkItemTracking.Client.FieldDefinitionNotExistException: TF26027: A field definition Cost in the work item type definition file does not exist. Add a definition for this field or remove the reference to the field and try again.

Server stack trace: 
   at Microsoft.TeamFoundation.WorkItemTracking.Client.FieldDefinitionCollection.get_Item(String name)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.FieldCollection.get_Item(String name)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.get_Item(String name)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.IE.Qwiq.IWorkItem.get_Item(String name)
   at Microsoft.IE.Qwiq.Mapper.Attributes.AttributeMapperStrategy.Map(Type targetWorkItemType, IWorkItem sourceWorkItem, Object targetWorkItem, IWorkItemMapper workItemMapper)
   at Microsoft.IE.Qwiq.Mapper.IndividualWorkItemMapperBase.Map(Type targetWorkItemType, IEnumerable`1 workItemMappings, IWorkItemMapper workItemMapper)
   at Microsoft.IE.Qwiq.Mapper.WorkItemMapper.ParseWorkItems(Type type, IEnumerable`1 collection)
   at Microsoft.IE.Qwiq.Mapper.WorkItemMapper.Create(Type type, IEnumerable`1 collection)
   at IEPortal.Web.Areas.CustomReports.Data.SimpleWorkItemToIssueMapper.Map(IWorkItem record)
   at IEPortal.Web.Areas.Planning.Data.GenericTreeMapper`2.Map(ITreeNode`1 record)
   at System.Linq.Enumerable.<>c__DisplayClass2`3.<CombineSelectors>b__3(TSource x)
   at System.Linq.Enumerable.<>c__DisplayClass2`3.<CombineSelectors>b__3(TSource x)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at WebGrease.Css.Extensions.ListExtensions.ToSafeReadOnlyCollection[T](IEnumerable`1 enumerable)
   at IEPortal.Web.Areas.CustomReports.Controllers.ProjectController.Index(String id)
   at lambda_method(Closure , ControllerBase , Object[] )
   at System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary`2 parameters)
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary`2 parameters)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<BeginInvokeSynchronousActionMethod>b__39(IAsyncResult asyncResult, ActionInvocation innerInvokeState)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`2.CallEndDelegate(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3d()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass46.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3f()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass46.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3f()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass46.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3f()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass21.<>c__DisplayClass2b.<BeginInvokeAction>b__1c()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass21.<BeginInvokeAction>b__1e(IAsyncResult asyncResult)

/cc @pelavall @MattKotsenas

Unclear message when querying by ID in REST for work item that does not exist

Repro

var authType = AuthenticationTypes.PersonalAccessToken;
var options = new AuthenticationOptions(uri, authType, types => new VssCredentials[] { cred });
var fac = Microsoft.Qwiq.Client.Rest.WorkItemStoreFactory.Default;
var wis = fac.Create(options);
wis.Query(12345);

System.ArgumentNullException
Value cannot be null. Parameter name: item

   at Microsoft.Qwiq.Client.Rest.WorkItem..ctor(WorkItem item, Lazy`1 wit, Func`2 linkFunc)
   at Microsoft.Qwiq.Client.Rest.Query.CreateItemLazy(WorkItem workItem)
   at Microsoft.Qwiq.Client.Rest.Query.<RunQueryImplLazy>d__21.MoveNext()
   at Microsoft.Qwiq.ReadOnlyObjectWithNameCollection`1.Ensure()
   at Microsoft.Qwiq.ReadOnlyObjectWithNameCollection`1.GetEnumerator()
   at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source)
   at Microsoft.Qwiq.Client.Rest.WorkItemStore.Query(Int32 id, Nullable`1 asOf)
   at Castle.Proxies.Invocations.IWorkItemStore_Query_2.InvokeMethodOnTarget()
   at Castle.DynamicProxy.AbstractInvocation.Proceed()
   at Microsoft.Qwiq.Exceptions.ExceptionHandlingDynamicProxy.Intercept(IInvocation invocation)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Qwiq.Exceptions.ExceptionHandlingDynamicProxy.Intercept(IInvocation invocation)
   at Castle.DynamicProxy.AbstractInvocation.Proceed()
   at Castle.Proxies.IWorkItemStoreProxy.Query(Int32 id, Nullable`1 asOf)

Revisit Qwiq.Core.Tests dependencies on Fakes

LinkCollectionTests and LinkMapperTests are the only ones which relay on Fakes to test. We should revisit the value these tests add. If they are valuable we need to find a different framework, preferably free, which can support the tests. If no we should remove them and our dependency to fakes.

Qwiq.Identity 10.0.1 and others have conflicting version issues which can cause assembly load errors.

For an example Qwiq.Identity has, as a dependency on Qwiq.Core 10.0.1-ci0004, which does not exist. When you try to use Qwiq.Identity in code it will fail to do being unable to resolve Qwiq.Core 10.0.1.4.

This issue occurs with a number of the packages, where version 10.0.1 of the package will depend on 10.0.1-ci004 of another package.

A work around for this issue to use the Qwiq packages with version 10.1.0-beta0011.

Cannot run QWIQ.Core 5.0.0 inside of an Azure Worker role

Version 5.0.0 runs fine inside of a local console app, but when running inside of an Azure Worker Role, even locally, it gets a runtime error that Microsoft.WindowsAzure.ServiceRuntime cannot find Microsoft.WITDataStore64.dll. debugged it with @pelavall and found that it wasn't copying native dll's over into the package. Adding the following items groups to my .csproj for my worker role fix the issue:

<ItemGroup>
    <Content Include="..\packages\Microsoft.TeamFoundationServer.ExtendedClient.14.89.0\lib\native\x86\Microsoft.WITDataStore32.dll">
      <Link>Microsoft.WITDataStore32.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </Content>
    <Content Include="..\packages\Microsoft.TeamFoundationServer.ExtendedClient.14.89.0\lib\native\amd64\Microsoft.WITDataStore64.dll">
      <Link>Microsoft.WITDataStore64.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </Content>
  </ItemGroup>

REST client only returns 100 projects on WorkItemStore

When using the REST WorkItemStore.Projects property, only the first 100 projects are returned by default.

See: https://www.visualstudio.com/en-us/docs/integrate/api/tfs/projects

This causes exceptions during work item mapping when the team project is not within the returned project collection. The error indicated is an InvalidOperationException with a message of "No project for specified value 'FOO'". There is currently no way to override the fetch behavior, so users are blocked

During mapping Microsoft.Qwiq.DeniedOrNotExistException thrown

While mapping I encounter a Microsoft.Qwiq.DeniedOrNotExistException indicating a field I am attempting to map does not exist or I do not have access to it, but it does not specify which field, making it tedious to debug.

Stack trace

   at Microsoft.Qwiq.ReadOnlyObjectWithNameCollection`1.get_Item(String name)
   at Microsoft.Qwiq.FieldCollection.get_Item(String name)
   at Microsoft.Qwiq.WorkItem.get_Item(String name)
   at Microsoft.Qwiq.Mapper.Attributes.AttributeMapperStrategy.MapImpl(Type targetWorkItemType, IWorkItem sourceWorkItem, Object targetWorkItem)
   at Microsoft.Qwiq.Mapper.Attributes.AttributeMapperStrategy.Map[T](IDictionary`2 workItemMappings, IWorkItemMapper workItemMapper)
   at Microsoft.Qwiq.Mapper.WorkItemMapper.<Create>d__7`1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Microsoft.WebPlatform.Services.Portal.Planning.Api.Data.Hierarchy.ModelsByIdRepository`1.GetAll(IEnumerable`1 argument)

The call to AssignFieldValue will throw an AttributeMapException if the type cannot be placed into the destination.

var fieldValue = sourceWorkItem[fieldName];
AssignFieldValue(targetWorkItemType, sourceWorkItem, targetWorkItem, property, fieldName, convert, nullSub, fieldValue);

See https://github.com/MicrosoftEdge/Microsoft.Qwiq/blob/master/src/Qwiq.Mapper/Attributes/AttributeMapperStrategy.cs#L136

I would expect the same behavior for reading the field value from the IWorkItem instance if an exception were to be thrown.

NOTE: This may be the result of the change in #151: https://github.com/MicrosoftEdge/Microsoft.Qwiq/pull/151/files#diff-35bda63b09dc7ee2495362f4f91cde92L171

Experiment: Remove WITDataStore64/32.dll

PR #23 introduces changes to assemblies pulled from NuGet rather than private binplace. Conversation with @rjmurillo and @MattKotsenas on 9/11 to attempt an experiment removing the assembly for WITDataStore to determine if we can use only the NuGet binaries.

There are no integration tests present that could easily validate at this time. Will need to pull it into Agents or Portal to experiment (or port some tests).

Created VSO 4564432

QWIQ.Core references a different version of ActiveDirectory than what it pulls in

When installing the QWIQ.Core package, it also pulls down Microsoft.IdentityModel.Clients,ActiveDirectory package version 2.1.6. However, QWIQ.Core's csproj references version 2.2.3, so it results in a runtime error that it cannot find the assembly. worked with @pelavall and was able to mitigate the issue by either updating the package manually to 2.2.3 for my project, or by adding a binding redirect in my project's app.config to have 0.0.0.0-2.2.3.0 to just use 2.1.6.0.

Support "Business Objective" Scenario WIT

Request from @uofmrob

OSG will make schema updates during TH2 that will introduce a new group for scenarios (the "Super Scenario"). Nancy will have additional information in the upcoming week.

In Redstone, WIT linking occurs in CP -> S -> S -> D -> T, which breaks the assumption of CP -> S -> D -> T. For this planning page, we need

 S ---> S -> D -> T
    \-> D -> T

Created VSO 3651638

BrowserFlowException when Chromium Edge is default browser

Cannot complete sign in flow for SOAP client

Microsoft.VisualStudio.Services.Client.Controls.BrowserFlowException: 'SP324098: Your browser could not complete the operation.'

Inner Exception
InvalidCastException: Unable to cast COM object of type 'System.__ComObject' to interface type 'IInternetSession'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{79EAC9E7-BAF9-11CE-8C82-00AA004BA90B}' failed due to the following error: No such interface supported (Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).

Workaround: Go to Settings and change the default browser back to other Edge

Expose credentials used to create TFS connection

When using the CredentialsFactory, the credentials actually used to make the connection are not exposed. They are, however, available as properties on the Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore type:

  • UserDisplayName
  • UserIdentityName
  • UserSid

They are also available from the TeamFoundationServer property's properties:

  • AuthenticatedUserDisplayName
  • AuthenticatedUserIdentity
  • AuthenticatedUserName

Using the mapper with the REST client fails

When using the mapper with the REST client, the default fields are set, but do not include the required fields for mapping, resulting in System.InvalidOperationException to be thrown: Field 'System.TeamProject' is required.

POCO

[WorkItemType("Bug")]
public class Bug : IIdentifiable<int?>
{
     [FieldDefinition(CoreFieldRefNames.Id, true)]
     public int? Id { get; set; }
}

Work item store configuration

private static readonly Uri Uri = new Uri("https://foo.visualstudio.com/defaultcollection");
private static Func<AuthenticationTypes, IEnumerable<VssCredentials>> Credentials = UnitTestCredentialsFactory;
private static AuthenticationOptions AuthenticationOptions { get; } = new AuthenticationOptions(Uri, AuthenticationTypes.Windows, Credentials);
private static IEnumerable<VssCredentials> UnitTestCredentialsFactory(AuthenticationTypes types)
        {
            if (types.HasFlag(AuthenticationTypes.Windows))
            {
                // User did not specify a username or a password, so use the process identity
                yield return new VssClientCredentials(new WindowsCredential(false)) { Storage = new VssClientCredentialStorage(), PromptType = CredentialPromptType.DoNotPrompt };

                // Use the Windows identity of the logged on user
                yield return new VssClientCredentials(true) { Storage = new VssClientCredentialStorage(), PromptType = CredentialPromptType.PromptIfNeeded };
            }
}
private static Func<IWorkItemStore> Store { get; } = () =>
                                                                          {
                                                                              var options = AuthenticationOptions;
                                                                              var wis = Client.Rest.WorkItemStoreFactory.Default.Create(options);
                                                                              return wis;
                                                                          };

Improve efficiency of mapping operation by requring call to PartialOpen

From Performance Tuning the Work Item Tracking Object Model

Because of the paging and lazy evaluation scheme discussed earlier, viewing or editing unpaged fields on a WorkItem in a WorkItemCollection will cause an additional round-trip. This round-trip retrieves all the additional work item data that is not paged in as part of the query. This operation can be expensive.

Consumers can call PartialOpen on a WorkItem to minimize this overhead. You can then view and edit most of the fields on the WorkItem, but the object model optimizes to send a minimal set of data over the network.

Qwiq.Linq does not support Convert

Qwiq.Linq does not support Convert this prevents being able to use queries which have a DateTime value in the IWorkItem.Fields.

Example:

var now = DateTime.UtcNow();
_workItems.Where(wi => (DateTime)wi["Changed Date"] < now).ToList()

This code will fail due to not supporting the conversion of object to DateTime. There does not seem to be work arounds for this in all cases. In some cases .ToString can be used, but in this particular case, strings do not support >. Other case mapping can be used to ensure a correctly typed property exists when querying.

Qwiq does not support variables in queries

You can use variables in queries to pass user input or calculated values. To create a query that includes variables, create placeholders in the query string using @variable, where "variable" is the name of the variable. Then pass the name (less the "@") and value of each into a dictionary as shown.

// Define a query that uses a variable for the type of work item. 
string queryString = "Select [State], [Title] From WorkItems Where [Work Item Type] = @Type";

// Set up a dictionary to pass "User Story" as the value of the type variable.
Dictionary<string, string> variables = new Dictionary<string, string>();
variables.Add("Type", "User Story");

// Create and run the query.
Query query = new Query(workItemStore, queryString, variables); 
WorkItemCollection results = query.RunQuery();

Cannot use scoped PATs with SOAP client

When using a PAT for authentication with VSTS, the "All Scopes" option must be selected in order to authenticate successfully. Select any other scope option (including all options under Selected scopes) will result in a TF30063 message (access denied).

This is "by design" for the SOAP client, as the TFS Client OM does not understand the concept of how scopes work.

https://social.msdn.microsoft.com/Forums/windowsapps/en-US/5564cb2b-4e46-4a41-b12c-bc50304c777b/authentication-issue-using-personal-access-token-when-all-options-of-selected-scope-are-selected?forum=TFService&forum=TFService

Behind the scenes, the TFS Client OM you're using is hitting our older SOAP endpoints and not the newer REST endpoints. Individual scopes only work with the newer REST endpoints due to the mechanics of how scopes work, but we are working on a solution for the SOAP APIs to have scopes as well, although I don't have a timeline on when that will be completed.

Depreciation of the SOAP interface in ADO/TFS

Beginning in Jan 2020 and with TFS 2020 on-prem, the SOAP interface will no longer be supported.
https://docs.microsoft.com/en-us/azure/devops/integrate/concepts/wit-client-om-deprecation?view=azure-devops

Right now we support the reading capabilities through the REST interfaces, but do not generate patches to send back to the server. We also have quite a few SOAP-isms in the code to make the REST and SOAP clients behave in similar ways so we really just can swap one implementation for another.

It's a year out, but I'd like to get the discussion going about what we would like to do about this change. Some ideas:

  • Drop support in the next major for SOAP (e.g. delete) and go REST all the way. We would need to give people a heads up by marking the SOAP classes as deprecated.
  • Stop support for SOAP in the major, but kep the code. Code marked as deprecated with instructions to move to REST, but we won't do any enhancements to the SOAP implementations and minimal bug fixes. This allows people to use the latest version against older TFS instances.
  • Some version of the first two where we phase it out over time. We don't have any telemetry so we can't measure the impact. All we have is the scream test (either by the community saying something or NuGet downloads dropping)

@MattKotsenas @pelavall Other ideas?

`System.InvalidOperationException` encountered when an item is linked to another in multiple ways

The FindEquivilentLink() function assumes that two work items can only have a single related link type.

       var relatedLink = link as IRelatedLink;
       if (relatedLink != null)
       {
           return item.Links.Cast<Tfs.Link>().OfType<Tfs.RelatedLink>().SingleOrDefault(rl => rl.RelatedWorkItemId == relatedLink.RelatedWorkItemId);
       }

https://github.com/MicrosoftEdge/IEPortal.Qwiq/blob/master/Qwiq/Qwiq.Core/LinkHelper.cs#L14

For example:
A deliverable has a parent link of 123 and a related link to 123

System.NullReferenceException thrown when referring to a field in WIT that is null but target does not accept null

Problem

This occurs in AttributeMapperStrategy.MapImpl

Repro:

public class Model : IIdentifiable
{
  [FieldDefinition("Priority")] 
  public virtual int Priority { get; internal set; }

  [FieldDefinition("System.Id")]
  public virtual int Id { get; internal set; }
}

For each item mapped the mapper will attempt to pull the field for Priority from the WIT. Each time at DelegateAccessor.set_Item the System.NullReferenceException is thrown because the value retreived for Priority is null, but the model requires a non-nullable value. In the general catch statement, a trace message is attempted to be logged to indicate the conversion failed, which results in another null reference exception. Eventually a warning is logged to indicate the field could not be mapped.

This causes several problems:

  • The field is attempted to be mapped each time it is encountered for the WIT, though it is invalid. In a legitimate case, some items may have Priority while others do not.
  • Each causes two first chance System.NullReferenceException to be thrown.
  • The exception is caught in the generalized catch block, and trace message is logged.
  • Large amounts of trace messages are sent to logging sync (such as a remote service), causing throttling

Proposal

  • Adjust catch block to provide a more specific message as to why the field could not be mapped in the null case.
  • Provide a means of substituting a value when conversion fails or the value is null
  • Eliminate System.NullReferenceException first chance exceptions while executing in this state

Improve efficiency of DirectLink / Tree work item queries

Setup

Given a work item structure

  • Deliverable { Id = 1 }
    • Task { Id = 2 }
    • Bug { Id = 3 }

Problem

The existing implementation:

var deliverablesTasks = vsoContext.Deliverables.Where(p=>p.Id == 1).Children<Deliverable, Task>();
var deliverablesBugs = vsoContext.Deliverables.Where(p=>p.Id == 1).Children<Deliverable, Bug>();
...

Would need to perform the following:

  1. Query on WorkItems where WIT = Deliverable and Id = 1
    SELECT * FROM WorkItems WHERE [System.WorkItemType] = 'Deliverable' AND [System.Id] = 1
  2. Map the resulting Deliverable and fields
  3. Query WorkItemLinks to get the Task children
    SELECT * FROM WorkItemsLinks WHERE Source.[System.WorkItemType] = 'Deliverable' AND Source.[System.Id] = 1 AND [System.Links.LinkType] = 'System.LinkTypes.Hierarchy-Forward' AND Target.[System.WorkItemType] = 'Task'
  4. Query on WorkItems where Id=2
    SELECT * FROM WorkItems WHERE [System.Id] IN (2)
  5. Map the resulting Task and fields
  6. Query on WorkItems where WIT = Deliverable and Id = 1 (repeat of 1)
    SELECT * FROM WorkItems WHERE [System.WorkItemType] = 'Deliverable' AND [System.Id] = 1
  7. Map the resulting Deliverable and fields (repeat of 2)
  8. Query WorkItemLinks to get the Bug children
    SELECT * FROM WorkItemsLinks WHERE Source.[System.WorkItemType] = 'Deliverable' AND Source.[System.Id] = 1 AND [System.Links.LinkType] = 'System.LinkTypes.Hierarchy-Forward' AND Target.[System.WorkItemType] = 'Bug'
  9. Query on WorkItems where Id=3
    SELECT * FROM WorkItems WHERE [System.Id] IN (3)
  10. Map the resulting Bug and fields

The operation requires six trips to VSO in order to hydrate the items.

Proposal

When performing a DirectLink (e.g. Parent / Child, Child / Parent), or a Tree (Parent / Child / Grandchild), use

  1. **IWorkItemStore.QueryLinks method to first query for all link relationships, **
    SELECT * FROM WorkItemsLinks WHERE Source.[System.WorkItemType] = 'Deliverable' AND Source.[System.Id] = 1 AND [System.Links.LinkType] = 'System.LinkTypes.Hierarchy-Forward' AND (Target.[System.WorkItemType] = 'Task' OR Target.[System.WorkItemType] = 'Bug')
  2. Load all work items by ID
    SELECT * FROM WorkItems WHERE [System.Id] IN (1, 2, 3)
  3. Map the resulting work item types

The call to IWorkItemStore.QueryLinks returns IEnumerable<IWorkItemLinkInfo>, which would house the following results given our scenario.

SourceId TargetId LinkTypeId IsLocked
0 1 0 false
1 2 2 false
1 3 2 false

The results represent the tree having one level of children (additional levels set the SourceId and TargetId accordingly). The fields are then mapped using a 'flat' TFS query of regular IDs, which is inherently fast.

Note: this works for DirectLink and Tree when no ASOF is requested. When ASOF is requested, only Flat and DirectLink queries are supported.

http://blogs.msdn.com/b/jsocha/archive/2012/02/22/retrieving-tfs-results-from-a-tree-query.aspx

MockWorkItem.Fields cannot be enumerated if initialzed with a constructor which takes field values

When constructing MockWorkItem() and passing values for a field collection they are ultimately used in setting the private variable _fields on WorkItemCore which is the bases for retrieving fields via indexing. However the variable _fields in WorkItemCore is distinct from the _fields variable in Workitem, which backs the Fields property. In effect, MockWorkItem() enables initializing a workitem with retrievable values via workitem["property"] but not workitem.Fields["property"].Value, nor enumerating the values of the workitem.Fields property.

NotSupportedException thrown when calling CreateRelatedLink on WorkItem

Seen in version 9.0 and 10.0

The method CreateRelatedLink is not overridden in the SOAP implementation. ๐Ÿ˜ณ

Repro

var wis = ...;
var wi = wis.Query(12345);

// Dies
wi.AddParentLink(wis, 45678);

// Also dies
var lt = wis.WorkItemLinkTypes[CoreLinkTypeReferenceNames.Hierarchy];
var link = item.CreateRelatedLink(parentId, lt.ReverseEnd);
wi.Links.Add(link);

Stack

Qwiq.Core.dll!Qwiq.WorkItem.CreateRelatedLink(int relatedWorkItemId, Qwiq.IWorkItemLinkTypeEnd linkTypeEnd)
Qwiq.Core.dll!Qwiq.Exceptions.ExceptionHandlingDynamicProxy.Intercept(Castle.DynamicProxy.IInvocation invocation)

public virtual IRelatedLink CreateRelatedLink(int relatedWorkItemId, IWorkItemLinkTypeEnd linkTypeEnd = null)
{
throw new NotSupportedException();
}

IWorkItem should not have the CreateWorkItemLinkMethod, it doesn't do what it's name suggests

This method is misleading in it's underlying tfs implementation. With the Tfs binaries you cannot create a WorkItemLink. The public constructors on the type, underneath the covers create a RelatedLink. This is only exposed once it is added to a LinkCollection and is enumerated by the underlying api. You can create a WorkItemLink, however, on construction the m_fileInfo.FieldId is set to 37. This is a magic number which means RelatedLink. This behavior can be see if you use ILSpy to look into the Microsoft.TeamFoundation.WorkItemTracking.Client. See WorkItemLink.WorkItemLink() constructors for the magic number(37==relatedlink) and LinkCollection.Populate/LinkCollection.CreateLinkObject for the mapping of magic numbers to the concrete link types.

The IdentityVisitor has an implicit dependency on the PartialEvaluator visitor

The chain of visitors which is fed into the WiqlQueryBuilder is required to be in a particular order. In the case of the IdentityVisitor, if the PartialEvaluator is not placed in line before the IdentityVisitor, it will not execute correctly as some of the expressions have not been simplified to the extent that the IdentityVisitor can interpret them.

As implemented there are no safeguards around constructing these types appropriately. We should revisit their implementation to make it such that you can only construct these types such that they will work as intended.

Move to batch reading mode to load revisions

From Performance Tuning the Work Item Tracking Object Model

It is sometimes desirable to retrieve a specific set of WorkItems directly by ID if you know the IDs in advance. To do this, use the method in the following example. It takes in a batch collection that specifies the IDs and revisions of the desired WorkItems and a query string that specifies the Fields to page.

public WorkItemCollection Query(BatchReadParameterCollection batchReadParams, string wiql)

This method minimizes the round-trips used for a query, especially if the goal is to get a specific revision of each WorkItem. The following example specifies the series of calls in which you must specify IDs and revisions of work items.

  1. Run a query for the desired WorkItems. This returns a collection of the latest Revisions for the WorkItems.
  2. Open each WorkItem in the returned WorkItemCollection.
  3. For each WorkItem in the returned collection, open the desired Revision from its RevisionCollection.

Steps 1 and 2 earlier require round-trips to the server. For many WorkItems, the number of round-trips increases linearly.

By using the batch read version of the Query, the system returns the collection of specified work item revisions with a constant order of round-trips.

Call .Close() on WorkItems

From working with the Agents project it appears that the Tfs.WorkItem object leaks if you don't call .Close() on it when you're done. We should do this in the Issues project to reduce memory pressure.

MockWorkItemStore does not allow for custom link types

VSO/ADO permits the addition of customized link types in WITs. I need to mock out a ProducingFor-Forward link but can't find a hook. MockWorkItemTypeLink permits passing a reference name, but only one of the four specified in the default configuration.

Hundreds to thousands of exceptions are thrown during mapping when field not exist on WIT

Scenario: Use a single POCO for two WITs with differing schemas

WIT: Foo
Field: Alpha

WIT: Bar
Field: Beta

C#

[WorkItemType("Foo")]
[WorkItemType("Bar")]
public class Poco
{
  [FieldDefinition("Alpha", string.Empty)]
  public string Alpha { get; set; }

  [FieldDefintion("Beta", string.Empty)]
  public string Beta { get; set; }
}

WIQL

SELECT [System.Id], [System.WorkItemType], [System.TeamProject], [Alpha], [Beta]
FROM WorkItems
WHERE [System.WorkItemType] IN ('Foo', 'Bar')

Problem

During mapping of each work item type to the CLR class Poco, exceptions of DeniedOrNotExistException and AttributeMapException are thrown: When mapping Foo the Beta field does not exist and each exception is thrown, and mapping Bar does the same for the Alpha field.

When querying for Foos and Bars, there are n instances of DeniedOrNotExistException thrown and n instances of AttributeMapException thrown, where n is the number of work items returned.

Using the default IWorkItemStore configuration, exception proxies are configured. During execution, each exceptions is unwound and mapping to a more generic exception is attempted, which requires walking the inner exceptions. This operation does not find a match, yet each exception is evaluated individually. This costs (n exceptions * m exploders + n exceptions * o mappers).

In the default configuration of IWorkItemStore, each failed map for a single property results in

  • 1 DeniedOrNotExistException thrown
  • 1 AttributeMapException thrown
  • 2 ExceptionExploder operations (one for each exploder)
  • 1 ExceptionMapper operation

Executing and hydrating the work items with the REST client takes about 3 seconds in my scenario to load 168 items, but takes another 25 seconds to perform mapping.

Set the PageSize property of the WorkItemCollection

Set WorkItemCollection.PageSize to a value greater than 50 (default)

From Performance Tuning the Work Item Tracking Object Model

Paging and lazy evaluation occurs on WorkItemCollections returned by a query. When you execute a query, the system returns only the fields selected in the query expression as part of the WorkItemCollection. For example, the query in the following example selects the Title field from a group of work items. The WorkItemCollection returned from this query will only include the selected "Title" field values for the contained WorkItems. Only when you read or edit other fields on the WorkItem, will the system make a round-trip to fetch the other field values.

SELECT System.Title FROM Workitems WHERE (ID < 1000)

Additionally, the selected fields returned as part of the WorkItemCollection are paged in chunks. You can set the page size between 50 and 200 by using the PageSize property to tune performance, as needed.

https://msdn.microsoft.com/en-us/library/microsoft.teamfoundation.workitemtracking.client.workitemcollection.pagesize(v=vs.120).aspx

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.