subpointsolutions / metapack Goto Github PK
View Code? Open in Web Editor NEWA NuGet platform for packaging, delivering and deploying SharePoint customization with PnP/SPMeta2 support.
A NuGet platform for packaging, delivering and deploying SharePoint customization with PnP/SPMeta2 support.
"Action-CLI-Regression" should be enabled under CI build and O365 profile.
We should be sure that MetaPack CLI actually work and can provision at least O365 based packages.
A public NuGet Gallery for MetaPack needs to be introduced
All reflection based operations should be implemented within a dedicated service. Refactoring and clean up is required.
Metapack should support custom model packaging and deployment workflow.
API should allow to inject custom model packaging provider to accomplish the following scenarios:
Implementation should work as NuGet based plugin similar to Cakebuild.
Regression testing should be reengineered to meet the following criteria:
Metapack should be able to work with multiple NuGet Galleries.
-source parameter should support ';' to provide multiple NuGet galleries
app.config should provide ability to configure multiple NuGet galleries as well
MetaPack should support digital signing of a package with certificates. Most likely, a content of the NuGet packages is to be signed with a certificate. Model packaging workflow should support that case providing an easy API for signing and verifying the certificate.
Such feature would add value to the platform enabling secure packages distribution.
metapack version should show current version of the client.
MetaPack GUI client should support "brandability" scenario so that it can be used as "my product installer". The following scope is to be considered:
Metapack CLI should be delivered via chocolatey, the easiest deployment and updates channel available.
So that no trace is provided at all.
metapack trace and logs should also be available in JSON format.
API should have additional trace service for that, and CLI should have additional flag to indicate machine-readable trace mode. Flag as the followig will do:
--output json
Such feature is needed by 3rd part API integrations, CLI and desktop client so that they would be able to capture trace in platform/API independent way.
Similar to other projects, docs should be placed within the project and merged with the main repo every build.
Cake build needs to be migrated to the latest SubPointSolutions.CakeBuild package.
Basic API needs to be covered by the regression tests:
Investigate and document guides on running metapack API under Azure functions.
Regression testing has to address the following areas across both API and CLI:
Logging should be improved to meet the following criteria:
Needs to be covered by regression testing - both API and CLI.
Once metapack works with multiple NuGet galleries ( Make metapack work with multiple NuGet Galleries #20), a local NuGet Gallery needs to be added.
The goal is to have a special local folder within metapack. This "local" gallery will be housing ISVs specific packages. The idea is that you can get metapack, put your packages into a folder, and then the final distribution would be able to see these packages. Such scenario is extremely useful for ISVs who provide pre-packaged solutions for customers.
Investigate the following options:
MetaPack CLI should have a concept of "metapackfile" and "metapackfile.lock" similar, for instance, to Ruby's "Gemfile" and "Gemfile.lock"
Both "metapackfile" and "metapackfile.lock" aim to support the following scenarios:
metapackfile describes mappings between SharePoint site urls (or connections) and packages to be deployed. JSON or Yaml can be seen as following:
galleries:
- url: http://metapackgallery.com/api/v2
- url: https://packages.nuget.org/api/v2
- url: http://www.myget.org/F/subpointsolutions-staging/api/v2
deployments:
- TeamSite:
name: team-site
url: https://contoso.sharepoint.com/sites/intranet
spversion: sp2013 | sp2016 | o365
spapi: csom | ssom
packages:
- id: contoso.metadata
version: 1.0.0
- id: contoso.teamsite
version: 1.2.0-beta1
- LeaveRequestSite:
name: leave-request-site
url: https://contoso.sharepoint.com/sites/intranet/leave-request
packages:
- id: contoso.leaverequest
- id: contoso.teamsite
version: 1.2.0-beta1
Such yaml aims to define:
JSON representation can be seen as following:
{
"galleries": [
{
"url": "http://metapackgallery.com/api/v2"
},
{
"url": "https://packages.nuget.org/api/v2"
},
{
"url": "http://www.myget.org/F/subpointsolutions-staging/api/v2"
}
],
"deployments": [
{
"TeamSite": {
"url": "https://contoso.sharepoint.com/sites/intranet",
"packages": [
{
"version": "1.0.0",
"id": "contoso.metadata"
},
{
"version": "1.2.0-beta1",
"id": "contoso.teamsite"
}
],
"name": "team-site"
}
},
{
"LeaveRequestSite": {
"url": "https://contoso.sharepoint.com/sites/intranet/leave-request",
"packages": [
{
"id": "contoso.leaverequest"
},
{
"version": "1.2.0-beta1",
"id": "contoso.teamsite"
}
],
"name": "leave-request-site"
}
}
]
}
Credential managements is to be define later but here are a few options:
Such configuration would enable orchestration of a complex provision over multiple packages, SharePoint sites and tenants. Configuration is a pure draft, to be refined and enhanced as we go.
The other bit of the configuration is "metapackfile.lock" file.
The aim is to lock down versions of the packages per SharePoint sites. That should be done by similar yaml/json config in the following way:
locks:
global:
packages:
- id: contoso.metadata
version: 1.0.0
- id: contoso.metadata.fields
version: 1.2.0
local:
- url: https://contoso.sharepoint.com/sites/intranet
packages:
- id: contoso.metadata
version: 1.0.0
- id: contoso.metadata.fields
version: 1.2.0
- url: https://contoso.sharepoint.com
packages:
- id: contoso.intranet
version: 1.2.0
- id: contoso.metadata.fields
version: 1.2.0
Global locks - all packages across the board.
Local locks - packages per particular site url.
The aim is to lock down versions which would (can be) deployed to target urls by default. MetaPack would check information in "metapackfile.lock" finding particular versions of the packages before deployment.
All commands should be able to switch default output. By default, Console.Out should be used.
MetaPack CLI should support 'list' command to install packages. Similar to NuGet, the following spec should be followed:
metapack list
Examples:
cls
Write-Host "Testing list command on local SharePoint" -fore green
.\metapack list --url 'http://dev13:31415'
Write-Host "Testing list command on O365" -fore green
.\metapack list --url 'https://contoso.sharepoint.com/sites/intranet' --username '[email protected]' --userpassword 'meme' --spversion 'o365'
--force flag needs to be introduced to enforce already deployed provision.
It should be possible to deploy only selected models from the whole package.
MetaPack should provide model container ID, so that one or more ID would be provided at the deployment time and only selected models will be deployed.
Options with key "_metapack.model.id" it to be used.
Metapack should have documentation covering the following areas:
Install scenario sometimes throws Microsoft.SharePoint.Client.Runtime {"File Not Found."} exception. Needs to be investigated and covered with regression tests.
Microsoft.SharePoint.Client.Runtime {"File Not Found."}
at Microsoft.SharePoint.Client.ClientRequest.ProcessResponseStream(Stream responseStream)
at Microsoft.SharePoint.Client.ClientRequest.ProcessResponse()
at Microsoft.SharePoint.Client.ClientRequest.ExecuteQueryToServer(ChunkStringBuilder sb)
at Microsoft.SharePoint.Client.ClientRequest.ExecuteQuery()
at Microsoft.SharePoint.Client.ClientRuntimeContext.ExecuteQuery()
at Microsoft.SharePoint.Client.ClientContext.ExecuteQuery()
at MetaPack.NuGet.Services.SharePointCSOMFileSystem.AddFileToSharePoint(Folder rootFolder, String filePath, Stream fileStream)
at MetaPack.NuGet.Services.SharePointCSOMFileSystem.AddFiles(IEnumerable`1 files, String rootDir)
at NuGet.PackageManager.ExpandFiles(IPackage package)
at NuGet.PackageManager.OnExpandFiles(PackageOperationEventArgs e)
at NuGet.PackageManager.ExecuteInstall(IPackage package)
at NuGet.PackageManager.Execute(PackageOperation operation)
at NuGet.PackageManager.Execute(IPackage package, IPackageOperationResolver resolver)
at NuGet.PackageManager.InstallPackage(IPackage package, FrameworkName targetFramework, Boolean ignoreDependencies, Boolean allowPrereleaseVersions, Boolean ignoreWalkInfo)
at NuGet.PackageManager.InstallPackage(IPackage package, Boolean ignoreDependencies, Boolean allowPrereleaseVersions)
at MetaPack.Client.Common.Commands.DefaultInstallCommand.<Execute>b__15_0(ClientContext context)
at MetaPack.Client.Common.Services.SharePointService.WithSharePointContext(String url, String userName, String userPassword, Boolean isSharePointOnline, Action`1 action)
at MetaPack.Client.Common.Commands.DefaultInstallCommand.Execute()
CI build should produce correct CLI package:
MetaPack Desktop client needs to be migrated under GitHub repository.
Metapack should support both CSOM/SSOM provision.
CSOM should be default implementation for provision.
Additional parameter should be added to the command line to suggests which runtime to use: SSOM or CSOM.
Implementation should work as NuGet based plugin similar to Cakebuild.
Newly build chocolatey package needs to be regressed. Pester is to be used potentially integrated with Cakebuild pipeline.
The following scenarios are to be tested:
SP2013 CSOM/SSOM might be delayed but core testing is a must.
MetaPack CLI should support 'push' command to push a package to NuGet gallery. Similar to NuGet, the following spec should be followed:
metapack push
The following activities are to be completed for the first beta:
Model packaging should be simplified for both SPMeta2 and PnP providers. API should hide all complexity of adding models, configuring the order of provisioning and unique ID management.
The following should be implemented:
So that we can indicate file name for logging.
As described, all commands should have a simple trace/progress callback so that external API can benefit from it.
Currently, NuGetListCommand only outputs all packages to console trace. Command should have additional property with collection of fetched packages.
Metapack API should be able to provision both SPMeta2/PnP models without NuGet packages. SolutionPackageDeploymentService should be able to provide such option.
That need is driven by 3rd party tooling that would be using only model deployment options without NuGet packaging.
That said, API should still run provision in isolation not interfering with the current application domain. That ensures API's ability to run provision under particular tool version, API and runtime.
MetaPack should allow to pick up the tool version while provisioning models.
Both tool version and provision runtime version (m2/pnp version) should be configurable.
MetaPack CLI should support 'install' command to install packages. Similar to NuGet, the following spec should be followed:
metapack install
Examples:
Write-Host "Testing list command on local SharePoint" -fore green
.\metapack install --url 'http://dev13:31415' --source 'https://www.myget.org/F/subpointsolutions-ci/api/v2' --id 'SPMeta2.CI' --version '1.2016.1015.0557'
Write-Host "Testing list command on O365" -fore green
.\metapack install --url 'https://contoso.sharepoint.com/sites/intranet' --source 'https://www.myget.org/F/subpointsolutions-ci/api/v2' --id 'SPMeta2.CI' --version '1.2016.1015.0557' --username 'meme@meme' --userpassword 'meme' --spversion 'o365'
MetaPack CLI should support 'update' command to install packages. Similar to NuGet, the following spec should be followed:
metapack update
Examples:
Write-Host "Testing list command on local SharePoint" -fore green
.\metapack update --url 'http://dev13:31415' --source 'https://www.myget.org/F/subpointsolutions-ci/api/v2' --id 'SPMeta2.CI' --version '1.2016.1015.0557'
Write-Host "Testing list command on O365" -fore green
.\metapack update --url 'https://contoso.sharepoint.com/sites/intranet' --source 'https://www.myget.org/F/subpointsolutions-ci/api/v2' --id 'SPMeta2.CI' --version '1.2016.1015.0557' --username 'meme@meme' --userpassword 'meme' --spversion 'o365'
GUI shouldn't crash on any exception happening.
As mentioned, DefaultUpdateCommand needs to be covered with regression tests.
Currently, metapack seems to fetch all installed versions of a target package. Tha results in a huge performance degradation.
The following needs to be implemented:
Log trace looks as following indicating fetch of all packages:
[Verbose]: Fetching files for path with filter: *.nupkg : MetaPack.SPMeta2.CI.1.2017.0311.2349
[Verbose]: Fetched files for path with filter: *.nupkg : metapack.spmeta2.ci.1.2017.0311.2349
[Verbose]: Opening file for path: metapack.spmeta2.ci.1.2017.0311.2349\MetaPack.SPMeta2.CI.1.2017.0311.2349.nupkg
[Verbose]: Opened file for path: metapack.spmeta2.ci.1.2017.0311.2349\MetaPack.SPMeta2.CI.1.2017.0311.2349.nupkg
[Verbose]: Fetching files for path with filter: *.nupkg : MetaPack.SPMeta2.CI.1.2017.0305.0748
[Verbose]: Fetched files for path with filter: *.nupkg : metapack.spmeta2.ci.1.2017.0305.0748
[Verbose]: Opening file for path: metapack.spmeta2.ci.1.2017.0305.0748\MetaPack.SPMeta2.CI.1.2017.0305.0748.nupkg
[Verbose]: Opened file for path: metapack.spmeta2.ci.1.2017.0305.0748\MetaPack.SPMeta2.CI.1.2017.0305.0748.nupkg
[Verbose]: Fetching files for path with filter: *.nupkg : MetaPack.SharePointPnP.CI.1.2017.0305.0803
[Verbose]: Fetched files for path with filter: *.nupkg : metapack.sharepointpnp.ci.1.2017.0305.0803
[Verbose]: Opening file for path: metapack.sharepointpnp.ci.1.2017.0305.0803\MetaPack.SharePointPnP.CI.1.2017.0305.0803.nupkg
[Verbose]: Opened file for path: metapack.sharepointpnp.ci.1.2017.0305.0803\MetaPack.SharePointPnP.CI.1.2017.0305.0803.nupkg
[Verbose]: Fetching files for path with filter: *.nupkg : MetaPack.SPMeta2.CI.1.2017.0311.2331
[Verbose]: Fetched files for path with filter: *.nupkg : metapack.spmeta2.ci.1.2017.0311.2331
[Verbose]: Opening file for path: metapack.spmeta2.ci.1.2017.0311.2331\MetaPack.SPMeta2.CI.1.2017.0311.2331.nupkg
[Verbose]: Opened file for path: metapack.spmeta2.ci.1.2017.0311.2331\MetaPack.SPMeta2.CI.1.2017.0311.2331.nupkg
[Verbose]: Fetching files for path with filter: *.nupkg : MetaPack.SPMeta2.CI.1.2017.0305.0707
[Verbose]: Fetched files for path with filter: *.nupkg : metapack.spmeta2.ci.1.2017.0305.0707
[Verbose]: Opening file for path: metapack.spmeta2.ci.1.2017.0305.0707\MetaPack.SPMeta2.CI.1.2017.0305.0707.nupkg
[Verbose]: Opened file for path: metapack.spmeta2.ci.1.2017.0305.0707\MetaPack.SPMeta2.CI.1.2017.0305.0707.nupkg
[Verbose]: Fetching files for path with filter: *.nupkg : MetaPack.SPMeta2.CI.1.2017.0311.2306
[Verbose]: Fetched files for path with filter: *.nupkg : metapack.spmeta2.ci.1.2017.0311.2306
[Verbose]: Opening file for path: metapack.spmeta2.ci.1.2017.0311.2306\MetaPack.SPMeta2.CI.1.2017.0311.2306.nupkg
[Verbose]: Opened file for path: metapack.spmeta2.ci.1.2017.0311.2306\MetaPack.SPMeta2.CI.1.2017.0311.2306.nupkg
[Verbose]: Fetching files for path with filter: *.nupkg : MetaPack.SPMeta2.CI.1.2017.0305.0751
[Verbose]: Fetched files for path with filter: *.nupkg : metapack.spmeta2.ci.1.2017.0305.0751
[Verbose]: Opening file for path: metapack.spmeta2.ci.1.2017.0305.0751\MetaPack.SPMeta2.CI.1.2017.0305.0751.nupkg
[Verbose]: Opened file for path: metapack.spmeta2.ci.1.2017.0305.0751\MetaPack.SPMeta2.CI.1.2017.0305.0751.nupkg
[Verbose]: Fetching files for path with filter: *.nupkg : MetaPack.SPMeta2.CI.1.2017.0312.0020
[Verbose]: Fetched files for path with filter: *.nupkg : metapack.spmeta2.ci.1.2017.0312.0020
[Verbose]: Opening file for path: metapack.spmeta2.ci.1.2017.0312.0020\MetaPack.SPMeta2.CI.1.2017.0312.0020.nupkg
[Verbose]: Opened file for path: metapack.spmeta2.ci.1.2017.0312.0020\MetaPack.SPMeta2.CI.1.2017.0312.0020.nupkg
Release CLI packages should be pushed to Chocolatey Gallery in an automated manner.
MetaPack.SPMeta2 should support incremental provision as per Implement 'incremental' provision support in SubPointSolutions/spmeta2#553
Should be an additional parameter for the MetaPack.SPMeta2 tool.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.