Code Monkey home page Code Monkey logo

psdevops's Introduction

PowerShell Tools for DevOps

Gallery Downloads Build Status Build Status

PSDevOps helps you automate DevOps using PowerShell.

What is PSDevOps?

PSDevOps is a PowerShell module that makes it easier to automate DevOps with PowerShell.

What do you mean 'Easier to Automate'?

If you're familiar with PowerShell, you might know about the Object Pipeline.
This allows you to pass objects together, instead of parsing text at each step. While this is great, not many PowerShell commands or REST apis take full advantage of this feature.

PSDevOps does.

Almost every command in PSDevOps is pipeline-aware.
Related commands can often be piped together, and command results can often be piped back into a command to return additional information.

Additionally, PSDevOps output is often 'decorated' with extended type information.

This allows the PowerShell types and formatting engine to extend the return object and customize it's display.

Automate Azure DevOps

The Azure DevOps REST API can be challenging to work with, and hard to remember.

PSDevOps provides dozens of commands to automate Azure DevOps.

Commands are named simply and clearly, using PowerShell's verb-noun convention, for example, Get-ADOProject, Get-ADORepository, Get-ADOBuild

To see all commands from PSDevOps, run:

Get-Command -Module PSDevOps

Unlike many modules, these commands make use of the full feature set of PowerShell. Here are a few things PSDevOps does differently:

Using the Object Pipeline

The Object Pipeline is a core feature of PowerShell that allows you to send structured information from step to step.

Almost every function in PSDevOps is "pipeline aware", and can accept multiple types of objects

Connect-ADO -Organization $MyOrganization -PersonalAccessToken $MyPat
Get-ADOProject | Get-ADOTeam | Get-ADODashboard

Many commands can be piped back into themselves to return additional results, for instance:

Get-ADOBuild -Project MyProject -First 5 | # Get the first 5 builds 
    Get-ADOBuild -ChangeSet # Get the associated changesets.
Get-ADOAgentPool | # Gets agent pools from the organization
    Get-ADOAgentPool  # Gets individual agents within a pool, because the prior object returned a PoolID.

Invoke-ADORESTApi

In orer to ensure that you can always work with Azure DevOps, even if there isn't already a function in PSDevOps, there's Invoke-ADORestAPI.

Invoke-ADORestAPI can be used like Invoke-RESTMethod, and also has a number of additional features.

For instance:

  • -AsJob (Launch long-running queries as a job)
  • -Body (autoconverted to JSON)
  • -ExpandProperty (Expand out a particular property from output)
  • -PSTypeName (apply decoration to output)
  • -UrlParameter (Replace parameters within a URL)
  • -QueryParameter (Pass parameters within a querystring)

Help

Like any good PowerShell module, PSDevOps has help. Run Get-Help on any command to learn more about how to use it.

Get-Help Get-ADOBuild -Full

Commands that wrap the REST api should have a .LINK to the MSDN documentation on the API to help you understand what they are doing.

Custom Formatting

The Azure DevOps REST API can return a lot of unhelpful information. To make it easier to work with Azure DevOps in Powershell, PSDevOps includes several custom formatters.

For a simple example, try running one of the following commands:

Get-ADOProject
Get-ADOTeam -Mine

Extended Types

The Azure DevOps REST api often returns data that is inconsistently named or weakly typed.

Where possible, PSDevOps uses the Extended Type System in PowerShell to augment the values returned from Azure DevOps.

For example, when you run Get-ADOAgentPool, it will add two properties to the return value: PoolID (an alias to ID) and DateCreated (which converts the string date in .CreatedOn to a [DateTime]).

Supporting -WhatIf and -Confirm

Most commands in PSDevOps that change system state SupportShouldProcess, and have the automatic parameters -WhatIf and -Confirm.

-Confirm works the same in PSDevOps as it does in any PowerShell module. Passing -Confirm will always prompt for confirmation, and Passing -Confirm:$false will never prompt for confirmation.

PSDevOps does a bit better than most modules when it comes to supporting -WhatIf. In most modules, -WhatIf will write a message to the host about what might have run. In PSDevOps, passing -WhatIf should return the values about to be passed to the REST API.

New-ADOProject -Name Foo -Description bar -Confirm # Prompts for confirmation

New-ADOProject -Name Foo -Description bar -WhatIf  # Returns the data that would be passed to the REST endpoint. 

Creating Complex Pipelines

While Azure DevOps templates are nice, they don't give you syntax highlighting for the scripts of each step.
Also, cross-repository templates can be painful.

PSDevOps allows you to create Azure DevOps pipelines using New-ADOPipeline.

For instance, this create a cross-platform test of the current repository's PowerShell module.

New-ADOPipeline -Job TestPowerShellOnLinux, TestPowerShellOnMac, TestPowerShellOnWindows

This creates a multistage pipeline that does PowerShell static analysis, tests the current module (crosssplatform), and updates the PowerShell gallery using a Secret:

New-ADOPipeline -Stage PowerShellStaticAnalysis, TestPowerShellCrossPlatform, UpdatePowerShellGallery

This little one liner works wonderfully to build a CI/CD pipeline in Azure DevOps around almost any PowerShell modules.

Parts are stored in a \ado\PartName\ as either a .ps1 or a .psd1 file. PS1 files will implicitly become script tasks. PSD1 files will be treated as task metadata, and can reference other parts.

Any module that contains an \ADO directory and is tagged 'PSDevOps' or requires PSDevOps can contain parts. Parts found in these modules will override parts found in PSDevOps.

Advanced Azure DevOps Pipeline Logging

PSDevOps can also be used to help you write information to the a pipeline's timeline. This can be considerably easier than memorizing the Logging Commands Documentation.

PSDevOps makes this much nicer by abstracting away this ugliness into easy-to-use commands:

  • Add-ADOAttachment
  • Set-ADOBuild
  • Set-ADOEndpoint
  • Set-ADOVariable
  • Write-ADOError
  • Write-ADOProgress
  • Write-ADOWarning

Dealing with DevOps

DevOps is a hybrid discipline combining the skillset of Devolopers and Operations.
With DevOps, the focus is on automation, and PowerShell is often the language of choice.

By convention, most developers write their scripts according to a psuedostandard:

*-*.ps1 Scripts containing a function
*.*.ps1 'Special' Scripts, often used by particular modules *.ps1 Simple scripts that are run interactively.

PSDevOps has a command, Get-PSDevOps, that helps to identify these scripts and their requirements.

Get-Module PSDevOps | Get-PSDevOps

PSDevOps GitHub API

PSDevOps also provides a few functions to work with the GitHub API.

  • Connect/Disconnect-GitHub
  • Invoke-GitHubRESTAPI

Invoke-GitHubRESTAPI works like Invoke-ADORestAPI, as a general-purpose wrapper for GitHub REST API calls.

It also has a number of additional features, for example:

  • -AsJob (Launch long-running queries as a job)
  • -Body (autoconverted to JSON)
  • -ExpandProperty (Expand out a particular property from output)
  • -PSTypeName (apply decoration to output)
  • -UrlParameter (Replace parameters within a URL)
  • -QueryParameter (Pass parameters within a querystring)

Because GitHub's REST api is predictable and exposed with OpenAPI, Invoke-GitHubRESTAPI also enables two very interesting scenarios:

  1. Connect-GitHub can automatically create a shortcut for every endpoint in the GitHub API
  2. Invoke-GitHubRESTAPI can automatically decorate return values more apporopriately.

This means that PSDevOps can integrate with GitHub's REST API with a very small amount of code, and easily customize how GitHub output displays and works in PowerShell.

Write GitHub Actions

You can automatically generate GitHub actions off of any PowerShell script or command.

First, create a /GitHub/Actions folder in your module directory, then put one or more .ps1 files in it.

Then,

Import-BuildStep -ModuleName YourModule

Then, you can generate your action.yml with some code like this.

New-GitHubAction -Name "Name Of Action" -Description 'Action Description' -Action MyAction -Icon minimize -ActionOutput ([Ordered]@{
    SomeOutput = [Ordered]@{
        description = "Some Output"
        value = '${{steps.MyAction.outputs.SomeOutput}}'
    }    
})

Write GitHub Workflows

You can use PSDevOps to write complex Github Workflows using the same techniques as Azure DevOps pipelines:

New-GitHubWorkflow -Name RunPester -On Demand, Push, PullRequest -Job TestPowerShellOnLinux

As with Azure DevOps, parts of the workflow can be defined within the \GitHub subdirectory of PSDevOps or any module.

Advanced GitHub Workflow Logging

PSDevOps also includes commands to make logging within a GitHub workflow easier. They are:

  • Hide-GitHubOutput
  • Write-GitHubDebug
  • Write-GitHubError
  • Write-GitHubOutput
  • Write-GitHubWarning

psdevops's People

Contributors

dfinke avatar startautomating avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

psdevops's Issues

Get-ADOUser Errors out

Hello PSDevOps Team,

I am Verly glad to find a module to manage Azure DevOps, I am trying to get a list of users at the Org level.

but I am hitting an error while I run the following command. I am running it in 7.2.0-preview.10. I am successfully connected to the organization by running Connect-ADO. and have right privileges too.

Get-ADOUser -Organization <my-org-name>

Output:
image

Thanks once again for your AWESOME work :-)

Improve Layout Support

It would be nice to be able to edit the layout of a given work item type using cmdlets.

order properties when generating yaml

When generating a yml file for a work flow, order properties logically in terms of the code readability, e.g.

steps: 
  - name: InstallPester
    shell: pwsh
    env: 
        PesterMaxVersion: ${{secrets.PesterMaxVersion}}
    run: |

shell and env are important metadata that give context to "run" and since run is arbitrarily long, you can easily find the other properties if they come first.

Get-ADORepository -PullRequestID is not Mandatory

This causes an ambiguity between parameter sets which errors out some tests.

@cjboden I'll take care of fixing this (I think your tests didn't end up running it because you wouldn't be able to see repos). If you end up adding more, make sure each new parameter set has at least one unique "key" parameter that lets PowerShell know which parameter set it could use.

Implement Set-ADOParameter

I would like to pitch the idea of implementing a Set-ADOParameter cmdlet/function.

Set-ADOParameter -Organization OrgName -Project ProjName -ApiVersion 5.1 -PersonalAccessToken "1232423421...." -BranchName "$/Financials/Trunk/Development"

It would then create new variables, that would last for the current session. We would then update the cmdlets/functions to have default values, which would be the names of the variables.

E.g.

function Get-ADOBuild
{
...
    [Parameter(ValueFromPipelineByPropertyName)]
    [Alias('Org')]
    [string]
    $Organization = $Script:PsDevOpsOrganization,

    [Parameter(ValueFromPipelineByPropertyName)]
    [string]
    $Project = $Script:PsDevOpsProject,

    [Parameter(ParameterSetName='build/builds')]
    [string]
    $BranchName = $Script:PsDevOpsBranchName,

    [Alias('PAT')]
    [string]
    $PersonalAccessToken = $Script:PsDevOpsPersonalAccessToken,

    [string]
    $ApiVersion = $Script:PsDevOpsPersonalAccessToken,
}

Afterward calling the Get-ADOBuild cmdlet/function would be as slim as

Get-ADOBuild

If you need to override the different parameters, even after setting the parameters in the same session, you can simply override the value by using the parameters for each cmdlet/function.

I believe it would increase the user experience and make it easier for people to start using the module, because they can reduce the bloat in their console and scripts, while still having all the benefits from the module.

Add Get-WorkItemQueryResult cmdlet?

We're looking to synchronize the work items in Azure DevOps with another system that we cannot control, and we're thinking that we can use your module to do so. I'm going to start exploring the Wiql - Query By Wiql endpoint to make that work in our organization.

I thought I'd simultaneously open this issue since I noticed that PSDevOps does not have a Get-WorkItemQueryResult cmdlet. I'm not sure if the stub I build out will be helpful but wanted to get your thoughts on the feasibility of adding this cmdlet to your module.

Are you interested in that sort of contribution? Also any hints as I start setting up a development environment?

Invoke-ADORestAPI -AsByte is slow

It appears that it is unrolling each byte. This will take a while. It would be better to directly return a byte[] before any results are unrolled.

Could one 'query' work items while connected anonymously?

Running Connect-ADO -Organization ayn without credentials or PAT looks like it connected successfully. However, Get-ADOWorkItem -Organization ayn -Project PowerShell -Query 'Select [System.ID] from WorkItems' fails with error below

Invoke-ADORestAPI : Response was HTML, Request Failed.
At C:\Users\ayan\Documents\PowerShell\Modules\psdevops\0.5.3\Get-ADOWorkItem.ps1:240 char:29
+             $queryResults = Invoke-ADORestAPI @invokeParams |
+                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (
<!DOCTYPE h...ody>
</html>
:String) [Invoke-ADORestAPI], Exception
    + FullyQualifiedErrorId : ResultWasHTML,Invoke-ADORestAPI

...even though this is a public project and accessible online without authentication. PFB link
https://dev.azure.com/ayn/PowerShell/_workitems

When I Connect-ADO with a PAT it works fine. Is there a way to connect anonymously like on can thru the browser, or does one need to log in even for public projects?

Bootstrap using ADO as a PSRepository

In order to facilitate the use of Artifact Feeds as a place to store PowerShell modules, a command that can Register artifact feeds as a PSRepository would be nice.

Add-ADOAreaPath

Add-ADOAreaPath does not handle nested paths correctly. Please update with fix.

Getting Started and Prerequisites

Hi,
It might be good to have getting started section with an example connection and listing of certain features.
Also are there any prerequisites in terms of version of PowerShell or any other feature or dependencies on certain tool/packages.

Thanks

Feature Request: Retrieve Pull Request by ID

It would be nice to be able to retrieve a specific pull request by ID using Get-ADORepository. Perhaps something like this:

Get-ADORepository -PullRequest -PullRequestID 341

Add support for parameter comments using New-ADOPipeline

I like to comment my pipeline parameters when writing templates. Please consider adding support for something like this:

param (
    # List of required modules to import.
    [string[]] $requiredModules
)

โฉ

parameters:
  # List of required modules to import.
  - name: requiredModules
    ...

Add-ADOTeam doesn't seem to work

Try to add a team to a group. Invoke-ADORestAPI returns:

"Invoke-ADORestAPI : The remote server returned an error: (411) Length Required.
At C:\Users\user\Documents\WindowsPowerShell\Modules\PSDevOps\0.4.5\Add-ADOTeam.ps1:135 char:13
Invoke-ADORestAPI @invokeParams"

Adding an else clause into Invoke-ADORestAPI with the following line "$WebRequest.ContentLength = 0" seems to fix it.

Picklists

I'm having an issue tracking the picklist ID associated with a field. I am looking for a way to programmatically update picklists and finding this picklist ID given a process ID and field name would help do the trick.

Add Set-ADOTeamAdmin

Add a new Set-ADOTeamAdmin to PSDevOps. This will allow an admin group from another Project, as well as individual users, to be added as a Team Administrator

Feature request: Adding and removing Agent pools (and agents)

I would be very convenient if the module could add and remove agent pools, using commands like New-ADOAgentPool and Remove-ADOAgentPool.

Similarly, it would be very helpful to be able to remove agents from a pool. For example, when a self-hosted agent VM has been decommissioned. A command for that could be Remove-ADOAgent.

Of course I don't expect you to just implement this. Everyone has a very busy schedule. It's just an idea for some new features that would certainly be useful for me. In any way, thanks and keep on doing the good work ๐Ÿ˜„

Improve support for YML arrays in New-ADOPipeline

When using YML arrays in a PowerShell step you need to convert it into a array by doing, as far as I know, something like this:

$requiredModules = "${{ join(';',parameters.requiredModules) }}" -split ';' | Where-Object { $_ }
  1. Join the array into a string with the pipeline join function
  2. Split the now string into a PowerShell array
  3. Use ... | Where-Object { $_ } to get rid of any null values (added if passing in [])

Currently the output generated looks something like this:

$parameters.requiredModules -split ';' -replace '^[''"]' -replace  '[''"]$'

Make Setting Azure DevOps Permissions Easier

Set-ADOPermission gives a rather raw view of a complicated API. It would be nicer if individual objects could be piped to Set-ADOPermission in order to secure them (much as Get-ADOPermission can get permissions related to a repo)

Create Work Item States

Would like a new cmdlet to create and update custom work item states in custom process templates

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.