Jenkins Uitils

A collection of useful groovy functions for our Unreal Engine 4 Project with Perforce as source control. This package contains utilities for:

building with ue4 automation tools
running tests with ue4 automated test tools
communicating with p4 and swarm
communicating with mantis
communicating with discord
deploy to steam
parse logs with the Log Parser Plugin

See examples: Example folder

Motivation

This collection was written because I had no access to the underlaying Build & Test Server therefore I could not install certain plugins. I needed a discord and ue4 integration. This project is a learning project to learn groovy scripting for jenkins. Any feedback is welcome.

White list requirements

Please white list the following in your jenkins: https://jenkins-domain/scriptApproval/

method org.jenkinsci.plugins.p4.groovy.P4Groovy run java.lang.String java.lang.String[]

OS: Windows only.

collections of global functions:

newReviewNotif reviewId, REVIEW_URL, WEBHOOK [, description = "",reviewStatusParam = "New/Updated"]

notifes a discord webhook about a new review. Will indicate if its a new or an updated review. See p4c.isReviewUpdate()

notifReviewFailed REVIEW_URL, WEBHOOK [, description = ""]

notifes a discord webhook about that a review faild. If log.me() has been used the posed message will display the last logged stage!

notifReviewSuccess REVIEW_URL, WEBHOOK [, description = ""]

notifes a discord webhook about that a review sucessed

notifFailed change_URL, WEBHOOK [, description = ""]

notifes a discord webhook about that a build faild. If log.me() has been used the posed message will display the last logged stage!

notifSuccess change_URL, WEBHOOK [, description = ""]

notifes a discord webhook about that a build faild

example:

//... more code
    post { 
        always {
            cleanWs()
        }
        success {
            script{
                   notifSuccess CHANGE_URL, WEBHOOK
            }
        }
        failure {
            script{
                   notifFailed CHANGE_URL, WEBHOOK
            }
        }
    }

download url,filename[,credentials = null]

Will download a file via either cmd & curl or powershell. If credentials are given it uses cmd and basic auth. In this case it uses withCredentials([usernamePassword(... to get the credentials.

retrieveAndUnpackArchive(url,filename,destination[,format = "zip",credentials = null])

Will download a archive and unpack it at its destiination. It will remove the archive at the end and leaves a archivename.format.txt in the destination Note: See download, zip.unpack([...])

mkdir(path,name,[throwError=true])

creates a new directory. By default if the directory exists it will throw an error otherweise it will continue.

example: mkdir "${env.WORKSPACE}\\Content\\", "Data"

discord functions

The discord global variables are just utilites they allow to create a proper discord message as well as send utilities

discord.createMessage(title,status,fields[,url = null,content = null])

creates a discord message which then can be send. url and content are optional if not provieded they will not be added to the message.

discord.send(str,hook)

sends a message to the provieded webhook. Message can be a string should be a Discord confrom JSON string

discord.sendFile(file,hook)

sends a file to the provieded webhook

discord.mentionUser(message[,filterFile="discord_filter.json"])

Will replace any user set in the discord_filter.json file and mention them. It will replace all users with what has been spcifed in the file.

discord.mentionGroup(message[,filterFile="discord_filter.json"])

Will replace any group/role set in the discord_filter.json file and mention them.

Note: File format:

{
    "discord_users":{
        "Username":"ID"
    },
    "discord_groups":{
        "roleName":"&ID"
    }
}

log functions

Provides an interface to log warning, errors and infos

log.error(message)

logs an error

log.warning(message)

logs a waring

log.info(message)

logs an info

Example output: Warning: Test warning from log.warning

log.setup(rulefile = 'msbuildparserules.txt', rules = null)

In case you wanted to parse the log, the Log Parser Plugin needs a config file. the call log.setup() creates such for you. If you do not provide an argument the default name for the file is rulefile = 'msbuildparserules.txt'. If no rules are provied it uses the default rules.

log.me() Will keep track of the current stage globally. This can be accessed with log.lastStage() or env.GLOBAL_STAGE_NAME

log.lastStage() Will return the value of the last tracked stage with log.me() or changed value of env.GLOBAL_STAGE_NAME

Note: Is being used in notifFailed and notifReviewFailed if log.me() has been used.

Default rules:

# Divide into sections based on project compile start
start /BUILD COMMAND/
start /COOK COMMAND/
# Compiler Error
error /(?i)error [A-Z]+[0-9]+:/
error /(?i)Error:/
# Compiler Warning
warning /(?i)warning [A-Z]+[0-9]+:/
warning /(?i)Warning:/
warning /Couldn't find/

log.parse(projectRulePath = 'msbuildparserules.txt',parsingRulesPath = 'msbuildparserules.txt',showGraphs = true)

argument details see Log Parser Plugin. Important: This function can only be used if there is a msbuildparserules.txt (rule file) present or after the log.setup has been called.

mantis functions

Provides currently just a update function to update reports.

mantis.setUrl(url)

needs to be called in order to store the url to mantis globally.

mantis.setToken(token)

needs to be called at the beginning to set the token.

mantis.setup(url,token)

sets the url and the token

mantis.update(id,status,resolution,handler = null)

updates a report based on the id with the status (closed etc.) and the resolution (fixed etc.) and optional the handler (who)

Example:

pipeline {
   agent any

   stages {
      stage('Hello') {
         steps {
             script{
                mantis.setUrl("https://bugs.tld")
                mantis.setToken("TOKEN")
                mantis.update(9,"closed","fixed")
             }
         }
      }
   }
}

Comment:

Functions to parse context for #[bugID] [status] [resolutions] is planned. This can be used to parse a git or p4 commit message to close a bug after the build / test has been successful. Or leave a comment on mantis if the test failed.

perforce (p4c) functions

p4c stands for perforce client which makes use of the p4groovy plugin from jenkins itself. This collection enables you to manage swarm reviews and retrive the discrcibtion

perforce

p4c.pull(credential,workspace_template,format = "jenkins-${JOB_NAME}")

calls p4sync under the hood with a writable workspace and parallel runners. This will pull down the repositry. If this is a build with review action it will also pull down the correct review and populate P4_REVIEW / P4_CHANGE

p4c.getCurrentChangelistDescription(credential,client,view_mapping) / getCurrentReviewDescription(credential,client,view_mapping)

Will request via p4 describe via p4groovy p4.run("describe","-s","-S","12344") this returns the desciption of the current changelist. The function will call getReviewId() internally to get the current review / changelist number.

Troubleshoot: Mostlikely if <description: restricted, no permission to view> is the result the review (if called on are review) has been commited.

Important: the underlaying call will make the client/workspace WIRTABLE.

p4c.getChangelistDescription(id,credential,client,view_mapping)

Will request via p4 describe via p4groovy p4.run("describe","-s","-S","12344") this returns the desciption. Mostlikely if <description: restricted, no permission to view> is the result the review (if called on are review) has been commited.

Important: the underlaying call will make the client/workspace WIRTABLE.

p4c.getReviewId()

returns the current review Id

p4c.getChangelist()

returns the current changelist based on evaluate if P4_CHANGE exists or the variable change or json

p4c.getReviewStatus()

returns the current review status based on evaluate if the variable status or json exists

p4c.getReviewPass()

returns the current review pass url based on evaluate if the variable pass or json exists

p4c.getReviewFail()

returns the current review fail url based on evaluate if the variable fail or json exists

p4c.isReviewUpdate() - workaround

returns true if the current review is an update or a new commit (false).

Important: only works if the pass / fail parameter is given!

p4c.isCommitted()

helper that returns true/false if a changelist has been already commited!

Internal reviewObject()

returns a json object of the current review build request.

p4c.swarmUrl(credential,client,mapping)

returns the swarm review of the current client. All parameters are required! (Calls p4)

Example:

stage('p4 sync'){
    steps{
        script{
            p4c.pull(P4USER,P4CLIENT)
        }
    }
}

Note: The swarm_url param means the actual swarm url not the review url.

p4c.comment(review,user,ticket,swarm_url,comment)

leaves a comment at the given swarm review.

Note: please use the swarm header if possible.