mockito / shipkit Goto Github PK

Currently GitHub repo for querying issues is hardcoded in org.mockito.release.notes.improvements.GitHubTicketFetcher.GitHubIssues This is a showstopper for making the release tools useful in other projects.

I'm having problems using release tools in Mockito project. Mockito release build uses JDK6. Stack trace (full log):

> Could not resolve all dependencies for configuration ':classpath'.
   > Could not resolve gradle.plugin.org.mockito:mockito-release-tools:0.3.9.
     Required by:
         :mockito:unspecified
      > Could not resolve gradle.plugin.org.mockito:mockito-release-tools:0.3.9.
         > Could not get resource 'https://plugins.gradle.org/m2/gradle/plugin/org/mockito/mockito-release-tools/0.3.9/mockito-release-tools-0.3.9.pom'.
            > Could not GET 'https://plugins.gradle.org/m2/gradle/plugin/org/mockito/mockito-release-tools/0.3.9/mockito-release-tools-0.3.9.pom'.
               > sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
      > Could not resolve gradle.plugin.org.mockito:mockito-release-tools:0.3.9.
         > Could not get resource 'https://plugins.gradle.org/m2/gradle/plugin/org/mockito/mockito-release-tools/0.3.9/mockito-release-tools-0.3.9.pom'.
            > Could not GET 'https://plugins.gradle.org/m2/gradle/plugin/org/mockito/mockito-release-tools/0.3.9/mockito-release-tools-0.3.9.pom'.
               > sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Currently considered solution: avoid building on Java6, use animal sniffer to detect Java6 incompatibilities (mockito/mockito#1021)

Useful resources:

• http://stackoverflow.com/questions/29584328/gradlew-bat-and-gradlew-sslhandshakeexception
• https://github.com/arteme/gradle-trust-all
• http://stackoverflow.com/questions/3242335/how-to-use-ssl-with-a-self-signed-certificate-in-groovy
• https://discuss.gradle.org/t/unable-to-download-plugins-due-to-ssl-pkix-error/21376
• https://discuss.gradle.org/t/gradle-1-12-wrapper-fails/2534/19
• https://discuss.gradle.org/t/gradle-1-12-wrapper-fails-unable-to-find-valid-certification-path-to-requested-target/2079
• https://discuss.gradle.org/t/urgent-ssl-apache-configuration-for-services-gradle-org-is-bad/8808/13
• http://stackoverflow.com/questions/29584328/gradlew-bat-and-gradlew-sslhandshakeexception#answer-30924895

Subtask of #19.

Currently, the "org.mockito.release-notes" plugin is not usable outside of Mockito project.

• Document org.mockito.release-notes Gradle plugin and the extension
• Convert the groovy plugin to java
• Decouple auth token from Mockito
• #26, Make the target repository for querying issues configurable (currently hardcoded to Mockito)

External contributors don't know how to end-to-end test the release notes generation. See this excellent pull request: #12 (the user is asking how to make e2e test). Without this, it is hard to approve and merge PRs with confidence.

At very least we need a clear documentation how to do that with Mockito. Ideally, we have a test bed project for that on GitHub.

Currently when we fork processes during the release the output looks like this (full log):

:api:bintrayUpload - publishing to Bintray
  - dry run: false
  - repository: mockito-release-tools-example-repo
  - version: 0.10.0
  - Maven Central sync: true
:api:bintrayUpload FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':api:bintrayUpload'.
> Could not upload to 'https://api.bintray.com/content/mockito/mockito-release-tools-example-repo/mockito-release-tools-example/0.10.0/org/mockito/release-tools-example/api/0.10.0/api-0.10.0.jar': HTTP/1.1 409 Conflict [message:Unable to upload files: An artifact with the path 'org/mockito/release-tools-example/api/0.10.0/api-0.10.0.jar' already exists]
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 1.891 secs
:travisRelease FAILED
FAILURE: Build failed with an exception.
* Where:
Script '/home/travis/build/mockito/mockito-release-tools-example/gradle/cd.gradle' line: 214
* What went wrong:
Execution failed for task ':travisRelease'.
> Process 'command './gradlew'' finished with non-zero exit value 1

• We should prefix the standard output from processes that we fork from the build. This way, it should be easier to investigate problems in Travis CI log.
• We should also provide custom exception message when forked process fails. The message should hint to the user to investigate the log output with given prefix.
• Every forked process should neatly write to the log the exact command line that is executed (including arguments).
• We need a way to hide output from commands that can potentially leak secret variables (for example GitHub write token used for push can get exposed when git push fails).

[Gradle] :api:bintrayUpload - publishing to Bintray
[Gradle]   - dry run: false
[Gradle]   - repository: mockito-release-tools-example-repo
[Gradle]   - version: 0.10.0
[Gradle]   - Maven Central sync: true
[Gradle] :api:bintrayUpload FAILED
[Gradle] FAILURE: Build failed with an exception.
[Gradle] * What went wrong:
[Gradle] Execution failed for task ':api:bintrayUpload'.
[Gradle] > Could not upload to 'https://api.bintray.com/content/mockito/mockito-release-tools-example-repo/mockito-release-tools-example/0.10.0/org/mockito/release-tools-example/api/0.10.0/api-0.10.0.jar': HTTP/1.1 409 Conflict [message:Unable to upload files: An artifact with the path 'org/mockito/release-tools-example/api/0.10.0/api-0.10.0.jar' already exists]
[Gradle] * Try:
[Gradle] Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
[Gradle] BUILD FAILED
[Gradle] Total time: 1.891 secs
:travisRelease FAILED
FAILURE: Build failed with an exception.
* Where:
Script '/home/travis/build/mockito/mockito-release-tools-example/gradle/cd.gradle' line: 214
* What went wrong:
Execution failed for task ':travisRelease'.
> Process 'command './gradlew'' finished with non-zero exit value 1

Introduction

The success of the project depends on popularity. We want to gamify continuous delivery, promote frequent releases and release automation goodness. Let's use real estate available in the release notes file to promote the project and get more folks aware of what we're doing and why.

Suggested implementation

During updating release notes file we can add (or modify if exist) this sentence on the top of release note file:

This release notes was automatically generated by shipkit X times.

Where X start from 1 and it's incremented by each new released version. This sentence should be formatted as small if possible.

We can make this potentially configurable. Parsing and loading the release notes file should not be difficult.

Code

• this is how we update release notes file
• and that's how we create release notes content for new version

Looking forward to reviewing your pull request! :)

Let's communicate with the world about the goals of Mockito release tools (#21). An article or publication on the matter (Mockito blog, LinkedIn publishing platform, twitter).

Problem

Artifact created by mockito-release-tools pulls in "junit" transitive dependency. When release plugin is used here is the build environment (Gradle buildscript classpath):

...
+--- gradle.plugin.org.mockito:mockito-release-tools:0.0.3
|    \--- com.googlecode.json-simple:json-simple:1.1.1
|         \--- junit:junit:4.10 (*)
...

We don't want junit to be on client classpath when release plugin is used because it is a test dependency and should not be littering the build classpath.

Root cause

json-simple library pulls is junit as compile dependency, see issue fangyidong/json-simple#91. To solve the problem we should use a different json library. Let's choose a smallest, simplest json library that has no other dependencies. I suggest we use to fork for json-simple that seems to be well maintained: https://cliftonlabs.github.io/json-simple. We can use 1.x version or 2.x (has some deprecations, some updates in our code would be needed).

It would be useful to produce separate Java libraries from mockito-release-tools. Example submodules:

• org.mockito:mockito-release-tools-git
• org.mockito:mockito-release-tools-github
• org.mockito:mockito-release-notes-generator
• org.mockito:mockito-release-notes-gradle-plugin
• org.mockito:mockito-continuous-delivery-gradle-plugin

This way it is easier to manage the code, ensure the right architecture. It could be easier for the community to reuse some parts of the pipeline.

Asked on Gradle forums how to publish multiple jar to Gradle plugin portal: https://discuss.gradle.org/t/publishing-plugin-with-other-jars-from-a-multi-project-plugin/21551

Use case

In order to reward contributors and further motivate for more high quality contributions it would be great to list contributors in the relevant section of pom.xml file. As of 3/18/17 generation of pom.xml logic is a part of Mockito project automation and it is not generic enough to be part of mockito release tools.

Design

This idea needs to be fleshed out further. An example technical design that we could try implementing:

• currently every contribution is explicitly mentioned in the release notes but it should also automatically include the contributor in the pom.xml of every future version of mockito.
• see TODO with brainstorming ideas in the code
• need a way to generate pom.xml template with contributor information for past releases so that we include contributors of the past (we can simplify and ignore contributors of 1.x version).
• Maven pom reference for "contributors" section: http://maven.apache.org/pom.html#Contributors

Action plan

This is a great ticket for external contribution! Please consider helping out!

• The next action is to think through the design, perhaps do some prototyping and comment back on this ticket with the technical design we actually want to execute. The previous design note was my quicky brainstorming.
• Submit PR! (@mstachniuk, wdyt?)

Hi, I have problem with running any of example tasks from readme.

I've cloned mockito-release-tools-examples

run './gradlew testRelease' - resulting with: Missing 'bintray.key' value. Please configure Bintray extension or export 'BINTRAY_API_KEY' env variable.

run './gradlew testNotableRelease' - resulting with: Task 'testNotableRelease' not found in root project 'example-release'.

run './gradlew performRelease -m' (-m means dry run) - resulting with: Missing 'bintray.key' value. Please configure Bintray extension or export 'BINTRAY_API_KEY' env variable.

run './gradlew previewReleaseNotes' - resulting with: FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':previewReleaseNotes'. > Can't deserialize JSON:

Can you give some more informations about running examples?

Thanks!

Mockito build cannot publish at the moment because release needed is always false. From the build log:

  Current branch 'HEAD' matches 'master|release/.+': false
:assertReleaseNeeded FAILED

Note that the task has identified the current branch as 'HEAD'. It should identify it as "master" or something like that.

I'm working on a resolution right now.

Sometimes it's needed to ignore some users in release notes and in contributors list in pom.xml. It can be e.g. a technical account like continuous-delivery-drone in Mockito project. The scope of this task is to add a possibility to ignore defined users in release notes and contributors list in pom.xml.

Problem

Publishing new version when it is binary equal to the previous version is confusing for the clients. At the moment we don't check if new version is different than the current release. If someone makes a change in .gitignore it can trigger build and potentially a publication.

Suggested implementation

Only a suggested plan, please execute as you see fit!

Robustness of the algorithm

The comparison algorithm is not on the critical path and we are allowed to be pragmatic in coming up the implementation and we can iterate on the solution to make it better. Here are the effect of false positive / false negative signal from the algorithm:

• false negative, publications incorrectly considered non-equal: extra version is released that does not have changes on top of previous version. It is minor problem for projects that leverage the "notable releases" feature. For other projects it should not be a huge deal because versions are shipped frequently in continuous delivery.
• false positive, publications incorrectly considered equal: release is not triggered and needs to be triggered semi-manually. Not a critical problem.

Gradle plugins / tasks

1. We have a "releaseNeeded" task that determines whether the release should be triggered. This task needs to depend on some other task(s) that pull down and compare publications.
2. We could create "PublicationsComparatorPlugin" that applies "BaseJavaLibraryPlugin" and "VersioningPlugin". New plugin would add task that that compares publications. New plugin would be applied by "JavaLibraryPlugin".
3. releaseNeeded would depend on all tasks added at step 2) from projects that use PublicationsComparatorPlugin.

Implementation

1. It is problematic to use Gradle's configurations to download previous versions of jars / poms for comparisons because Gradle automatically replaces dependencies with local project dependency with the version that is currently being built. We could reach out to Gradle folks on the forum and ask how to do it. If we cannot get it to work, we need to hit the url directly and download file without using Gradle's dependency resolution.
2. The task should be incremental, e.g. when nothing changes and build dir is intact, we don't download jars again.
3. Current publication model is that we we publish all jars from the project during the release. We want to stick with this model of releasing because it is simple and powerful - the project is a unit of deployment/release and all jars go out with the same version. If only one jar has changed, we still publish all jars with new version.
4. Stuff to consider when comparing publications:

• pom.xml content
• jar content / binaries (for that, perhaps it's simpler to just compare the sources jar, it will cover compiled classes and javadocs at the same time)
• MANIFEST.MF from jars

5. There is already some code to be reused, legacy of Mockito automation :) See "org.mockito.release.internal.comparison" package.

Problem

Currently mockito-release-tools works only with github.com. A path to it is hardcoded in the code :(. This project needs to works with GitHub Enterprise API otherwise it cannot be used in enterprise setting. A property to configure GitHub Enterprise needs to optional and by default it should works with github.com.

Suggested implementation

ReleaseConfiguration.GitHub should have an optional setting for the url base host:

releasing {
    github.apiUrl = "https://github.in.your.company.com/api/v3"
    github.url = "https://github.in.your.company.com"
}

• ReleaseConfiguration.GitHub class that needs new properties.
• Example of hardcoded GitHub API url, there are other places but this one is a good starting point.

In order to make the local builds faster (and testing faster!) let's make updateNotableReleaseNotes task incremental (NotableReleaseNotesGeneratorTask).

Suggested design

NotableReleaseNotesGeneratorTask class currently fetches release notes data from git / GitHub every time it runs. Instead, let's split the logic into discrete steps to and leverage incremental build. The first step of NotableReleaseNotesGeneratorTask currently is to fetch Collection from generator object. Let's push that logic to a separate Gradle task, "fetchNotableNotes" (type: NotableNotesFetcherTask). The new new task will:

• fetch the data and construct Collection
• save it to the file (we can use Java serialization for now but we really want something nicer, like JSON)
• declare correct inputs and outputs so that it is incremental, inputs are various properties currently available on NotesGeneration type. Output is a file.

The existing NotableReleaseNotesGeneratorTask needs to:

• load release notes data (Collection) from the file
• depend on "fetchNotableNotes" task

Testing and prototyping

• see Mockito Release Tools readme for info on testing
• run "./gradlew updateNotableReleaseNotes" in example project

Let's create initial contributing guide and incrementally build it when giving code reviews.

Mockito's contributing guide: https://github.com/mockito/mockito/blob/release/2.x/.github/CONTRIBUTING.md

Let's document the design and implementation principles, and guidelines. Examples:

• why we're implementing Gradle plugins in Java
• public VS internal API separation
• semantic versioning

To automate the release we need to to manage the version of the project (version bumps, updating the "version.properties" file, etc.).

Currently, version bumping is implemented with "versionFile" extension. Ideally, there is a plugin that contains this functionality.

Goal

Mockito Continuous Delivery Pipeline 2.0 (mockito/mockito#911) requires implementation of a "notable" release and "regular" release.

Use cases

1. Automatically publish to notable release repo and update concise release notes file when:

• we are building minor or major version e.g. "x.0.0" or "x.y.0", for example: 1.0.0, 3.5.0
• or "[notable release]" is found in commit message

2. It is possible to "nominate" already released version as notable

Implementation

1. Workflow

• When criteria are met:
  • we're building is new major/minor (x.0.0 or x.y.0)
  • or "[notable release]" substring in commit message
  • and default criteria are met (successful build, releasing from right branch, etc.)
• The plugin does:
  • publication to regular Bintray repo
  • publication to notable Bintray repo
  • update "docs/release-notes/notable.md" file with concise, notable notes
  • update "docs/release-notes/official.md" with detailed notes
  • update notable version in "version.properties" file and add new notable version

2. New content of "version.properties" file:

#All libraries will be built and published with following version:
version=2.7.5
#Notable versions are published to Central + JCenter and are documented in "docs/release-notes/notable.md"
notableVersions=2.1.0, 2.2.0, 2.3.0, 2.4.0, 2.5.0, 2.6.0, 2.7.0

3. Notable release notes file is regenerated (and checked-in) with every release build based on the "notableVersions" information from the "version.properties" file.

I've ported current version of release tools to Mockito, PR is open: mockito/mockito#1018 I noticed that the release is slow due to getting contributors from GitHub. When I commented out the code that gets the contributor/GH id mapping the build takes 10secs instead of over a minute.

Suggested plan:

• - disable getting contributors from the notable release notes (notable notes don't use contributors) - #59
• - try to address the issue so that the build is fast

Problem

Producers and consumers of software libraries don’t have an easy life. As a consumer of a library I don’t know if new version of the library brings API incompatibilities until I try to use it and it blows up, hopefully during compilation or testing, and not on production! As a producer of a library I can unintentionally sneak in API incompatibilities, unless I am very careful and/or great engineers review my code and spot the problems.

Let’s make the life of producers and consumers of libraries delightful!!!

Solution

What if producer gets nice report or even build failure if he changes the API without bumping major version? What if consumer gets nice report, or even early build failure if known API incompatibilities exist in the new version of a library he attempts to use? What if the incompatibilities are computable statically and it is not necessary to compile code/run tests to identify issues? What if the tools can tell you with good accuracy what version of a library you can consume safely?

First steps

While we don’t necessarily need to scope the entire solution we can to scope down an initial feature set that can help us validate if the problem is worth solving and how. Below features are suggestions and should be critically reviewed by whoever decides to own this feature.

Does my change break API compatibility?

To get started, we can create a Gradle task that compares 2 binaries, identifies API incompatibilities and reports them. Tools that identify API incompatibilities already exist and can be leveraged (mockito/mockito#738). In the project we already have code that pulls down the previously released binaries (implemented as part of #84).

Suggested implementation

Suggested implementation should be enough to get started. However, please discuss and suggest different approach as you see fit.

• Create a new plugin ApiCompatibilityPlugin that will add new Gradle task of type CheckApiCompatibilityTask (new task type). The new plugin is applied by our continuous delivery plugin, however, we don’t hook up the new task to the workflow yet.
• When the task runs, it pulls down previously released binaries reusing the code we already have
• Then, it reports binary incompatibilities. For example it produces a report file.
• Bonus: add configuration to the task to produce report AND throw exception if there are incompatibilities
• Bonus 2: add configuration to the task to specify what are public packages and what are “internal” packages. Incompatible changes to internal packages are OK and should not fail the task. In the compatibility report it is easy to discriminate changes to “internal” API VS changes to “public” API.
• Bonus 3: make bonus 2 feature configurable on a task (property on the task).

Future ideas

Below future ideas are wild brainstorming :)

• Create a service that reports API incompatibilities and attaches information to GitHub Pull Request (similar to how Travis CI reports build results, codecov reports coverage).
• Create a concept of “API snapshot” that is stored in a file and ships to binary repository along with the publications (jar, sources, javadoc). Tools can use the snapshot to compare APIs between arbitrary versions of a library.
• Create plugin for consumers so that they can check compatibility with libraries they consume. The plugin should aid consumer in bumping to new version of a desired library.

Applying one of our plugins, "org.mockito.mockito-release-tools.versioning" or "org.mockito.mockito-release-tools.continuous-delivery" plugin in a project that does not have 'version.properties' file yields nasty stack trace. We need to handle this use case gracefully. Example solutions:

• fail with decent exception telling the user what to do
• generate the file for the user and log information that the file should be checked-in
• create some sort of bootstrap task that will setup all necessary files for getting continuos delivery set up for a project

The continuous delivery plugin needs to configure the pom file with sensible defaults, like:

• SCM link
• name and description
• issues link
• web site link
• developers
• contributors

Suggested name for new plugin: "org.mockito.mockito-release-tools.pom"

Example pom.xml customization from Mockito project itself. The goal is to hide such complexity inside the plugin so that standard Java library gets this pom customization out of the box.

Problem

The message used when release automation commits code is hardcoded in the plugin implementation. In Mockito project, we have a use case where we want to run the build even for the automated version bumps. We want to do it to work around the issue with codecov (mockito/mockito#984).

Suggested implementation

Make the automated commit message configurable and settable via new "releasing.git.commitMessagePostfix" configuration setting. Full-blown example:

releasing {
  def buildNo = System.getenv("TRAVIS_BUILD_NUMBER")
  git.commitMessagePostfix = buildNo? " by CI build $buildNo [ci skip-release]" : " [ci skip-release]"
}

Test coverage

• let's use the new configuration setting in the release-tools-example project
• unit tests in the code

Problem

Originally copied from Mockito issue 700. Current release notes are not concise enough for rapid release model and hence the are hard to reason about. In short term we will make the release notes more concise. In the long term we need some kind of service that can list the changes between 2 arbitrary versions. In addition to compact format, we will also:

• Add links to commits for given version (originally reported at mockito/mockito#743): mockito/mockito@v2.7.3...v2.7.4
• Add link to bintray: https://bintray.com/mockito/maven/mockito/2.7.5
• Add link to central: http://search.maven.org/#artifactdetails%7Corg.mockito%7Cmockito-core%7C2.7.5%7Cjar
• Signify the repository the release went to (see #4)

Proposed design of release notes

Release notes are automatically generated by Mockito Continous Delivery Pipeline.

2.7.5 - 2 authors, 5 commits - 2017-02-07 15:36 - notable release in Central and JCenter+Bintray
🍸 Christian Schwarz: Support for return type 'Void' in DoesNothingAnswer (#933)

2.7.4 - Allon Murienik, 1 commit - 2017-02-06 15:04 - minor release in Bintray
🍸 Fix whitespace issues throughout the code (#928)

2.7.3 - Christian Schwarz, 1 commit -2017-02-06 12:51 - minor release in Bintray
🍸 Small improvements (#912)

2.7.2 - 2 authors, 2 commits - 2017-02-04 12:05 - minor release in Bintray
🍸 Szczepan Faber: Deleted dead code (#924)
🍸 Rafael Winterhalter: Added subproject for configuration-free inline mock making. (#920)

2.7.1 - no commits - 2017-01-31 16:48 - minor release in Bintray

2.7.0 - 2 authors, 10 commits - 2017-01-31 16:48 - minor release in Bintray
🍸 No pull requests referenced in commit messages.

2.6.5 - Szczepan Faber, 10 commits - 2017-01-31 16:48 - minor release in Bintray
🍸 No pull requests referenced in commit messages.

Discontinued tabular release notes idea

👎 We discontinued the tabular release notes idea because:

• not concise enough
• hard to read as plain text due to markdown tables
• hard to parse / manipulate markdown tables

Currently there are some moments at the configuration part of the build, when ReleaseConfiguration properties values are default or null, even though user overrides them in one of the .gradle files. It depends on when exactly this file is applied. It shouldn't be like that - we should be always sure that at each moment only the value provided by the user is used.

As discussed on Slack with @szczepiq and @mstachniuk it would be best to have a conventional file of the specified name and location (for example /gradle/shipkit.gradle) which would be loaded by mockito-release-tools automatically when ReleaseConfigurationPlugin is applied (but after application of default values to allow user to override them).

Additionally when user doesn't have the conventional file in their project, we should create this file, similarly to what we do with version.properties file if it doesn't exist.

We want to use JSON format instead of Java serialization one for storing release notes. It would be probably best to use json-simple for this, since it's already a dependency.

Problem

We should be able to to simplify release configuration and get rid of "releasing.build.branch" completely:

releasing {
  //we should be able to get rid of the below and configure automatically:
  build.branch = System.getenv("TRAVIS_BRANCH")
}

It is super annoying that I have to configure TRAVIS_BRANCH env variable even for local testing or releasing from local (I'm not Travis!). Fixing it will make the tools easier to use today, and easier to integrate with other tools in the future.

Workflow

1. travisReleasePrepare task ensures that TRAVIS_BRANCH is checked out.
2. in other workflows (non-travis), for example local testing, deployment from local, we don’t read nor require the TRAVIS_BRANCH property at all.

Suggested implementation

See the suggested starting point design at: #102

To keep the code clean and also to maximize the value of the code review, let's make the tools validate consistent code style for us.

• enable Gradle Checkstyle plugin and make the build fail on coding style issues
• enable Gradle Codenarc plugin, fail on issues
• clean up the codebase and submit PR
• ensure it is enabled for test code (see #22 (comment))

Goal

When release notes are cleaner and prettier they manifest more credibility and can attract more adopters of our tools. We need lots of users to succeed!

Implementation

Currently, the release notes have bare link to the publication repository. It would be great if it had a download badge just like we have it on Mockito main page.

Coding!

• The publication repository is currently rendered in release notes via DetailedFormatter class.
• The repository link markdown is constructed here.
• Example badges:

Testing

• e2e testing with example project - update previous version in the version.properties file (use some lower version) and run ./gradlew previewReleaseNotes

Full example

Current way of presenting release in notes:

2.8.31 (2017-05-22) - 1 commit by Rafael Winterhalter - published to maven/mockito-development

Desired way of presenting release in notes:

2.8.31 (2017-05-22) - 1 commit by Rafael Winterhalter -

Producer-client lifecycle

• When working on release tools I constantly build the sample project with new version of tools. It would be great to streamline this so that when we run "build" in the release tools project we have confidence that stuff works.
• I frequently need to bump versions of tools in the client projects (mockito, example project) by creating PRs and merging them. That's an overhead I'd like to eliminate. In general, I'd like the clients to be on recent version of our tools.

Above are the problems of the challenges of the producer-client lifecycle. Let's figure out a set of plugins an features that will streamline evolution and adoption of new versions.

Suggested implementation

Let's try to identify a scalable solution that can evolve into a general-purspose feature that allows convenient evolution of versions of OSS libraries. Below is the result of our internal discussions with @mstachniuk and @wwilk:

1. In producer (mockito-release-tools) we can configure a list of “clients” we care about most. We would start with 2: (mockito + mockito-release-tools-example).
2. For each client we provide a way to locally run e2e test with that client. For example: “./gradlew e2eTest” will run all clients, “./gradlew e2eTests:mockito:e2eTest” will run only e2eTest for Mockito client. e2e test is a) shallow clone b) bump version of producer to locally built c) performRelease -Preleasing.dryRun
3. Running “build” includes running all “e2eTest” tasks. So if one of the clients is broken the build fails and there is no release on CI.
4. When CI job is successful and we publish new version of producer we automatically create PR in GitHub for each client. The PR contains the version bump of the producer. For some clients we may choose not to publish PRs, those clients will be used only for e2e testing.
5. Bonus: we can add a functionality that will make the CI job of the producer automatically merge the PR created for the client. We would use it for some clients, like “example" projects. We don’t want to use it for Mockito, the Mockito team will decide whether they want to merge or not. So the “wait for PR and automatically merge” functionality needs to be configurable per client.

Suggested solution would scale because

• we acknowledge that there are different clients: those that we only use for testing, friendly projects that we submit PRs with version bumps, finally projects we own and want to force push the change
• we move the building of the clients (or at least some of that 😉) into their own CI pipeline
• we don’t bypass GitHub pull request model with force pushing changes without PR (Mockito team would probably not agree to force push version bumps)
• we don’t bypass client team’s validations when pushing code to their project. If we push version bump directly and it causes client project build failure who is going to fix it? :)
• we would wrap the feature in a sweet little plugin that is reusable all over the world :DDD
• we have easy way to build/reproduce locally

Starting point

To get started, let's get a single client local e2e test (example project) hooked up in the release-tools project. For version bumping, we can use Mockito's pattern of keeping the version of tools in gradle.properties file.

Future

In future this could potentially turn into more general capability we would add to release tools. It would be great if clients of release automation could automatically validate if their changes work with selected open source projects on GitHub. For example, I would love if every change of Mockito was tested with some other GitHub projects like guava. This potentially can be extremely useful feature.

fetchNotableReleaseNotes task fails with below stack trace when one of the input tags is missing. Let's have more graceful handling of this scenario, the exception message should indicate a potential problem and suggest a resolution.

Stack trace:

:fetchNotableReleaseNotes FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':fetchNotableReleaseNotes'.
> Problems parsing date: [fatal: ambiguous argument 'v2.7.0': unknown revision or path not in the working tree.
  Use '--' to separate paths from revisions, like this:
  'git <command> [<revision>...] -- [<file>...]']. Required format is: [yyyy-MM-dd HH:mm:ss Z].

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

When version declared in the version file does not end with blank line the version bumping fails.

Example usage of "incrementVersion" API in *.gradle file:

project.versionFile.incrementVersion()

Example version file:

version=1.5.0

See the original ticket in Mockito project.

Currently Travis CI is not enabled for mockito-release-tools repo. We don't have validation of master branch and pull requests. It would be great to configure Travis CI in similar way as core Mockito library is supported.

Release notes need to indicate the target repository where the artifacts were published. This way, we can support Mockito Continuous Delivery Pipeline 2.0 (mockito/mockito#911).

Example links:

• Bintray repository: https://bintray.com/mockito/maven/mockito/2.7.1
• Maven central: http://repo1.maven.org/maven2/org/mockito/mockito-core/2.7.1/
• Search in Central (there is a delay up to 48 hours): http://search.maven.org/#artifactdetails%7Corg.mockito%7Cmockito-core%7C2.7.1%7C
• Option: can we use visual badges?

Use case

Run build once, make some change, run build again, then compare build results. Sounds familiar? For certain kinds of changes, doing such A/B testing is super useful. It would be an excellent addition to release tools. After all, we want to really improve the lifecycle of software consumers and producers (easier, safer version bumps!). Example use cases:

• bumping version of a dependency and validating if the project still works ok (#85). Bumped version can be of some compile dependency but also of a built-time dependency such as the version of a some Gradle plugin we use.
• making change in build logic and validating if stuff still works.

Example workflow

Example implementation of the build comparison workflow:

1. Clone a project from GitHub
2. Copy the clone to another directory so that we have 2 identical clones
3. Run a set of commands in clone 1, for example "./gradlew build -PsomeToolVersion=1.4"
4. Run a set of commands in clone 2, for example "./gradlew build -PsomeToolVersion=1.5"
5. Compare build directories
6. Analyze comparison results

Suggested design

Suggested starting point design - check out #114 (It's only a suggestion! Pick your own path if you want, just keep the team informed about the design decisions! :)

The purpose of this ticket is to flesh out what else is needed to release 1.0.

Everything below is negotiable. Feel free to add things. Instead of removing, use strike through.

• Mockito build on latest Shipkit
  • - automated publication to Maven Central enabled
  • - notable/detailed release notes tidy up - #157
  • - contributors fetching from @mstachniuk feature in (some work on @szczepiq's end)
  • - publications prevention from @wwilk feature in
• API
  • - Javadoc on all public types
  • - All types intended to be public in 'public' package, internal types under 'internal' - #178
  • - Plugins rename
• Documentation
  • - see #432
• Testing / dogfooding / bootstrapping
  • - a new sample project that pushes to Maven Central (currently our sample project pushes to Bintray). The new sample project would be a "minimal" project, with least amount of configuration. It would also act as documentation.
  • - shipkit dogfoods itself (release notes, versioning)
• Misc
  • - bootstrap mechanism so that we can demonstrate how easy it is to setup CD. We could screencast it
  • - Avatar for shipkit-org user so that we look good commits view. Looking good is important because it adds credibility :)

Problem

Now we have 2 ways of getting contributors and it adds some complexity to the code. During the build we use both ways of getting contributors information, wrapped in separate Gradle tasks. Let's have one robust service for getting contributors and one Gradle task

Workflow

1. First, we hit repo/contributors endpoint and get all of the guys (https://developer.github.com/v3/repos/#list-contributors).

• Let’s use max page size :D

2. Since we may not get the most recent contributors due to GitHub performance limitations, let’s do another hit for commits (https://developer.github.com/v3/repos/commits/#list-commits-on-a-repository).

• Let’s limit the commits to last 24hrs, earlier commits should be covered by repo/contributors request

3. We combine the results

Implementation

1. Update GitHubAllContributorsFetcher. @mstachniuk, can you take a stab at it at some point? :)
2. Rework Gradle tasks, eliminate dead code - I'll do it with pleasure! (@szczepiq)

Let's start using an extension object to configure the plugins. This is more idiomatic approach used by Gradle plugins and we should adhere to standards as much as possible. Here's how the configuration would look in Mockito. Please give feedback:

//minimal
releasing {
    github {
        repository = "mockito/mockito"
        writeAuthUser = "szczepiq"
        readOnlyAuthToken = "a0a4c0f41c200f7c653323014d6a72a127764e17"
    }
}

//fully configured
releasing {
    github {
        repository = "mockito/mockito"
        writeAuthUser = "szczepiq"
        writeAuthToken = System.getenv("GH_WRITE_TOKEN")
        readOnlyAuthToken = "a0a4c0f41c200f7c653323014d6a72a127764e17"
    }
    releaseNotes {
        file = "doc/release-notes/official.md"
        notableFile = "doc/release-notes/notable.md"
        labelMapping = [
            '1.* incompatible': 'Incompatible changes with previous major version (v1.x)',
            'java-9': "Java 9 support",
            'java-8': "Java 8 support",
            'new feature': "New features",
            'BDD': 'Behavior-Driven Development support',
            'bug': "Bugfixes",
            'enhancement': "Enhancements",
            'android': "Android support",
            'docs': 'Documentation'
        ]
    }
    git {
        genericUser = "Mockito Release Tools"
        genericEmail = "<[email protected]>"
        releasableBranchRegex = "master|release/.+"  // matches 'master', 'release/2.x', 'release/3.x', etc.
        branch = System.getenv("TRAVIS_BRANCH")
    }
    team {
        developers = ['szczepiq:Szczepan Faber', 'bric3:Brice Dutheil', 'raphw:Rafael Winterhalter', 'TimvdLippe:Tim van der Lippe']
        contributors = []
    }
}

Notable release notes were not generated in Mockito release build: https://travis-ci.org/mockito/mockito/jobs/221223473#L793

GitHub has nice releases (https://github.com/mockito/mockito/releases), but we are not using them right now. It would be great if the changelog can be published there. The corresponding endpoint is https://developer.github.com/v3/repos/releases/#create-a-release

It would also be nice if "notable versions" would be released as a full release, while all other versions are released as pre-release (see the boolean parameter to the endpoint).

Imagine

Imagine the world where you call pull in a new version of some Open Source library and not worry if it breaks compatibility. Imagine that you can submit a pull request to some project, have it reviewed timely, and have the new version with your fix available to you in minutes after your PR is merged. Imagine that for any dependency you consider upgrading, you can view its neatly and consistently maintained release notes. Imagine that you can set up practical Continuous Delivery automation in your project in minutes, by using a well behaving and documented Gradle plugin. Imagine that you can focus on code and features while the release management, versioning, publishing, release notes generation is taken care for you automagically.

This is the goal of "mockito release tools project".

True North Star

The goal Mockito Release Tools project is to provide easy-to-setup Continuous Delivery tooling. We would like other teams to take advantage of rapid development, frictionless releases and semantic versioning just like we do it in Mockito. We plan to make our release tools generic and neatly documented. It will be a set of libraries and Gradle plugins that greatly simplify enabling Continuous Delivery for Java libraries.

We need help

We cannot achieve our goals without your help. Consider joining our efforts! Help is requested for:

• Implementing features - see the issue marked with "please contribute" label.
• Using the release tools in your project and giving us feedback.
• Spreading the word of what we're doing, letting us know about other project with similar goals. You can GitHub issue tracker for reaching out.

If the vision of the project connects with you help us!!!

Problem

We cannot use mockito project for testing the release tools. This is too risky and may cause undesired mockito versions to be published to the community.

Plan

Let's create a sample project in the "Mockito" organization in GitHub. This sample project will be using mockito-release tools. Goals:

• convenient testing of new features / changes to the release tools
• create another client for release tools. This will force further improvements in release tools (currently the release tools are coupled with Mockito use case)

Detailed action plan

• Create separate "example" project. Completed, see "mockito-release-tools-example".
• Prepare org.mockito.release-notes Gradle plugin and roll it out to the example project (#20)
• Example project publishes to Bintray
• Example project has automated publication with Travis CI
• End-to-end testing with mockito. Either create forked mockito project or a testable branch in main repo
• Example project publishes to Maven Central

Problem

Release tools project currently defines its version in root "build.gradle" file and it is problematic because:

• it is not bumped automatically with the release
• it triggers merge conflicts in "build.gradle" file instead of merge conflicts in a simpler 'version.properties' file.

Suggested implementation

1. Have the release tools project consume the 'org.mockito.release-tools.versioning' plugin from the previously released version of tools.
2. Update the travisRelease task so that when we release we bump version automatically.
3. Bonus - we should configure versioning plugin in 'pluginBundle' section of 'build.gradle' file. See the documentation of Gradle plugins portal: https://plugins.gradle.org/docs/publish-plugin.

Future

We should make the release tools plugin actually consume the entire continuous delivery plugin. We can get there step by step.

Current release notes list the authors which is great because it motivates to contribute! However, the author is listed by name, without the link to GitHub profile. The profile would be useful for more exposure / fame for the contributors.

Last time I worked on it, there was no way to link user to GitHub profile.

TODO:

• try to find GitHub user name given that we have Git author/author email. Engage community, ask on GitHub tracker, ask on twitter, etc.
• as suggested by @TimvdLippe we can use GitHub search API: https://api.github.com/search/[email protected]&in=email

🚀 Looking for help!!!

If you like to party, try to implement this interface and submit a pull request! :) To get started, run "GitHubTicketFetcherTest" test, it actually hits GitHub to get data.

/**
 * Maps Git authors to GitHub users 
 */
public interface GitHubUserMapper {

    /**
     * Provides mapping from Git author email to GitHub user name.
     * 
     * @param authorEmails email addresses of git authors
     */
    Map<String, String> mapToGitHubUser(Collection<String> authorEmails);
}

TBD.

Problem

Current release notes contain too much detail (full explanation: mockito/mockito#941). This is also important for Continuous Delivery Pipeline 2.0 (mockito/mockito#911) because the release notes need to discriminate between notable and minor releases.

Ideal solution would be a service that shows release notes between arbitrary versions and allows zooming in / out. Given that ideal solution does not exist now and it is costly to implement lets implement something simple.

Proposed solution

Let's add capability to generate a concise release notes report, only for noteworthy versions, including only noteworthy issues/pull requests. Example:

Mockito Continuous Delivery Pipeline automatically generates detailed release notes. This page contains notable releases and changes for users considering upgrading to newer version.

2.7.0 - 2017-01-29

Authors: 5 (link to detailed release notes), commits: 40 (link to commits), improvements: 12 (link to detailed release notes), notable:

• New strict mocking API - MockitoSession (#857), Szczepan Faber.
• Mockito 2.6.4 hangs on JDK 1.8.0_31 (#892), Rafael Winterhalter.
• Update to Byte Buddy 1.6.4: Fixes bridge method resolution for generic types (#891), Rafael Winterhalter.
• Java 9 support, upgrading to Objenesis 2.5 (#882), Allon Murienik.

2.6.1 - 2017-01-12

• Built-in support for Android, new 'mockito-android' artifact (#872), Tim van der Lippe
• Mockito JUnit Runner supports strict stubbing (#854), Szczepan Faber
• Fixes #820 ReturnsArgAt to handle returning vararg as arrays (#821), Brice Dutheil
• When custom exception fillInstackTrace() returns null, Exception mock cannot work properly (#866), Brice Dutheil
• Updated to Byte Buddy 1.6.0 (#864), Rafael Winterhalter

Let's improve testing of the code that interacts with GitHub so that we have decent coverage for this critical functionality.

• Currently we have 2 integration tests that hit GitHub API. Both tests are ignored (search in codebase for @ignore flag). We should make those tests use example repo instead of real mockito repo and enable the tests. Mockito repo is big and hence the tests are slow. Also, Mockito repo is a real project with real issues reported, it would be better to use mockito-release-tools-example project as target for remote requests to GitHub API.
• We can also create a test that mocks GitHub API, check-in stub JSON result and have neat unit tests for the code.

As discussed in the design of Continuous Delivery Pipeline 2.0, we would like to ship a new notable version when special keyword is used in the commit message: "[ci notable release]". The rationale is included in mockito/mockito#911.

Implementation hints

In the source code we already check for presence of "ci skip-release" keyword to skip the release of new version. Search for "ci skip-release" to see how it's done.