chris-crone / containerized-go-dev Goto Github PK

Containerize your Go development environment!

License: The Unlicense

Dockerfile 54.91% Makefile 17.20% Go 27.89%

containerized-go-dev's Introduction

Example Containerized Go Development Environment

This repository contains an example Go project with a containerized development environment. The example project is a simple CLI tool that echos back its inputs.

Why should I containerize my development environment?

There are several advantages to containerizing your development environment:

You make explicit the tools and versions of tools required to develop your project
Your builds will be more deterministic and reproducible

These will both make it easier for people to collaborate on your project, as everyone will have the same environment, and make it easier to debug things like CI failures.

Prerequisites

The only requirements to build and use this project are Docker and make. The latter can easily be substituted with your scripting tool of choice.

You will also need to enable the BuildKit builder in the Docker CLI. This can be done by setting DOCKER_BUILDKIT=1 in your environment.

macOS

Install Docker Desktop
Ensure that you have make (included with Xcode)
Run export DOCKER_BUILDKIT=1 in your terminal or add to your shell initialization scripts

Windows

Install Docker Desktop
Ensure that you have make
If using PowerShell, run $env:DOCKER_BUILDKIT=1
If using command prompt, run set DOCKER_BUILDKIT=1

Linux

Install Docker
Ensure that you have make
Run export DOCKER_BUILDKIT=1 in your terminal or add to your shell initialization scripts

Getting started

Building the project will output a static binary in the bin/ folder. The default platform is for macOS but this can be changed using the PLATFORM variable:

$ make                        # build for your host OS
$ make PLATFORM=darwin/amd64  # build for macOS
$ make PLATFORM=windows/amd64 # build for Windows x86_64
$ make PLATFORM=linux/amd64   # build for Linux x86_64
$ make PLATFORM=linux/arm     # build for Linux ARM

You can then run the binary, which is a simple echo binary, as follows:

$ ./bin/example hello world!
hello world!

To run the unit tests run:

$ make unit-test

To run the linter:

$ make lint

There's then a helpful test alias for running both the linter and the unit tests:

$ make test

Structure of project

Dockerfile

The Dockerfile codifies all the tools needed for the project and the commands that need to be run for building and testing it.

Makefile

The Makefile is purely used to script the required docker build commands as these can get quite long. You can replace this file with a scripting language of your choice.

CI

The CI is configured in the ci.yaml file. By containerizing the toolchain, the CI relies on the toolchain we defined in the Dockerfile and doesn't require any custom setup.

containerized-go-dev's People

Contributors

Stargazers

Watchers

Forkers

arossmann ulrich matheuseabra rizalgowandy ksana-tools poidag-zz kokizzu robitx hairandbeardguy lbrun25 p4ch0u obazoud piotrek-k pcandrews jsawo rubrafmex thecarp uzbekdev1 cobra16319 cristianf jayjamieson ganimp84 peter-ro isomorpheric alexanelli achimgrolimund werayootk maour firas-hojirat atin777 sivaramsk neo-cypher pmoracho kmoltke ninoxdz breee jvasoya mahadihassanrazib supesharisuto wniels holdenmorris jasonc761

containerized-go-dev's Issues

Issue with exporting go test coverage output

first of all thank you for sharing this system,
I tried this system but unable to use for unit-test target with exporting test coverage output file for use with CI system.
I did mount the folder --output

go test -coverprofile=/out/cover.out ./...

any idea what could be missing here ?

Note to Chris - Thank you - Inspired me to build another containerized dev repo

Hi Chris @chris-crone

thanks a lot for your open project and the content in your blogs.

Maybe you also like to check my repo at https://github.com/TomFreudenberg/golang-cli-cmd-dev-starter

It may give some additional points.

Cheers
Tom

Fails for version 1.16

If I bumb the FROM to version 1.16, the docker build fails with the following message:

 > [build 4/4] RUN go build -o /out/example .:
#8 0.337 go: cannot find main module

Can be fixed by changing the RUN in the dockerfile:

RUN go env -w GO111MODULE=auto && go build -o /out/example .

Container image misses `go` and `make` commands

Hi,

First of all, thank you for providing this repo as an example of a containerized Go dev environment!

When launched this repo as a Dev Environment in Docker Desktop 3.5.2 (66501) and opened in VSCode, both go and make are missing from the container image:

vscode ➜ /com.docker.devenvironments.code (main) $ go version
bash: go: command not found
vscode ➜ /com.docker.devenvironments.code (main) $ make
bash: make: command not found

Could it be because the main language of this repository is Dockerfile instead of Go?

Is there any way to cache or cherry-pick which files are copied to the client?

All of my images require vips so I can't use a scratch (I don't think..). As such, I'm running with Alpine:

Sorry for the mess of docker files. I need to switch back to using a build-base and base images.

# syntax = docker/dockerfile:1-experimental
FROM --platform=${BUILDPLATFORM} golang:1.16.3-alpine3.13 AS build
RUN apk update
RUN apk upgrade
RUN apk add gcc g++
RUN apk add --update --no-cache --repository https://dl-3.alpinelinux.org/alpine/edge/community vips-dev
RUN apk --no-cache --update add ca-certificates --repository https://dl-3.alpinelinux.org/alpine/edge/main vips
WORKDIR /src
COPY go.mod .
COPY go.sum .
RUN --mount=type=cache,target=/go/pkg/mod \
  go mod download

RUN --mount=target=. \
  --mount=type=cache,target=/root/.cache/go-build \
  GOOS=linux CGO_ENABLED=1 \
  go build -a -installsuffix cgo -o /out/api-server ./server/cmd/api/.

FROM --platform=${BUILDPLATFORM} alpine:latest AS application
RUN apk update
RUN apk add --update --no-cache --repository https://dl-3.alpinelinux.org/alpine/edge/community  vips-dev
RUN apk --no-cache --update add ca-certificates --repository https://dl-3.alpinelinux.org/alpine/edge/main vips
WORKDIR /app
COPY --from=build /out/api-server .
ENTRYPOINT [ "./api-server" ]

Which definitely works, (thank you! the build times can definitely creep up on Go without caching in conjunction with cgo).

The only issue I'm running into is that because I need alpine, the payload for copying back to the client is 227.21MB which happens in full, every time.

▶ docker build -f server/cmd/api/Dockerfile . --output build/api --target application --platform local
[+] Building 21.9s (25/25) FINISHED                                                                                                      
 => [internal] load build definition from Dockerfile                                                                                0.0s
 => => transferring dockerfile: 37B                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                   0.0s
 => => transferring context: 2B                                                                                                     0.0s
 => resolve image config for docker.io/docker/dockerfile:1-experimental                                                             0.4s
 => CACHED docker-image://docker.io/docker/dockerfile:1-experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa46  0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                                                    0.0s
 => [internal] load metadata for docker.io/library/golang:1.16.3-alpine3.13                                                         0.0s
 => [build  1/11] FROM docker.io/library/golang:1.16.3-alpine3.13                                                                   0.0s
 => CACHED [build  2/11] RUN apk update                                                                                             0.0s
 => CACHED [build  3/11] RUN apk upgrade                                                                                            0.0s
 => CACHED [build  4/11] RUN apk add gcc g++                                                                                        0.0s
 => CACHED [build  5/11] RUN apk add --update --no-cache --repository https://dl-3.alpinelinux.org/alpine/edge/community vips-dev   0.0s
 => CACHED [build  6/11] RUN apk --no-cache --update add ca-certificates --repository https://dl-3.alpinelinux.org/alpine/edge/mai  0.0s
 => [application 1/6] FROM docker.io/library/alpine:latest                                                                          0.0s
 => [internal] load build context                                                                                                   2.3s
 => => transferring context: 2.25MB                                                                                                 2.2s
 => CACHED [application 2/6] RUN apk update                                                                                         0.0s
 => CACHED [application 3/6] RUN apk add --update --no-cache --repository https://dl-3.alpinelinux.org/alpine/edge/community  vips  0.0s
 => CACHED [application 4/6] RUN apk --no-cache --update add ca-certificates --repository https://dl-3.alpinelinux.org/alpine/edge  0.0s
 => CACHED [application 5/6] WORKDIR /app                                                                                           0.0s
 => CACHED [build  7/11] WORKDIR /src                                                                                               0.0s
 => CACHED [build  8/11] COPY go.mod .                                                                                              0.0s
 => CACHED [build  9/11] COPY go.sum .                                                                                              0.0s
 => CACHED [build 10/11] RUN go mod download                                                                                        0.0s
 => CACHED [build 11/11] RUN --mount=target=.   --mount=type=cache,target=/root/.cache/go-build   GOOS=linux CGO_ENABLED=1   go bu  0.0s
 => CACHED [application 6/6] COPY --from=build /out/api-server .                                                                    0.0s
 => exporting to client                                                                                                            18.7s
 => => copying files 227.21MB                                                                                                      18.6s

I'm assuming that I need the files copied back for caching and build purposes, which is fine. However, is there a way I can cherry pick what comes back? Is that feasible?

Thanks again,
Chance

the Dockerfile build binary before tests and lint

Maybe I've been doing it wrong all my life, but what's the point of building first, before testing or linting ?
I usually see linting first, then tests, and if everything is fine (which should produce a working binary), build.

Sorry to ask than in an issue... :)
BTW, great work explaining all these features

Error in unit test: read-only file system

My application creates files and in my test I check whether the creation was successful or not. with go test everything is fine, but with the containerised build I get the error of
read-only file system
I tried it with setting the mount to "bind"
FROM base AS unit-test RUN --mount=target=. \ --mount=type=bind,target=/root/.cache/go-build \ go test -v .
but without success.

Any hints for me?