githubnemo / compiledaemon Goto Github PK
View Code? Open in Web Editor NEWVery simple compile daemon for Go
License: BSD 2-Clause "Simplified" License
Very simple compile daemon for Go
License: BSD 2-Clause "Simplified" License
CompileDaemon is great. Thank you.
It could use an additional feature to beep if a build breaks. I'm using it while building a web app. I usually have it running on some terminal in the background and I never get to know if there is an error. I reload the web pages but go through the user steps, discover that the code has had no effect, and only then maybe check if the build had failed or not. So much time lost.
It would really help to have a configurable beep to alert me when the build fails.
Hey! - I'm making a little simple API And I've been trying to get CompuleDaemon to see any file changes and rebuild with docker. Any changes I make doesn't get reflected on my endpoints. Could I have some help/pointers?
File Structure:
- api
-- go.mod
-- go.sum
-- main.go
-- controllers
--- users-controller.go
DockerFile:
# syntax=docker/dockerfile:1
FROM golang:latest
RUN apt-get install git
EXPOSE 8080
WORKDIR /go/src/app
COPY /api .
RUN go mod download -x
RUN ["go", "install", "github.com/githubnemo/CompileDaemon@latest"]
ENTRYPOINT CompileDaemon -build="go build main.go" -command="./main -my-options"
Since the logger buffer isn't flushed until another newline is found it's possible that the output that CompileDaemon generates can be affected by ANSII escape sequences that haven't been terminated due to no newline being printed.
For example, a program that writes output with color, writes a newline, then immediately writes the color reset escape sequence. Because the output is buffered the reset escape sequence won't be written until a newline is outputted. However, if the program is terminated and restarted it will never write the reset escape sequence and the output from CompileDaemon (eg "Running build command!") will be affected by the previous escape sequence.
Obviously this could be fixed by outputting the reset sequence before writing a newline, but it seems like the 'correct' solution would be for this software to flush the buffer when a program is recompiled.
How can I update vendor before refresh build? (Run cmd go mod vendor
each refresh build)
Hello,
Please help, I don't know where is the issue but after changing file CompileDaemon does not re-compile go application.
Currently, I'm using Windows, I don't want to use docker desktop for Windows so I install docker on Vagrant.
Here is my Dockerfile-local
FROM golang:alpine
# Set necessary environmet variables needed for our image
ENV GO111MODULE=on \
CGO_ENABLED=0 \
GOOS=linux \
GOARCH=amd64
WORKDIR /build
COPY go.mod .
COPY go.sum .
RUN go mod download
COPY . .
RUN apk add git
RUN go get github.com/githubnemo/CompileDaemon
ENTRYPOINT CompileDaemon --build="go build -o bin cmd/apiserver/server.go" --command="./bin/server"
And this is docker-compose file
version: '3'
services:
api-server:
build:
context: ./
dockerfile: Dockerfile-${ENV}
image: api_server:tag
container_name: api-server
hostname: api-server
restart: always
volumes:
- ./:/build
ports:
- 8080:8080
I missed something, didn't I? Or It's a Vagrant problem (Vagrant does not forward file changing event to docker).
Thank you very much for any help and sorry for my English.
In the command and I think the build although haven't tested
If you have a space in the option it will fail to run.
CompileDaemon --build="go build -o program main.go " --command="./program --option=\"1 2\""
Seems to split on spaces which is not always correct.
I'm trying to use CompileDaemon with go generate
, but have not been successful. I am running two instances of CompileDaemon to try accomplishing this. One instances watches the source files, which are .sql, and runs go generate
when they change. The other watches all .go source files and runs go build
. I believe my flags are set up so that the two are watching mutually exclusive files:
#!/bin/sh
CompileDaemon -build="go generate ./..."
-exclude="*.go"
-include="*.sql"
-pattern=".+\.sql$" &
CompileDaemon -build="go build -o /opt/bin/service"
-exclude="*.sql"
-include="*.go"
-pattern=".+\.go$" &
However, when I run this script, CompileDaemon gets stuck in an infinite loop. Is CompileDaemon incompatible with go generate
?
I've managed to start up the CompileDaemon with minimal effort. It seems to work great!
If I make a change in Atom and then hit save, the CompileDaemon does it thing and rebuild/restart. The problem is, Atom immediately freezes as soon as this happens. Is this something you've observed?
I get the error "CompileDaemon: not found" and I don't understand why I'm getting such error
Sometimes a running program fails because of the panic. Because CompileDaemon keeps running it is hard to notice that (except for the panic stack trace). I think it would be best if CompileDaemon would just restart the program, with some message (like, "program terminated, restarting"). Probably as opt-in CLI flag.
Error comes up on newer versions of go like; 1.7<
The below dependency file contains math/bits which is causing related issue;
https://cs.opensource.google/go/x/sys/+/master:unix/affinity_linux.go;l=10?q=math%2Fbits&sq=&ss=go%2Fx%2Fsys
github.com/githubnemo/CompileDaemon (download)
github.com/fatih/color (download)
github.com/mattn/go-colorable (download)
github.com/mattn/go-isatty (download)
Fetching https://golang.org/x/sys/unix?go-get=1
Parsing meta tags from https://golang.org/x/sys/unix?go-get=1 (status code 200)
get "golang.org/x/sys/unix": found meta tag main.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
Fetching https://golang.org/x/sys?go-get=1
Parsing meta tags from https://golang.org/x/sys?go-get=1 (status code 200)
golang.org/x/sys (download)
package math/bits: unrecognized import path "math/bits" (import path does not begin with hostname)
github.com/fsnotify/fsnotify (download)
github.com/radovskyb/watcher (download)
Hi, this is a cool project.
I have written a simple web server, and Im using this to monitor for code changes. Im using it like so:
./CompileDaemon -directory=/linked -command=./linked
It starts up fine, however when I change the code, it doesn't re build. I suspect its because /linked/linked
is a web server and never stops, but when I read your documentation, it mentions that CompileDaemon will kill the app and restart it. Any ideas?
When I run CompileDaemon -exclude-dir=./container_data -exclude-dir=./.git -exclude-dir=./docker -exclude-dir=./migrations
, my go app (local webserver) doesn't seem to work.
When I run go run .
I get a macos firewall allow dialog. Once I click "allow", it works.
How can I get CompileDaemon to display the firewall dialog?
Currently the program is left running, so if the build fails, you have an old version of the program running. It happened to me multiple times that I tried to use the HTTP app and it looked like working, but then there was no expected change in its behavior. Only then I realized that the new code didn't compile. It would be better if the program would be terminated always (after the build, if it is successful a new version is stared, otherwise nothing is started) so that I would see errors when browser tries to connect to a new version.
At least provide a flag to enable this behavior.
CompileDaemon seems to strip any color coming from the running binary that its watching.
README.md is missing install part for quick start:
go get github.com/githubnemo/CompileDaemon
Even, if it's pretty much obvious. For go-lang newcomers, like me, it takes a bit of time to figure this out.
Not really an issue, but I'm curious - what is this licensed under?
Thanks.
It would be great if multiple build commands could be run when file changes are detected:
1. docker build -t
2. docker-compose down
3. docker-compose up
The 1.1.0 version is tagged as v.1.1.0
, it should be fixed as v1.1.0
.
Hi, somehow I got this error when try to get this package:
# golang.org/x/sys/unix
../../../golang.org/x/sys/unix/syscall_darwin.go:81:9: undefined: readInt
../../../golang.org/x/sys/unix/syscall_darwin.go:85:9: undefined: readInt
../../../golang.org/x/sys/unix/syscall_darwin.go:89:9: undefined: readInt
../../../golang.org/x/sys/unix/syscall_darwin.go:159:14: undefined: sysctlmib
../../../golang.org/x/sys/unix/syscall_unix.go:232:9: undefined: anyToSockaddr
../../../golang.org/x/sys/unix/syscall_unix.go:311:15: undefined: anyToSockaddr
../../../golang.org/x/sys/unix/syscall_unix.go:421:10: undefined: UtimesNanoAt
../../../golang.org/x/sys/unix/syscall_unix.go:427:3: undefined: NsecToTimespec
../../../golang.org/x/sys/unix/syscall_unix.go:427:18: undefined: TimevalToNsec
../../../golang.org/x/sys/unix/syscall_unix.go:428:3: undefined: NsecToTimespec
../../../golang.org/x/sys/unix/syscall_unix.go:428:3: too many errors
I'm setting up a project in Go following this guide https://levelup.gitconnected.com/how-to-live-reload-code-for-golang-and-docker-without-third-parties-ee90721ef641 , on Linux it works fine. Using it from mac when I change a file from my editor, the file into the container change but CompileDaemon doesn't recognise the change and doesn't rebuild.
If I enter the container and save the changed file, CompileDaemon rebuild.
DOCKER FILE
FROM golang:latest
WORKDIR /go/src
DockerCompose
version: '3'
services:
app:
build:
context: .
volumes:
- ".:/go/"
container_name: golang_app
command: bash -c "go get github.com/githubnemo/CompileDaemon && CompileDaemon -command="./app""
ports:
- "8080:8080"
tty: true
depends_on:
- db
VERSIONs
OSX Catalina 10.15.2
Docker version 19.03.5 ( Using Docker Desktop application)
As of two hours ago there's now an empty fsnotify repo on github that compile daemon resolves to as its dependency.
http://gopkg.in/fsnotify.v1 -> https://github.com/go-fsnotify/fsnotify
This smells of a package injection attack but it seems to not have materialized.
At the very least, this has broken CompileDaemon because it resolves to the wrong dependency.
The maintainers of gopkg.in will probably figure it out, but we thought we'd let you know :)
I was attempting to make this fit an existing project that currently used reflex to watch for changes and Makefile targets after using this on some personal projects and loving the simplicity, and I realized that I wanted to perform different actions if different sets of files changed.
Example:
*.tmpl - go generate path/views/templates
which executes go-bindata
for those templates
*.scss - go generate path/views/styles
which executes sass->css compilation, then go-bindata
for the CSS
Is it a goal of the project to support this type of conditional targeting? Reading the README, I'd lean towards no, but wanted to check.
I'm using CompileDeamon in my Dockerfile.
I was using this command to only rebuild my app after changes:
CMD swag init -d src && CompileDaemon --build="go build src/main.go" --command="./main" --color
but I wanted to add also hot reloading with swagger recompile like:
CMD swag init -d src && CompileDaemon --build="swag init -d src && go build src/main.go" --command="./main" --color
and it not working correctly. When I run my docker-compose
then this container starts rebuilding every second without any change in files. I discovered that the problem is with an additional flag added to swag init
.
When I remove -d src
then CompileDeamon is not rebuilding the app without a change in code (correct behavior), so I think it's something wrong with interpreting command arguments.
It's problematic for me because of the project structure and I have to specify a directory for swag to compile it correctly :)
PS. Thanks for your work, CompileDeamon is great :D
Good day,
I followed the Tut at https://levelup.gitconnected.com/docker-for-go-development-a27141f36ba9 great work... Thanks.
I kept getting this build error:
step 6/7
...
go build golang.org/x/sys/internal/unsafeheader: module requires Go 1.12
ERROR: Service 'go-docker-image' failed to build: The command '/bin/sh -c go get
github.com/githubnemo/CompileDaemon' returned a non-zero code: 1
I changed the Dockerfile from golang:latest to golang:1.2 and it did the trick.
(maybe you could correct this in your repo for others)
Any idea on Golang 1.14 support?
(Seems to be the issue for me anyways)
Thanks again in advance.
JP
u@u-pc:~/Documents/Proyectos/GoApps$ go get github.com/githubnemo/CompileDaemon
fatal: unable to open .git/objects/pack/tmp_pack_3iLUeQ: No such file or directory
fatal: index-pack failed
package golang.org/x/sys/unix: exit status 1
When there are build errors, it only shows an overall status that there was an error. To actually find out what the errors are, I'm having to run the go build separately.
Hi!
I think this tool is very useful during development and testing, but there might be severe consequences if used in production.
In fact, if attackers would be able to upload arbitrary files, this would trigger re-building and running the malicious code just uploaded, leading to RCE (Remote Code Execution).
In practice, this would bring the same vulnerabilities affecting interpreted languages in go.
I suggest to place a banner in the README.md file, warning developers to be careful and don't use this tool in production environment.
I wanted to run two command , But i gives error.
CompileDaemon -command="fuser 50052/tcp && go run main.go --server_address :50052"
If you have port closing option kindly please help
First: Thank you.
After reading documentation and options, I can't run
and test
at the same time.
I think that could be possible with this package since most of the time we (devs) need it.
Currently:
#!/bin/sh
CompileDaemon \
-directory=. \
-recursive=true \
-build="go build -o server" \
-command="./server -port=4000 -host=localhost" \
-color=true
What I'm looking:
#!/bin/sh
CompileDaemon \
-directory=. \
-recursive=true \
-build="go build -o server" \
-command="./server -port=4000 -host=localhost && go test ./... " \
-color=true
Sorry If I missed something from docs.
Thank you for your time.
I successfully ran CompileDaemon a month ago when I downloaded it, but now whenever I try to run it I get:
filepath.Walk():bad file descriptor
I've tried deleting the src and bin files for CompileDaemon and running go get "github.com/githubnemo/CompileDaemon"
, but it didn't change anything.
I'm on a 2017 Macbook Pro.
I get this every time I try to run compile daemon.
Any advice?
Thanks.
Hi,
while using CompileDaemon on Windows 10 Pro I encountered a strange problem. When I run it as follows
CompileDaemon.exe -command="hello.exe"
I get the following output
2017/07/21 06:24:36 Running build command!
2017/07/21 06:24:37 Build ok.
2017/07/21 06:24:37 Restarting the given command.
2017/07/21 06:24:37 stdout: RUNNING6
2017/07/21 06:24:44 Running build command!
2017/07/21 06:24:45 Build ok.
2017/07/21 06:24:45 Hard stopping the current process..
2017/07/21 06:24:45 Could not kill child process. Aborting due to danger of infinite forks.
For debugging purposes I compiled CompileDaemon myself and added a print statement under the kill statement of the process. There I get the following error message :
TerminateProcess: Access denied
Obviously it isn't possible to kill a process on Windows which was created by exec.Command
. Do you have an idea what can be done to fix this problem?
Thanks,
Thomas
Hi there,
For ages I was trying to work out why the -polling
option was causing this to crash out in docker.
I was using this docker file:
FROM golang:latest
EXPOSE 80
WORKDIR /usr/src/app/go-api
COPY . /usr/src/app/go-api
RUN go mod download
RUN go get github.com/githubnemo/CompileDaemon
RUN GO111MODULE=on
# RUN CompileDaemon
ENTRYPOINT CompileDaemon -build="go build main.go" -directory="." -polling -command=./main
To which it ouputted the full list of options - not including -polling
" flag provided but not defined: -polling"
To fix this, I added @master
to the go get command :
FROM golang:latest
EXPOSE 80
WORKDIR /usr/src/app/go-api
COPY . /usr/src/app/go-api
RUN go mod download
RUN go get github.com/githubnemo/CompileDaemon@master
RUN GO111MODULE=on
# RUN CompileDaemon
ENTRYPOINT CompileDaemon -build="go build main.go" -directory="." -polling -command=./main
This now works!
This is an amazing project and I am very grateful for it - I just thought that this may come up for someone else!
I'm not sure why go get
is behaving like this, but I assume it is a bug.
Full code example:
main.go
package main
import (
"net/http"
"github.com/labstack/echo/v4"
)
func main() {
e := echo.New()
e.GET("/", func(c echo.Context) error {
return c.String(http.StatusOK, "Hello, dude!")
})
e.GET("/2", func(c echo.Context) error {
return c.String(http.StatusOK, "Hello, my friend, again!")
})
e.Logger.Fatal(e.Start(":80"))
}
Dockerfile
FROM golang:latest
EXPOSE 80
WORKDIR /usr/src/app/go-api
COPY . /usr/src/app/go-api
RUN go mod download
RUN go get github.com/githubnemo/CompileDaemon
RUN GO111MODULE=on # not sure if needed
ENTRYPOINT CompileDaemon -build="go build main.go" -directory="." -polling -command=./main
docker-compose.yml
services:
go-api:
build:
context: .
ports:
- "8081:80"
container_name: go-api
tty: true
volumes:
- ./:/usr/src/app/go-api
It's really annoying to search the console for the current output after multiple rebuilds, the console simply becomes a mess. It would be great to clear the console before approaching a new build, neither -build="reset && gb build all"
nor multiple -build="reset" -build="gb build all"
work though.
Any idea how to fix this?
This is great to just have go run .
re-run on a file change. It's perfect with the combination of a templating engine which rebuilds go files once the html files are updated.
However, CompileDaemon always compiles the binary when many times I only want to have go run executed with -command.
Is there any chance of providing an option to not build?
It would be great to have a newer tag than 1.1.0 in the repo, so that go get
does also download the newer version.
This would enable people to use -run-dir=/foo/bar/baz
It seems that it stopped working since the update of Go 1.18, the error is /bin/sh: CompileDaemon: not found
It works when I get Go image 1.17.
Here is my Dockerfile:
FROM golang:alpine
WORKDIR /app
COPY ./ /app
RUN go mod download
RUN go get github.com/githubnemo/CompileDaemon
ENTRY
POINT CompileDaemon --build="go build -o /app/.bin/api-admin ./cmd/admin" --command=./.bin/api-admin
Thanks
The project I'm working on has important files two folders back from where the main.go file is, so it would be nice to have something like -include-dir="../../core"
to be able to monitor any changes on these files as well, since we work with them a lot.
Hi,
Cannot run CompileDeamon as Entrypoint of dockerfile with golang 1.18
Working on golang:1.17.8-alpine3.15 with same ENTRYPOINT
tested with go get github.com/githubnemo/CompileDaemon@latest
I'm obviously missing something, but I can't seem to catch the SIGTERM signal and gracefully terminate the running server.
package main
import (
"context"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"sync"
"syscall"
"github.com/gorilla/mux"
)
func main() {
log.Println("Starting HTTP server on port", os.Getenv("PORT"))
httpServerExitDone := &sync.WaitGroup{}
httpServerExitDone.Add(1)
srv := startHTTPServer(httpServerExitDone)
c := make(chan os.Signal, 1)
signal.Notify(c,
syscall.SIGHUP,
syscall.SIGINT,
syscall.SIGTERM,
syscall.SIGQUIT)
go func() {
<-c
fmt.Println("\r- SIG sent to terminate")
if err := srv.Shutdown(context.TODO()); err != nil {
panic(err)
}
}()
httpServerExitDone.Wait()
log.Printf("main: done. exiting")
}
func startHTTPServer(wg *sync.WaitGroup) *http.Server {
r := mux.NewRouter()
srv := &http.Server{Addr: ":" + os.Getenv("PORT"), Handler: r}
r.Handle("/", http.FileServer(http.Dir("./public")))
go func() {
defer wg.Done()
if err := srv.ListenAndServe(); err != http.ErrServerClosed {
log.Fatalf("ListenAndServe(): %v", err)
}
}()
return srv
}
Results in:
2021/06/30 23:04:08 Gracefully stopping the current process..
2021/06/30 23:04:08 Restarting the given command.
2021/06/30 23:04:08 stderr: 2021/06/30 23:04:08 Starting HTTP server on port 3000
2021/06/30 23:04:08 stderr: 2021/06/30 23:04:08 ListenAndServe(): listen tcp :3000: bind: address already in use
2021/06/30 23:04:08 stderr: exit status 1
Go dropped go get for cmd-tools.
executing go install github.com/githubnemo/COmpileDaemon/cmd does not work.
Latest released tag v1.4.0 with hash 50a8debecc13686ba29356b28d30c33555e662f6
is not merged in master branch.
The following code displays "test" on the terminal when running my go module without CompileDaemon:
fmt.Fprintf(os.Stdout, "test")
When using CompileDaemon using the following command, there is not output on the terminal:
${my_go_path}/CompileDaemon -directory={directory_to_track_changes_on} -command="go run {path_to_main_file}" -color=true
Additionally, part of the code that runs starts an echo server on a local port. When I make changes, although live reloading happens, the server isn't restarted.
I am using VSCode and Docker. I want to live reload a go api and use the delve debugger in the same container instance.
Here's what I have so far: https://github.com/ivorscott/go-delve-reload
In the docker-compose.yml file you will see two services api and debug api. One uses CompileDaemon, the other uses Delve. Both containers are based off the same "dev" build stage in a multi-stage scenario.
Whenever I tried combining both approaches within one container I had issues. Ideally I want to have both features in one container.
If I kill CompileDaemon
, it does not kill the child process started with -command=โฆ
I have a local docker-compose setup with 10+ services, where each of them is built and run by CompileDaemon. Each service's local go folder is mounted as a volume to allow the developers at my company to make changes and CompileDaemon takes care of restarting the go server when any source file is changed.
All the services should restart after a file has changed
After several restarts with 10+ services, restarts with CompileDaemon stops working without any errors and I have to manually restart the container or restart the docker daemon for the notify events to work again.
MacOS Catalina 10.15.2 (19C57)
Docker Desktop 2.2.0.3 (42716)
Engine Version 19.03.5
Compose Version 1.25.4
ENTRYPOINT CompileDaemon \
--build="go build -o my_service" \
-exclude-dir=.git -exclude=".#*" \
--command="./my_service"
my_service:
build:
context: .
dockerfile: ./my_service.Dockerfile
volumes:
- $GOPATH/src/bitbucket.org/xxx/my_service:/go/src/bitbucket.org/xxx/my_service
If, after a thorough research we find that the problem is deeper then a few config files, maybe we can write a polling mechanism and watch for changes ourselves and ditch the inotify features. It is not the best solution but it will be a nice workaround if people just want things to work.
When run with -build=''
, the following error occurs:
2021/01/10 16:51:52 Running build command!
2021/01/10 16:51:52 Error while building:
The following code in the build()
function seems to be for conditionally skipping the build command:
Lines 151 to 158 in 39b144a
However, the return value of strings.Split
will never have length 0 because the provided separator is not empty. If provided an empty string, it will still return a slice containing a single empty string:
If s does not contain sep and sep is not empty, Split returns a slice of length 1 whose only element is s.
If sep is empty, Split splits after each UTF-8 sequence. If both s and sep are empty, Split returns an empty slice.
Maybe the condition could be replaced with if *flagBuild == ""
and checked before the call to strings.Split
:
if *flagBuild == "" {
// If the value of flagBuild is empty then we are done.
return true
}
args := strings.Split(*flagBuild, " ")
I have something like:
CompileDaemon -build="go build -o bin/darwin-amd64/x x/x.go" -command="./bin/darwin-amd64/x" -pattern=".+\.go$"
If I do a simple touch x.go
the binary gets compiled twice...
EDIT: Actually, starting it simply like this does the same problem:
CompileDaemon --exclude-dir=vendor
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.