ninenines / erlang.mk Goto Github PK

Please see discussion here:
talentdeficit/jsxn#3

Example:

clean::
    rm -rf priv/lib

And in erlang.mk it would be

clean::
    ... usual stuff ...

The motivation is – you can extend the target inside your own makefile. make clean is the main one that comes to my mind – you might want to clean some stuff explicitly – but there might be more examples. With a double colon, both targets will execute.

There is no option to make it analyze beam files instead of src folder.

Ref:

@dialyzer --no_native --src -r src $(DIALYZER_OPTS)

I suggest putting --src -r src as part of default $(DIALYZER_OPTS) so that it can be overridden.

If you have a better recommendation, please let me know. I'll create a pull request with this once you confirm.

Hi there,

I don't know if this is a real issue or just that I am doing something wrong. erlang.mk is trying to compile custom-behaviour dependant files before compiling the behaviour itself, thus causing an compilation error:

src/tetrerl_single.erl:8: behaviour tetrerl_game undefined

And this is the compilation order:

ERLC tetrerl.erl tetrerl_single.erl tetrerl_player.erl tetrerl_protocol.erl tetrerl_multi.erl tetrerl_controller.erl tetrerl_session.erl tetrerl_ping.erl tetrerl_game.erl tetrerl_app.erl tetrerl_lobby.erl tetrerl_scoreboard.erl

As you can see, tetrerl_single comes before tetrel_game where the behaviour is defined.

It would be nice if we could tell erlang.mk the order of compilation somehow, to compile first behaviours and then the rest of files. Any thoughts on this? Thanks.

or else (example) the deps target may fail if a dependecy is running dialyzer and doesn't override DIALYZER_PLT.

Suggestion: set the env var specifically for the built-plt and dialyzertargets.

this breaks any project that declares required compiler options in rebar.config

Loic,

Sorry to disturb you again. We use many libraries in our projects that generate warnings when compiled. We like that erlang.mk checks our code for warnings, so we do not want to remove the warning as error.

It would be great if there was a flag/environment variable to configure that erlang.mk does not consider warnings as error with dependencies. What do you think? Have you found this problem?

I am testing erlang on Freebsd but I am getting similar error to #103.

My setup is Freebsd 10, gmake 4.1 and erlang.mk 1.1

After added DEPS = cowboy I got this error next make

DEPPKG=$(awk 'BEGIN { FS = "\t" }; $1 == "cowlib" { print $2 " " $3 " " $4 }' /usr/home/git/test/.erlang.mk.packages.v2;) VS=$(echo $DEPPKG | cut -d " " -f1); REPO=$(echo $DEPPKG | cut -d " " -f2); COMMIT=$(echo $DEPPKG | cut -d " " -f3); if [ "$VS" = "git" ]; then git clone -n -- $REPO /usr/home/git/test/deps/cowlib; cd /usr/home/git/test/deps/cowlib && git checkout -q $COMMIT; else exit 78; fi
erlang.mk:140: recipe for target '/usr/home/git/test/deps/cowlib' failed
gmake[1]: *** [/usr/home/git/test/deps/cowlib] Error 78

I can see the missing ; on erlang.mk, DEPPKG line 137.

Also I tried to use bash -> edited chsh to use /usr/local/bin/bash.

the deps target checks for -f $$dep/Makefile although other filenames are valid

Suggestion: change check to make -n -C $$dep 2>&1 | head -1 | grep -q -v "no makefile found"

Hi,
I have

PROJECT = echo_get
DEPS = cowboy hanoidb
dep_hanoidb = git https://github.com/krestenkrab/hanoidb master
dep_cowboy = git https://github.com/brigadier/cowboy 1.0.x
include ../../erlang.mk

as a makefile, actually using one of examples, and getting the errors on make:

Dependency not available: sext-.* ({git,"git://github.com/esl/sext",
                                       {branch,"master"}})
Dependency not available: lager-.* ({git,"git://github.com/basho/lager",
                                     {branch,"master"}})
Dependency not available: snappy-1.1.* ({git,
                                         "git://github.com/fdmanana/snappy-erlang-nif.git",
                                         {branch,"master"}})
Dependency not available: plain_fsm-1.1.* ({git,
                                            "git://github.com/gburd/plain_fsm",
                                            {branch,"master"}})
Dependency not available: basho_bench-.* ({git,
                                           "git://github.com/basho/basho_bench",
                                           {branch,"master"}})
Dependency not available: triq-.* ({git,"git://github.com/krestenkrab/triq",
                                       {branch,"master"}})
Dependency not available: lz4-.* ({git,
                                      "git://github.com/krestenkrab/erlang-lz4.git",
                                      {branch,"master"}})
ERROR: compile failed while processing /home/evgeny/projects/bigboss/cowboy/examples/echo_get/deps/hanoidb: rebar_abort

I just don't understand, should I convert the whole dependency tree to erlang.mk or it is possible to compile it as is?

Basically, execute

erlc -o ./src -I ./include +noobj ./asn1/*

but with proper targets.

Adding this as an issue, as you're working on v2.0 anyways.

If your pwd contains spaces then very weird things will happen.

Typically the package list will be downloaded and stored in pwd.split(' ')[0].

Writing release notes for my application and there is nothing that I can reference to indicate what version of erlang.mk my application is dependent on.

Soft link of folder in iCloud Drive such as /Library/Mobile Documents/comapple~CloudDocs/test cannot be processed by erlang.mk.

make -f erlang.mk bootstrap
erlang.mk:641: warning: overriding commands for target /Users/gary/Library/Mobile' erlang.mk:158: warning: ignoring old commands for target/Users/gary/Library/Mobile'
erlang.mk:741: warning: overriding commands for target /Users/gary/Library/Mobile' erlang.mk:641: warning: ignoring old commands for target/Users/gary/Library/Mobile'

Hi,
ex rebar user, first time I use deps with erlang.mk and have what I think is a bug .
my module is 'a' depending of 'b' and 'c' ...
make exit in error with below log

Clonage dans '/home/eric/git/a/deps/b'...
remote: Counting objects: 27, done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 27 (delta 8), reused 26 (delta 7)
Unpacking objects: 100% (27/27), done.
Vérification de la connectivité... fait.
Clonage dans '/home/eric/git/a/deps/c'...
remote: Counting objects: 57, done.
remote: Compressing objects: 100% (45/45), done.
remote: Total 57 (delta 17), reused 39 (delta 8)
Unpacking objects: 100% (57/57), done.
Vérification de la connectivité... fait.
make[1]: entrant dans le répertoire « /home/eric/git/a/deps/b »
ERLC b.erl
grep: src/a.app.src: Aucun fichier ou dossier de ce type
Empty modules entry not found in a.app.src. Please consult the erlang.mk README for instructions.
make[1]: *** [app] Erreur 1
make[1]: quittant le répertoire « /home/eric/git/a/deps/c »
make[1]: entrant dans le répertoire « /home/eric/git/a/deps/c »
ERLC c.erl
grep: src/a.app.src: Aucun fichier ou dossier de ce type
Empty modules entry not found in a.app.src. Please consult the erlang.mk README for instructions.
make[1]: *** [app] Erreur 1
make[1]: quittant le répertoire « /home/eric/git/a/deps/c »
make: *** [deps] Erreur 2

looks like when compiling dependencies 'b' and 'c' , grep for modules presence is always done with 'a' project .app.src filename in the below dependencies ...
A bug or maybe I did not understood how it works...

It would be very nice if erlang.mk could handle the absence for wget by falling back (or forward) to curl instead. Some systems don't come with wget any more (i.e. OS X) but only have curl.

If run make with -j N then compile source may happen before all deps fetched and compiled.

For example cowboy:

v.yurin@hell:~/github/cowboy$ make -j2
--2014-11-12 20:39:50--  https://raw.githubusercontent.com/ninenines/erlang.mk/master/packages.v2.tsv
--2014-11-12 20:39:50--  https://raw.githubusercontent.com/ninenines/erlang.mk/master/packages.v2.tsv
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.31.17.133
185.31.17.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.31.17.133|:443... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.31.17.133|:443... connected.
connected.
HTTP request sent, awaiting response... HTTP request sent, awaiting response... 200 OK
Length: 7115 (6.9K) [text/plain]
Saving to: ‘/home/v.yurin/github/cowboy/.erlang.mk.packages.v2’

/home/v.yurin/github/cowboy/.erlang.mk.pac 100%[========================================================================================>]   6.95K  --.-KB/s   in 0s     

2014-11-12 20:39:51 (49.1 MB/s) - ‘/home/v.yurin/github/cowboy/.erlang.mk.packages.v2’ saved [7115/7115]

200 OK
Length: 7115 (6.9K) [text/plain]
Saving to: ‘/home/v.yurin/github/cowboy/.erlang.mk.packages.v2’

/home/v.yurin/github/cowboy/.erlang.mk.pac 100%[========================================================================================>]   6.95K  --.-KB/s   in 0s     

2014-11-12 20:39:51 (44.2 MB/s) - ‘/home/v.yurin/github/cowboy/.erlang.mk.packages.v2’ saved [7115/7115]

Cloning into '/home/v.yurin/github/cowboy/deps/cowlib'...
Cloning into '/home/v.yurin/github/cowboy/deps/ranch'...
remote: Counting objects: 927, done.
remote: Total 927 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (927/927), 295.01 KiB | 53.00 KiB/s, done.
Resolving deltas: 100% (546/546), done.
Checking connectivity... done.
 ERLC   cowboy_rest.erl cowboy_static.erl cowboy_middleware.erl cowboy_req.erl cowboy_loop.erl cowboy_constraints.erl cowboy.erl cowboy_handler.erl cowboy_websocket.erl cowboy_sub_protocol.erl cowboy_http.erl cowboy_clock.erl cowboy_router.erl cowboy_bstr.erl cowboy_app.erl cowboy_sup.erl cowboy_spdy.erl cowboy_protocol.erl
src/cowboy_protocol.erl:57: can't find include lib "cowlib/include/cow_inline.hrl"
src/cowboy_protocol.erl:272: undefined macro 'INLINE_LOWERCASE/9'
src/cowboy_protocol.erl:397: undefined macro 'INLINE_LOWERCASE/4'
src/cowboy_protocol.erl:257: function parse_hd_name/8 undefined
src/cowboy_protocol.erl:369: function parse_host/3 undefined
src/cowboy_protocol.erl:275: function parse_hd_name_ws/8 is unused
src/cowboy_protocol.erl:282: function wait_hd_before_value/8 is unused
src/cowboy_protocol.erl:295: function parse_hd_before_value/8 is unused
src/cowboy_protocol.erl:314: function wait_hd_value/9 is unused
src/cowboy_protocol.erl:329: function wait_hd_value_nl/9 is unused
src/cowboy_protocol.erl:343: function parse_hd_value/9 is unused
erlang.mk:257: recipe for target 'ebin/cowboy.app' failed
make: *** [ebin/cowboy.app] Error 1
make: *** Waiting for unfinished jobs....
remote: Counting objects: 223, done.
remote: Total 223 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (223/223), 92.66 KiB | 28.00 KiB/s, done.
Resolving deltas: 100% (130/130), done.
Checking connectivity... done.
v.yurin@hell:~/github/cowboy$ make -j2
make[1]: Entering directory '/home/v.yurin/github/cowboy/deps/cowlib'
 ERLC   cowboy_rest.erl cowboy_static.erl cowboy_middleware.erl cowboy_req.erl cowboy_loop.erl cowboy_constraints.erl cowboy.erl cowboy_handler.erl cowboy_websocket.erl cowboy_sub_protocol.erl cowboy_http.erl cowboy_clock.erl cowboy_router.erl cowboy_bstr.erl cowboy_app.erl cowboy_sup.erl cowboy_spdy.erl cowboy_protocol.erl
 ERLC   cow_mimetypes.erl cow_qs.erl cow_date.erl cow_http_te.erl cow_spdy.erl cow_http.erl cow_cookie.erl cow_http_hd.erl cow_multipart.erl
 APP    cowlib.app.src
make[1]: Leaving directory '/home/v.yurin/github/cowboy/deps/cowlib'
make[1]: Entering directory '/home/v.yurin/github/cowboy/deps/ranch'
 ERLC   ranch_acceptors_sup.erl ranch_listener_sup.erl ranch_conns_sup.erl ranch_acceptor.erl ranch_protocol.erl ranch_app.erl ranch_transport.erl ranch_sup.erl ranch_ssl.erl ranch_server.erl ranch.erl ranch_tcp.erl
 APP    cowboy.app.src
 APP    ranch.app.src
make[1]: Leaving directory '/home/v.yurin/github/cowboy/deps/ranch'

I need the corresponding syntax for erl_transform for lager, I can't find it anywhere

Please do note erlang.mk is GNU Make dependent and does not work on BSD make at README.md.

1. -Werror might be too cruel
2. https://github.com/extend/erlang.mk/blob/master/erlang.mk#L103 et al. breaks makefile inner logic -- such loops allow to go on throwing away build results. Should be reconsidered imho
3. No drop-in replacement for pre_hooks (corrected in target's Makefile but an automatic logic may be welcome; discussable)

Trying out erlang.mk on a dummy project, or with existing projects (ie: cowboy)
Always get the same error. This happens on both my mac and my linux box (linux release details below)

Looks like the dependency target generation isn't working, or at least isn't working on my machines.

This is with GNU Make 3.81 (both on linux and osx)

$ git clone [email protected]:extend/cowboy.git
Cloning into 'cowboy'...
remote: Counting objects: 4827, done.
remote: Compressing objects: 100% (2434/2434), done.
remote: Total 4827 (delta 3045), reused 4055 (delta 2339)
Receiving objects: 100% (4827/4827), 2.94 MiB | 1.29 MiB/s, done.
Resolving deltas: 100% (3045/3045), done.
$ cd cowboy/
$ make
make: *** No rule to make target `/home/dbudworth/gits/cowboy/deps/ranch', needed by `deps'.  Stop.
$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=13.04
DISTRIB_CODENAME=raring
DISTRIB_DESCRIPTION="Ubuntu 13.04"

When using erlang.mk in OS X, dependencies that have other dependencies get their own nested deps folder created. This means that running make tests won't work since only deps/*/ebin paths are added to ct_run.
This is not happening in Linux (where every dependency is under a single deps folder) so our guess is that the issue is related to a difference between the implementation of some core utility. Although we haven't been able to identify which yet, we are working on it.

I was encountering issues with erlang.mk not filling out the {modules, []} portion of app.src on OSX 10.9 (recently). It affected my own projects along with cowboy (cowlib, ranch, jiffy, gproc unaffected).

Using gnu-sed from Homebrew fixed the issue with no changes to erlang.mk.

I want to use https://github.com/rustyio/sync during development but not in releases. Currently I add sync to the DEPS but it would be nice to have a DEV_DEPS for stuff like this.

this parse transform is automatically added to right modules by
including EUnit header file:

   -include_lib("eunit/include/eunit.hrl").

https://github.com/erlang/otp/blob/maint/lib/eunit/include/eunit.hrl#L86

so I would like to remove parse_transform from this line:

TEST_ERLC_OPTS += -DTEST=1 -DEXTRA=1 +'{parse_transform, eunit_autoexport}'

otherwise we have warnings about already exported functions

make[1]: *** No rule to make target `/Users/suyejun/github/httpsrv/deps/ranch', needed by `deps'.  Stop.

Makefile:

PROJECT = httpsrv
DEPS = cowboy lager jsx

dep_cowboy = https://github.com/extend/cowboy.git 0.8.5
dep_lager = https://github.com/basho/lager.git 2.0.0
dep_jsx = https://github.com/talentdeficit/jsx.git v1.4.2

include erlang.mk

In order to rebuild automatically based on changes in header files, you can add a rule to your makefile:

%.erl: include/*.hrl src/*.hrl ; @touch $@

This could be mentioned in the readme, or added to erlang.mk..

Using $(shell find src/ -type f -name '*.erl') instead of src/*.erl we can find all .erl files within sub-directories. Same is applicable for $(wildcard test/*.erl test/*/*.erl) and $(wildcard templates/*.dtl)

Hi Loïc,

Not an issue, but a general question.

I do not see any tips on erlang.mk and erlang drivers (ports) integration.
This question raised in my mind while looking at an Erlang project using a Makefile to compile the ports.
Do you know some examples ? Is there some good practices ? Your opinion is welcome...
Regards

At the moment erlang.mk uses git in order to download dependencies. However, there are some others posibilities like:

• hg
• wget for archives like .ez or .tar.gz
• svn

In one of my project I use modified version of erlang.mk to get rabbitmq-erlang-client using hq and rabbit_common-x.y.z.ez using wget and unziping it. In order to distinguish tool which should be uses to download dependencies I propose keyword before link to package, like: dep_pkg tool link rev.
So it could look like:

DEPS = pkg1 pkg2 pkg3 pkg4
dep_pkg1 = pkg pkg1 master
dep_pkg2 = git git@host:pkg2 v1.0
dep_pkg3 = hg http://host/pk3 v2.0
dep_pkg4 = wget http://host/pkg4.ez

I'm curious what other users of erlang.mk think about that?

Unlike rebar, erlang.mk does not add autogenerated modules prop to .app file, it fills up both empty and existing one instead. If modules prop is missing at all, no modules will land .app at all and your app won't work. If modules prop value is anything but [], no modules will filled either.

Hi Loïc,
I love erlang.mk but many projects are using rebar. Mixing projects in dependencies using both is generally a nightmare. I do not like this split in Erlang community.
What about creating a erlang.mk plugin that creates a minimal rebar.config ?
(I did not really think seriously to such plugin at this time, maybe more complicated than I think, but...)
Is this something that you can tolerate or your allergy to rebar is so high that you would refuse ? ;>)

It would be nice to have a more elegant way of storing the packages then in a file inside a github repo. These things would be great to have:

• Central storage of available packages in a hosted database.
• CLI and/or web based tools to
  • publish packages
  • edit packages
  • search/browse

I know that is not something trivial, but it would be awesome to have something like that.
If this already exists for erlang and I'm just too blind to find it please point me in the right way and close this issue :)

app::
@echo 1

app::
@echo 2

app::
@echo 3

$ make
1
2
3

Suggested by Garrett Smith.

This would allow seamless integration of other Makefiles, for example for building C source code alongside erlang.mk.

What we could do is to generate $CFLAGS, $CXXFLAGS and $LDFLAGS (Erlang's general stuff, like proper include and lib paths, maybe some general flags like -fPIC).

It's not sufficient to build the contents of c_src explicitly, but if we create another Makefile inside c_src and reuse those $CFLAGS and other stuff in that Makefile.

When i run the command "make -f erlang.mk all", i got "behaviour cowboy_http_handler undefined".

Hi, I'm trying to run a test suite with my existing app config. Going through the erlang.mk options , CT_OPTS seemed like the way to go with this so I did try CT_OPTS = -config debug.config in my makefile. I still can't get access to my config file after doing this. How do you set external config files for common tests with erlang.mk? Thanks.

@essen what do you think about adding a target to erlang.mk so that we don't need to add on each project Makefile. For example

I've got a project with a number of dependencies that use rebar and have no Makefile.
Made sure that each has {modules, []} in the .src file for sed to update.
Updated to latest make and rebar.

What exactly needs to happen for a dependency to get its dependencies via rebar?

Is a Makefile mandatory?

Thankx!

It could be useful for those who keep templates not in "templates" directory - I prefer "priv/templates" for example.
something like
TPL_DIR ?= $(CURDIR)/templates
and use $(TPL_DIR) for "ebin/$(PROJECT).app" target

Following the cowboy getting started guide after the step adding DEPS=cowboy to Makefile the next make fails with error 78 in del_target/dep_fetch:

DEPPKG=$(awk 'BEGIN { FS = "\t" }; $1 == "cowboy" { print $2 " " $3 " " $4 }' /usr/home/peer/experiment/hello_erlang/.erlang.mk.packages.v2;) VS=$(echo $DEPPKG | cut -d " " -f1); REPO=$(echo $DEPPKG | cut -d " " -f2); COMMIT=$(echo $DEPPKG | cut -d " " -f3);      if [ "$VS" = "git" ]; then git clone -n -- $REPO /usr/home/peer/experiment/hello_erlang/deps/cowboy; cd /usr/home/peer/experiment/hello_erlang/deps/cowboy && git checkout -q $COMMIT; else exit 78; fi
gmake: *** [/usr/home/peer/experiment/hello_erlang/deps/cowboy] Error 78

Trying out the echoed steps in the shell works well. Tried out bash and /bin/sh which is a pure POSIX shell on FreeBSD.

Versions:

• FreeBSD 8.2
• gmake 3.82
• ERLANG_MK_VERSION = 1

Hi, I think there's a problem for changes to RELX_OUTPUT_DIR to be correctly taken into account. Shouldn't it be

@$(RELX) -c $(RELX_CONFIG) $(RELX_OPTS) -o $(RELX_OUTPUT_DIR)

?

As discussed with @asabil on IRC, we discovered that erlang.mk is a bit too smart about checking for the existence of relx.config before it has been generated.

I know you have strong feelings against including apps/ directory support.
Many projects also use alternative location for templates. (There's a PR for that).
Not all projects have a Makefile
Not all projects have {modules, []} in .src.

For erlang.mk adoption, wouldn't it be useful to solve these in erlang.mk?

e.g. PROJECT = my_projectCRLF

I want to add "https://github.com/synrc/fs.git" as deps to erlang.mk
Here is my Makefile

PROJECT = project
DEPS = fs
dep_fs = git https://github.com/synrc/fs.git
include erlang.mk

Then I run
make deps
And see that in ..../deps/fs/ebin there are only compiled *.beam but no fs.app

make deps
Cloning into '/project/deps/fs'...
remote: Counting objects: 95, done.
remote: Total 95 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (95/95), done.
make[1]: Entering directory /project/deps/fs' ERLC fanotify.erl fsevents.erl inotifywait.erl fs.erl fs_server.erl fs_sup.erl fs_event_bridge.erl fs_app.erl compile: warnings being treated as errors src/fs.erl:2: export_all flag enabled - all functions will be exported make[1]: *** [ebin/fs.app] Error 1 make[1]: Leaving directory/project/deps/fs'
make: *** [deps] Error 2

For example with the "cowboy" as dep everything is OK.

If I have a project in a directory similar to this with a space its name:

/home/alex/My Project/

and it has external dependencies then when I try to compile it I get a error similar to this:

$> make
erlang.mk:273: warning: overriding commands for target /home/alex/My' erlang.mk:177: warning: ignoring old commands for target/home/alex/My'
rm -rf rel/yada
--2014-04-23 21:07:47-- http://project/.erlang.mk.packages.v1
Resolving project (project)... failed: Name or service not known.
wget: unable to resolve host address ‘project’
--2014-04-23 21:07:47-- https://raw.github.com/extend/erlang.mk/master/packages.v1.tsv
Resolving raw.github.com (raw.github.com)... 185.31.17.133
Connecting to raw.github.com (raw.github.com)|185.31.17.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1272 (1.2K) [text/plain]
Saving to: ‘/home/alex/My’

100%[====================================================================================================================>] 1,272 --.-K/s in 0s

2014-04-23 21:07:47 (122 MB/s) - ‘/home/alex/My’ saved [1272/1272]

FINISHED --2014-04-23 21:07:47--
Total wall clock time: 0.4s
Downloaded: 1 files, 1.2K in 0s (122 MB/s)
rm: cannot remove ‘Project/.erlang.mk.packages.v1’: No such file or directory
make: *** [/home/alex/My] Error 1

It would be useful for erlang.mk to support dependencies for generating documentation.

My use case is to easily use https://github.com/uwiger/edown.git together with erlang.mk

Hi Loïc,
is there a good reason I miss to have to build erlang.mk with the plugins we want ?
why not using conditional includes like :
-include .erlang.mk/plugins/xxxx.mk
this include file only if present, without error if missing.

this would allow to upgrade the erlang.mk and all plugins needed locally, without having to recompile elsewhere the project.

a simple :
touch .erlang.mk/plugins/xxxx.mk
and 'make upgrade' would download the plugin newly needed...