Initial make or make clean; make complains about missing core_scrape_flag.h file about clasp HOT 18 CLOSED

Can you provide more details on where the file is being created in the build process? I couldn't seem to grep/search for the filename anywhere.

from clasp.

The __scrape_flag.h files are created by boost-build code in bundle.jam to indicate to boost-build that scraping has "changed". It was never intended that something like "make clean" would be run that cleaned out all of the build files (and that may be the way to fix this). I have a masters thesis written by one of my former students who wrote the boost-build code that creates the "__scrape_flag.h" files. I have to pull it out and read it to figure out what he did. He may drop in some time and provide some advice on how to fix this problem (hi Ed!).

from clasp.

I forked the repo and am going to take a look at it. Just pulled out my thesis to refresh my memory.

from clasp.

Fwiw, I can reproduce this reliably with by attempt at building with Debian packaging, but if I directly call make, it doesn't show up. See #34

This issue is my current blocker.

Hmm, if I run make twice the first time it fails, the second time it succeeds. The file appears to be
src/core/bin/boehm/clang-linux-3.6.0/release/link-static/core_scrape_flag.h. Which appears to be blank. Weird.

Ok, just to be clear, the reason that I can reproduce it reliably with the Debian build process is that the Debian build process forces a complete clean (else it won't proceed by default) and so it is not a comparable situation. It is clear this file is only built if some other file is already there, and that file is not available for whatever reason the first time around.

I can't find any mention of this file in the sources, so it is a bit difficult to track down where and how it is being built.

from clasp.

the common/symbolScraper.py script makes symbols_scraped_inc.h for each folder, and that generated file #includes whatever_scraped_flag.h. I have no idea what this is for.

from clasp.

The problem might be in bundler.jam:

    if ( ( mi in [ modules.peek : ARGV ] ) &&
      ( ( [ path.pwd ] = [ path.root $(dir) [ path.pwd ] ] ) ||
        ( --mi-all in [ modules.peek : ARGV ] ) ) ) {

        echo [ SHELL "cd $(dir); touch symbols_scraped_inc.h; rm symbols_scraped_inc.h; touch symbols_scraped_inc.h" ] ;

    }

    ...

    local lib = $(dir:B) ;

    make $(lib)_scrape_flag.h : symbols_scraped_inc.h : @bundler.scraped : <changes>$(out[2]) ;

    alias mi : $(lib)_initClasses_inc.h $(lib)_scrape_flag.h ;

i.e. mi is set after it's used in that if, which sets the flag.

from clasp.

It's actually in

rule scraped ( target + : sources * : properties * )
{

    if "<changes>0" in $(properties) {
      ALWAYS $(target) ;
    }
    else {
       NOUPDATE $(target) ;
    }

}

It seems that if changes is 0 and the flag does not yet exist ALWAYS does not create the flag. This is the case for core_scrape_flag.h on the first build. I think I can get this resolved tonight. I'll keep you all up to date.

from clasp.

Great, Edward. Thanks for the clarification. We look forward to hearing back from you.

from clasp.

May have to go to the boost build mailing list on this one. Fixed the missing file issue, but deleting symbols_scraped_inc.h doesn't trigger the rebuild it should. Hopefully I'll get a little insight and get a pull request in by the weekend,

from clasp.

Can you suggest a workaround so I can proceed with testing the Debian build? Restarting is not an option; the Debian build is (of course) cleaned on restarting.

from clasp.

Not sure this will work, but try running export PYTHONPATH=pwd:$(PYTHONPATH); python ../../src/common/symbolScraper.py symbols_scraped.inc *.h *.cc *.scrape.inc from the src/core directory before running make from the project root. Fingers crossed this will pre-generate the problem file.

from clasp.

Hi Edward,

Ok. If this is your suggested workaround, I'll give it a try. Thanks.

from clasp.

Hi Edward,

Ok, it errors out again with your suggested workaround, but on a different scrape file thing. Build log: https://gist.github.com/7ea773132a1ee04a7ad3
The relevant bit is

In file included from ../../src/llvmo/symbolTable.h:37:
../../src/llvmo/symbols_scraped_inc.h:2:10: fatal error: 'llvmo_scrape_flag.h' file not found
#include "llvmo_scrape_flag.h"

Here are all the *_scrape_flag.h's.

(jessiechroot)faheem@orwell:/usr/local/src/clasp/clasp-0.1$ find . -name *scrape_flag.h
./src/core/bin/boehm/clang-linux-3.6.0/release/link-static/core_scrape_flag.h
./src/serveEvent/bin/boehm/clang-linux-3.6.0/release/link-static/serveEvent_scrape_flag.h
./src/clbind/bin/boehm/clang-linux-3.6.0/release/link-static/clbind_scrape_flag.h
./src/cffi/bin/boehm/clang-linux-3.6.0/release/link-static/cffi_scrape_flag.h
./src/sockets/bin/boehm/clang-linux-3.6.0/release/link-static/sockets_scrape_flag.h
./src/mpip/bin/boehm/clang-linux-3.6.0/release/link-static/mpip_scrape_flag.h
./src/gctools/bin/boehm/clang-linux-3.6.0/release/link-static/gctools_scrape_flag.h
./src/asttooling/bin/boehm/clang-linux-3.6.0/release/link-static/asttooling_scrape_flag.h

Ideas?

from clasp.

Perhaps we have to run this in every source directory.

from clasp.

Using the following seems to work:

cd src/core; export PYTHONPATH=`pwd`:$(PYTHONPATH); python ../../src/common/symbolScraper.py symbols_scraped.inc *.h *.cc *.scrape.inc
cd src/llvmo; export PYTHONPATH=`pwd`:$(PYTHONPATH); python ../../src/common/symbolScraper.py symbols_scraped.inc *.h *.cc *.scrape.inc

from clasp.

Have narrowed this down to a very specific issue involving some surprising behavior in boost build. Submitted a request for help: http://lists.boost.org/boost-build/2014/10/27713.php.

from clasp.

Hi Edward,

Thanks for working on this.

I don't mean to tell you your business, and I don't know anything about Boost Build or the
Boost Build mailing list. However, would an actual small example program not be better?
Sometimes referred to as a Minimal Working example or MWE.

from clasp.

Cannot confirm on b05884f. Running make once works fine now.

from clasp.