Subject: Re: [Boost-build] Dependencies dropped during concurrent builds
From: Steven Watanabe (watanabesj_at_[hidden])
Date: 2013-10-31 11:50:38
AMDG
On 10/31/2013 06:24 AM, Nogradi, Chris wrote:
> Steve,
>
> Sorry about that, I retired for the evening.
>
> Well that target is a hack. In order to work around the problem that you are fixing, I had to break up a large generator into many and in order to run the script that generates the large number of targets once, I used a temp file to see if I had run it already in the action. Here is the generator:
>
> rule gen_files ( targets * : sources * : properties * )
> {
> ...
> JAM_SEMAPHORE on $(targets) = <s>verify-gen-semaphore ;
> local temp = $(targets[1]:G)verify_temp ;
> NOCARE $(temp) ;
> LOCATE on $(temp) = [ on $(targets[1]) return $(LOCATE) ] ;
> TEMP on $(targets) = $(temp) ;
> DEPENDS $(temp) : $(sources) ;
> DEPENDS $(targets) : $(temp) ;
> common.RmTemps $(temp) : $(temp) ;
> }
>
Can $(targets) ever have more than one target here?
If so, then the problem is definitely the
JAM_SEMAPHORE. (gen_files now waits until
all its targets are ready to update. JAM_SEMAPHORE
prevents more than one target from being ready
at the same time.). Ugh. I'm going to have to
rewrite the semaphore code, too.
> Here is the action:
>
> .RM = [ common.rm-command ] ;
> .IF_EXISTS = "if exist" ;
> .CREATE = [ common.file-creation-command ] ;
>
> actions gen_files bind TEMP
> {
> $(.IF_EXISTS) "$(TEMP)" exit 0
> $(.CREATE) "$(TEMP)"
> "$(.path)verify_code_gen.py" -x $(>[2]) -d $(>[1]) -e -s $(>[1]:D) -a $(<[1]:D) --lf @($(<[1]:S=.libs:W):E=$(.src-path)$(INCLUDES:J=;);$(IMP_INCLUDES:J=;))
> }
>
> Anyways, I reverting this ugliness and now the generator looks something like this (modules can be up to 30 or so strings):
>
> for local mod in $(modules)
> {
> .targets += VRFY_PUB_GEN_H($(mod)_pub_verify_%) VRFY_PUB_GEN_TEST_H($(mod)_pub_verify_test_%)
> VRFY_PRV_GEN_H($(mod:L)_prv_verify_%) VRFY_PRV_GEN_TEST_H($(mod:L)_prv_verify_test_%)
> VRFY_GEN_C($(mod:L)_verify_%) VRFY_GEN_TEST_C($(mod:L)_verify_test_%) ;
> }
> generators.register-composing verify_limits.gen_files : VRFY_DICT CTYPE_DEFS : $(.targets) ;
>
> When I run this version I get similar behavior to what I had originally reported to you that the build completes with no failures but not all targets are built:
>
> ...patience...
> ...patience...
> ...patience...
> ...patience...
> ...patience...
> ...found 123386 targets...
> ...updating 5684 targets...
> <snip>
> ...updated 5500 targets...
>
> If I run again, I get:
>
> ...patience...
> ...patience...
> ...patience...
> ...patience...
> ...patience...
> ...patience...
> ...found 133400 targets...
> ...updating 184 targets...
>
> And nothing is built.
>
> However if I now try to build one of the dlls, it actually builds it (-d+12 attached in out file). Then if I invoke the entire build, it actually finishes and produces all the dlls and the final exe (out2 attached).
>
> Thanks for your help diagnosing this.
>
It looks like the problem is sys_resource.dll.
Is there anything unusual about this dll?
In Christ,
Steven Watanabe