build ghc with make (ghc-boot-th broken deps)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

build ghc with make (ghc-boot-th broken deps)

Daneel Yaitskov-2
Hi List,

I pulled ghc repo (568d7279a) and trying to build with following env:
Ghc from stack (8.8.3). I made links to ghc and ghc-pkg in /usr/bin

./boot is passing well, but ./configure breaks on ghc-boot step.
I have no idea how could I fix this issue.

I manually modified dep versions in to what I have for ghc-8.8.3 and
issue with ghc-boot-th disappeared and appeared parsec

libraries/bootstrapping.conf/binary-0.8.7.0.conf

 depends:
    array-0.5.4.0 base-4.13.0.0 bytestring-0.10.10.0 containers-0.6.2.1

but there was
depends:
    array-0.5.3.0 base-4.12.0.0 bytestring-0.10.8.2 containers-0.6.0.1

Where is actual problem?
Old cabal?

10:45$ cabal --version
cabal-install version 1.24.0.2
compiled using version 1.24.2.0 of the Cabal library



Configuring ghc-boot-th-8.11.0.20200520...
"/usr/bin/ghc-pkg" update -v0 --force
--package-db=libraries/bootstrapping.conf
libraries/ghc-boot-th/dist-boot/inplace-pkg-config
"inplace/bin/ghc-cabal" configure libraries/ghc-boot dist-boot
--with-ghc="/usr/bin/ghc" --with-ghc-pkg="/usr/bin/ghc-pkg"
--package-db=/home/dan/demo/haskell/compiler/ghc/libraries/bootstrapping.conf
--disable-library-for-ghci --enable-library-vanilla
--enable-library-for-ghci --disable-library-profiling --disable-shared
--with-hscolour="/home/dan/.cabal/bin/HsColour"
--configure-option=CFLAGS="-Wall     -Werror=unused-but-set-variable
-Wno-error=inline -iquote
/home/dan/demo/haskell/compiler/ghc/libraries/ghc-boot"
--configure-option=LDFLAGS="  " --configure-option=CPPFLAGS="   "
--gcc-options="-Wall     -Werror=unused-but-set-variable
-Wno-error=inline -iquote
/home/dan/demo/haskell/compiler/ghc/libraries/ghc-boot   "
--constraint "binary == 0.8.7.0"   --constraint "transformers ==
0.5.6.2"   --constraint "mtl == 2.2.2"   --constraint "hpc == 0.6.1.0"
  --constraint "ghc-boot-th == 8.11.0.20200520"   --constraint
"ghc-boot == 8.11.0.20200520"   --constraint "template-haskell ==
2.17.0.0"   --constraint "text == 1.2.4.0"   --constraint "parsec ==
3.1.14.0"   --constraint "Cabal == 3.3.0.0"   --constraint "ghc-heap
== 8.11.0.20200520"   --constraint "exceptions == 0.10.4"
--constraint "ghci == 8.11.0.20200520"   --constraint "terminfo ==
0.4.1.4" --with-gcc="gcc" --with-ld="ld.gold" --with-ar="ar"
--with-alex="/home/dan/.cabal/bin/alex"
--with-happy="/home/dan/.cabal/bin/happy"
Configuring ghc-boot-8.11.0.20200520...
Error:
    The following packages are broken because other packages they
depend on are missing. These broken packages must be rebuilt before
they can be used.
installed package binary-0.8.7.0 is broken due to missing package
array-0.5.3.0, base-4.12.0.0, bytestring-0.10.8.2, containers-0.6.0.1

libraries/ghc-boot/ghc.mk:3: recipe for target
'libraries/ghc-boot/dist-boot/package-data.mk' failed
make[1]: *** [libraries/ghc-boot/dist-boot/package-data.mk] Error 1
Makefile:123: recipe for target 'all' failed
make: *** [all] Error 2


--

Best regards,
Daniil Iaitskov
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: build ghc with make (ghc-boot-th broken deps)

Ben Gamari-2
Daneel Yaitskov <[hidden email]> writes:

> Hi List,
>
> I pulled ghc repo (568d7279a) and trying to build with following env:
> Ghc from stack (8.8.3). I made links to ghc and ghc-pkg in /usr/bin
>
Hi Daneel,

> ./boot is passing well, but ./configure breaks on ghc-boot step.
> I have no idea how could I fix this issue.
>
Can you offer more detail here? How precisely does it break? Can you
paste the full output from ./configure?

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: build ghc with make (ghc-boot-th broken deps)

Daneel Yaitskov-2
Hi Ben,

Thanks for bothering.
I solved issue.
That was ancient cabal.
I build ghc successfully once I installed Cabal-3.
I thought that configure script will warn me about incompatible cabal :)

Btw could you shed some light on RTS locking?

I am interested in:

    ccall traceUserBinaryMsg(MyCapability() "ptr", msg "ptr", len);

because it looks slower on smaller chunks.
Writing data in 64 bytes chunks vs 4k (ghc 8.11) is twice slower.

https://github.com/yaitskov/eventslog-benchmark
benchmark [      LoadWithBinary-b1048576-t001-c0064] lasted for cpu
4.23 ms real 4.24 ms
benchmark [      LoadWithBinary-b1048576-t001-c0128] lasted for cpu
3.97 ms real 3.91 ms
benchmark [      LoadWithBinary-b1048576-t001-c1024] lasted for cpu
2.73 ms real 2.61 ms
benchmark [      LoadWithBinary-b1048576-t001-c2048] lasted for cpu
2.34 ms real 2.23 ms
benchmark [      LoadWithBinary-b1048576-t001-c4096] lasted for cpu
2.16 ms real 2.20 ms

Eventlog messages tend to be small.
So application developer has to think about bufferization to speed up.
OpenTelemetry gains 6x speed improvement with following hack.

mm :: IO (IORef (Int, Builder))
mm = newIORef (0, mempty)

traceBuilder :: MonadIO m => (Int, Builder) -> m ()
traceBuilder !sizedBuilder@(size, builder) = do
  liftIO $! do
    tls <- G.mkTLS mm
    ref <- G.getTLS tls
    (!bufferedSize, !bufferedBuilders) <- readIORef ref
    if bufferedSize + size > 1024
    then do
      traceBinaryEventIO . LBS.toStrict $! toLazyByteString bufferedBuilders
      writeIORef ref sizedBuilder
    else do
      writeIORef ref (bufferedSize + size, builder <> bufferedBuilders)



I see that some EventLog methods use mutexes, but user message function don't.
It just appends to buffer and can flush if it is full.
So I don't see where locking happening.
MyCapability is just arithmetic

#define MyCapability()  (BaseReg - OFFSET_Capability_r)

I guess capability locking happens somewhere upper - before C.

How to register RTS function, which would don't acquire capability lock?
I think locking for this case could be replaced with atomic variable.
C11 standard provides such IPC abstraction.

Thanks,
Daniil

On 5/20/20, Ben Gamari <[hidden email]> wrote:

> Daneel Yaitskov <[hidden email]> writes:
>
>> Hi List,
>>
>> I pulled ghc repo (568d7279a) and trying to build with following env:
>> Ghc from stack (8.8.3). I made links to ghc and ghc-pkg in /usr/bin
>>
> Hi Daneel,
>
>> ./boot is passing well, but ./configure breaks on ghc-boot step.
>> I have no idea how could I fix this issue.
>>
> Can you offer more detail here? How precisely does it break? Can you
> paste the full output from ./configure?
>
> Cheers,
>
> - Ben
>
>


--

Best regards,
Daniil Iaitskov
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: build ghc with make (ghc-boot-th broken deps)

Ben Gamari-2
Daneel Yaitskov <[hidden email]> writes:

> Hi Ben,
>
> Thanks for bothering.
> I solved issue.
> That was ancient cabal.
> I build ghc successfully once I installed Cabal-3.
> I thought that configure script will warn me about incompatible cabal :)
>
Hmm, interesting. I'm actually rather surprised that the problem was
cabal-install. Afterall, GHC's build system doesn't use cabal-install
apart from building Hadrian.

> Btw could you shed some light on RTS locking?
>
> I am interested in:
>
>     ccall traceUserBinaryMsg(MyCapability() "ptr", msg "ptr", len);
>
> because it looks slower on smaller chunks.
> Writing data in 64 bytes chunks vs 4k (ghc 8.11) is twice slower.
>
> https://github.com/yaitskov/eventslog-benchmark
> benchmark [      LoadWithBinary-b1048576-t001-c0064] lasted for cpu
> 4.23 ms real 4.24 ms
> benchmark [      LoadWithBinary-b1048576-t001-c0128] lasted for cpu
> 3.97 ms real 3.91 ms
> benchmark [      LoadWithBinary-b1048576-t001-c1024] lasted for cpu
> 2.73 ms real 2.61 ms
> benchmark [      LoadWithBinary-b1048576-t001-c2048] lasted for cpu
> 2.34 ms real 2.23 ms
> benchmark [      LoadWithBinary-b1048576-t001-c4096] lasted for cpu
> 2.16 ms real 2.20 ms
>
> Eventlog messages tend to be small.
>
> So application developer has to think about bufferization to speed up.
> OpenTelemetry gains 6x speed improvement with following hack.
>
...

Interesting. Please do open a GitLab ticket to track this. You might
also be interested in #17949, which is covers the case of traceEvent
being expensive when tracing is disabled.

> I see that some EventLog methods use mutexes, but user message function don't.
> It just appends to buffer and can flush if it is full.
> So I don't see where locking happening.
> MyCapability is just arithmetic
>
> #define MyCapability()  (BaseReg - OFFSET_Capability_r)
>
> I guess capability locking happens somewhere upper - before C.
>
There is no locking necessary. Capabilities are owned by a particular
operating system thread. The capability is held until the program yields
it (which generally only happens during a GC or foreign function call)
and during this time the program has full access to all of the state in
the Capability structure without the need for a lock. This is why each
Capability has its own eventlog buffer; it allows lock-free tracing.

Cheers,

- Ben

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment