Weird "missing hi file" problem with a serializable Core patch

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Weird "missing hi file" problem with a serializable Core patch

Shao, Cheng
Hi all,

Following a short chat in #ghc last week, I did a first attempt of
reusing existing Iface logic to implement serialization for
codegen-related Core. The implementation is included in the attached
patch (~100 loc). As a quick and dirty validation of whether it works,
I also modified the codegen pipeline logic to do a roundtrip: after
CorePrep, the Core bits are converted to Iface, then we immediately
convert it back and use it for later compiling.

With the patch applied, stage-1 GHC would produce a "missing hi file"
error like:

: Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi
      _build/stage1/libraries/ghc-prim/build/GHC/Types.hi:
openBinaryFile: does not exist (No such file or directory)

The error surprises me, since by the time we perform the Core-to-Core
roundtrip, the .hi file should already have been written to disk. Is
there anything obviously wrong with the implementation? I'd appreciate
any pointers or further questions, thanks a lot!

Best regards,
Cheng

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

ExtCore.patch (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Weird "missing hi file" problem with a serializable Core patch

Brandon Allbery
Without looking at the implementation, it looks to me like the filename is doubled for some reason. This may suggest places to look.

On Wed, Sep 16, 2020 at 2:57 PM Cheng Shao <[hidden email]> wrote:
Hi all,

Following a short chat in #ghc last week, I did a first attempt of
reusing existing Iface logic to implement serialization for
codegen-related Core. The implementation is included in the attached
patch (~100 loc). As a quick and dirty validation of whether it works,
I also modified the codegen pipeline logic to do a roundtrip: after
CorePrep, the Core bits are converted to Iface, then we immediately
convert it back and use it for later compiling.

With the patch applied, stage-1 GHC would produce a "missing hi file"
error like:

: Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi
      _build/stage1/libraries/ghc-prim/build/GHC/Types.hi:
openBinaryFile: does not exist (No such file or directory)

The error surprises me, since by the time we perform the Core-to-Core
roundtrip, the .hi file should already have been written to disk. Is
there anything obviously wrong with the implementation? I'd appreciate
any pointers or further questions, thanks a lot!

Best regards,
Cheng
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


--
brandon s allbery kf8nh

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Weird "missing hi file" problem with a serializable Core patch

Shao, Cheng
Thanks Brandon,

I checked the strace log and before the error is written, there's a log entry:

openat(AT_FDCWD,
"_build/stage1/libraries/ghc-prim/build/GHC/Types.hi",
O_RDONLY|O_NOCTTY|O_NONBLOCK) = -1 ENOENT (No such file or directory)

So it looks like GHC is indeed looking at the correct hi path, not the
doubled path.

On Wed, Sep 16, 2020 at 9:03 PM Brandon Allbery <[hidden email]> wrote:

>
> Without looking at the implementation, it looks to me like the filename is doubled for some reason. This may suggest places to look.
>
> On Wed, Sep 16, 2020 at 2:57 PM Cheng Shao <[hidden email]> wrote:
>>
>> Hi all,
>>
>> Following a short chat in #ghc last week, I did a first attempt of
>> reusing existing Iface logic to implement serialization for
>> codegen-related Core. The implementation is included in the attached
>> patch (~100 loc). As a quick and dirty validation of whether it works,
>> I also modified the codegen pipeline logic to do a roundtrip: after
>> CorePrep, the Core bits are converted to Iface, then we immediately
>> convert it back and use it for later compiling.
>>
>> With the patch applied, stage-1 GHC would produce a "missing hi file"
>> error like:
>>
>> : Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi
>>       _build/stage1/libraries/ghc-prim/build/GHC/Types.hi:
>> openBinaryFile: does not exist (No such file or directory)
>>
>> The error surprises me, since by the time we perform the Core-to-Core
>> roundtrip, the .hi file should already have been written to disk. Is
>> there anything obviously wrong with the implementation? I'd appreciate
>> any pointers or further questions, thanks a lot!
>>
>> Best regards,
>> Cheng
>> _______________________________________________
>> ghc-devs mailing list
>> [hidden email]
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
>
> --
> brandon s allbery kf8nh
> [hidden email]
_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Reply | Threaded
Open this post in threaded view
|

Re: Weird "missing hi file" problem with a serializable Core patch

Ben Gamari-2
In reply to this post by Shao, Cheng
Cheng Shao <[hidden email]> writes:

> Hi all,
>
> Following a short chat in #ghc last week, I did a first attempt of
> reusing existing Iface logic to implement serialization for
> codegen-related Core. The implementation is included in the attached
> patch (~100 loc). As a quick and dirty validation of whether it works,
> I also modified the codegen pipeline logic to do a roundtrip: after
> CorePrep, the Core bits are converted to Iface, then we immediately
> convert it back and use it for later compiling.
>
> With the patch applied, stage-1 GHC would produce a "missing hi file"
> error like:
>
> : Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi
>       _build/stage1/libraries/ghc-prim/build/GHC/Types.hi:
> openBinaryFile: does not exist (No such file or directory)
>
Hi Cheng,

Which module is being compiled when this error is produced? Could you
provide -ddump-if-trace output for the failing compilation?

> The error surprises me, since by the time we perform the Core-to-Core
> roundtrip, the .hi file should already have been written to disk. Is
> there anything obviously wrong with the implementation? I'd appreciate
> any pointers or further questions, thanks a lot!
>
Note that interface files are written after the Core pipeline is run.

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Weird "missing hi file" problem with a serializable Core patch

Shao, Cheng
Hi Ben,

The -ddump-if-trace output is attached here. The error is produced
when compiling GHC.Types in ghc-prim.

> Note that interface files are written after the Core pipeline is run.

Sorry for the confusion, I didn't mean the Core simplifier pipeline. I
mean the "Core -> Iface -> Core" roundtrip I tried to perform using
the output of CorePrep. By the time we do CorePrep, the hi files
should already have been written.

On Wed, Sep 16, 2020 at 11:48 PM Ben Gamari <[hidden email]> wrote:

>
> Cheng Shao <[hidden email]> writes:
>
> > Hi all,
> >
> > Following a short chat in #ghc last week, I did a first attempt of
> > reusing existing Iface logic to implement serialization for
> > codegen-related Core. The implementation is included in the attached
> > patch (~100 loc). As a quick and dirty validation of whether it works,
> > I also modified the codegen pipeline logic to do a roundtrip: after
> > CorePrep, the Core bits are converted to Iface, then we immediately
> > convert it back and use it for later compiling.
> >
> > With the patch applied, stage-1 GHC would produce a "missing hi file"
> > error like:
> >
> > : Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi
> >       _build/stage1/libraries/ghc-prim/build/GHC/Types.hi:
> > openBinaryFile: does not exist (No such file or directory)
> >
> Hi Cheng,
>
> Which module is being compiled when this error is produced? Could you
> provide -ddump-if-trace output for the failing compilation?
>
> > The error surprises me, since by the time we perform the Core-to-Core
> > roundtrip, the .hi file should already have been written to disk. Is
> > there anything obviously wrong with the implementation? I'd appreciate
> > any pointers or further questions, thanks a lot!
> >
> Note that interface files are written after the Core pipeline is run.
>
> Cheers,
>
> - Ben
>

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

if-trace.txt (29K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Weird "missing hi file" problem with a serializable Core patch

Sylvain Henry-2
> By the time we do CorePrep, the hi files should already have been written.

I don't think so. When we generate real code we write the interface after the backend has generated the output object. See Note [Writing interface files] in GHC.Driver.Main

Cheers,
Sylvain

On 17/09/2020 12:17, Cheng Shao wrote:
Hi Ben,

The -ddump-if-trace output is attached here. The error is produced
when compiling GHC.Types in ghc-prim.

Note that interface files are written after the Core pipeline is run.
Sorry for the confusion, I didn't mean the Core simplifier pipeline. I
mean the "Core -> Iface -> Core" roundtrip I tried to perform using
the output of CorePrep. By the time we do CorePrep, the hi files
should already have been written.

On Wed, Sep 16, 2020 at 11:48 PM Ben Gamari [hidden email] wrote:
Cheng Shao [hidden email] writes:

Hi all,

Following a short chat in #ghc last week, I did a first attempt of
reusing existing Iface logic to implement serialization for
codegen-related Core. The implementation is included in the attached
patch (~100 loc). As a quick and dirty validation of whether it works,
I also modified the codegen pipeline logic to do a roundtrip: after
CorePrep, the Core bits are converted to Iface, then we immediately
convert it back and use it for later compiling.

With the patch applied, stage-1 GHC would produce a "missing hi file"
error like:

: Bad interface file: _build/stage1/libraries/ghc-prim/build/GHC/Types.hi
      _build/stage1/libraries/ghc-prim/build/GHC/Types.hi:
openBinaryFile: does not exist (No such file or directory)

Hi Cheng,

Which module is being compiled when this error is produced? Could you
provide -ddump-if-trace output for the failing compilation?

The error surprises me, since by the time we perform the Core-to-Core
roundtrip, the .hi file should already have been written to disk. Is
there anything obviously wrong with the implementation? I'd appreciate
any pointers or further questions, thanks a lot!

Note that interface files are written after the Core pipeline is run.

Cheers,

- Ben


_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________
ghc-devs mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs