GHC 7.8 release?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
142 messages Options
1 ... 345678
Reply | Threaded
Open this post in threaded view
|

Re: base package

Joachim Breitner-2
Hi,

Am Freitag, den 15.02.2013, 23:00 +0100 schrieb Joachim Breitner:

> Am Freitag, den 15.02.2013, 14:50 +0000 schrieb Simon Marlow:
> > On 15/02/13 12:22, Joachim Breitner wrote:
> > > more progress: On top of base-pure, I created base-io involving GHC/IO
> > > and everything required to build it (which pulled in ST, some of Foreign
> > > and unfortunately some stuff related to Handles and Devices, because it
> > > is mentioned in IOException). This is the list of modules:
> > >
> >
> > You have a random collection of modules here :)
> >
> > I think you want to have the IO *monad* (GHC.IO) live in a lower layer,
> > separate from the IO *library* (GHC.IO.Device and so on).  Every Haskell
> > implementation will need the IO monad, but they might want to replace
> > the IO library with something else.
> >
> > Things like GHC.IORef, GHC.MVar can all live in a low-down layer because
> > they're just wrappers over the primops.
>
> Right, that is my aim, and I started with GHC.IO. But unfortunately, the
> IO monad calls failIO, which is an IOError which has a field of type
> ioe_handle :: Maybe Handle (and one of type CInt) which pulls in all the
> rest there, and so far I did not have a good idea how to untangle that.
>
> What would break if fail would not raise an IOError, but a separate
> exception type, e.g. IOFailError? Probably too much, as users expect to
> catch the exception raised by fail with an exception handler that
> matches IOError.
I’m still stuck at the problem of separating the definition of IO and
Monad IO from all file related stuff, which is prevented by the "Maybe
Handle" field in the IOError data type.

Given that IOError is abstract to the „regular“ user (i.e. not to
base-io-file), I guess we can afford to be a little bit hacky here. Two
ideas come to my mind:

1. Instead of declaring the field as "Maybe Handle", we define a
pseudo-handle datatype
        data NotYetAHandle = NotYetAHandle
and use "Maybe NotYetAHandle" in IOError, with the documented convention
that only code in base-io-file (and code further down the tree) may use
this field, and only after unsafeCoerce’ing it to a Maybe Handle. This
way, the base-io package does not have to include the definition, but
the IOError data type still has place for it.

If the NotYetAHandle constructor is not exported, and base-io-file
defines functions NotYetAHandle -> Handle and Handle -> NotYetAHandle
via unsafeCoerce, then the unsafeness is very local and nobody can break
the code without also using unsafeCoerce.

2. A little safer (and with a little more overhead) wold be to include
Data.Dynamic in base and change the field to a "Maybe Dynamic". Same
procedure as above, only that a violation of the convention might be
caught without crashes.

Is having a package that provides io without providing file-related
definition worth this kludge?

Greetings,
Joachimh


--
Joachim "nomeata" Breitner
Debian Developer
  [hidden email] | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: [hidden email] | http://people.debian.org/~nomeata


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: base package

Joachim Breitner-2
Hi,

Am Mittwoch, den 20.02.2013, 14:57 +0100 schrieb Joachim Breitner:
> I’m still stuck at the problem of separating the definition of IO and
> Monad IO from all file related stuff, which is prevented by the "Maybe
> Handle" field in the IOError data type.

re-reading „An Extensible Dynamically-Typed Hierarchy of Exceptions“
helped me to come up with this somewhat neat solution:

The Monad IO instance uses an exception different from IOError:

$ git show HEAD | filterdiff -i \*.cabal -i \*Fail\* -i \*/GHC/IO.hs
--- a/GHC/IO.hs
+++ b/GHC/IO.hs
@@ -46,8 +46,7 @@ import GHC.ST
 import GHC.Exception
 import GHC.Show
 import Data.Maybe
-
-import {-# SOURCE #-} GHC.IO.Exception ( userError )
+import GHC.IO.Fail
 
 -- ---------------------------------------------------------------------------
 -- The IO Monad
@@ -79,7 +78,7 @@ liftIO :: IO a -> State# RealWorld -> STret RealWorld a
 liftIO (IO m) = \s -> case m s of (# s', r #) -> STret s' r
 
 failIO :: String -> IO a
-failIO s = IO (raiseIO# (toException (userError s)))
+failIO s = IO (raiseIO# (toException (IOFail s)))
 
 -- ---------------------------------------------------------------------------
 -- Coercions between IO and ST
--- /dev/null
+++ b/GHC/IO/Fail.hs
@@ -0,0 +1,20 @@
+{-# LANGUAGE NoImplicitPrelude #-}
+module GHC.IO.Fail where
+
+import GHC.Base
+import GHC.Exception
+import Data.Typeable
+import GHC.Show
+
+
+-- | This exception is thrown by the 'fail' method of the 'Monad' 'IO' instance.
+--
+--   The Exception instance of IOException will also catch this, converting the
+--   IOFail to a UserError, for compatibility and consistency with the Haskell
+--   report
+data IOFail = IOFail String
+
+instance Typeable IOFail -- deriving does not work without package
+instance Show IOFail     -- name changes to GHC
+instance Exception IOFail
+

After this change,

    exposed-modules:
        GHC.IO.Fail,
        GHC.IO,
        GHC.IORef,
        GHC.ST,
        GHC.STRef

is possible (and of course ST can be moved away as well).

So far so good, but this breaks user code.  So the solution is to make
sure that to everyone who tries to catch an IOException (which will
likely be part of some base-io-file), an IOFail will look like a IOError
of type UserError:

$ git show HEAD|filterdiff -i \*Exception.hs
--- a/GHC/IO/Exception.hs
+++ b/GHC/IO/Exception.hs
@@ -45,9 +45,10 @@ import GHC.Show
 import GHC.Exception
 import Data.Maybe
 import GHC.IO.Handle.Types
+import GHC.IO.Fail
 import Foreign.C.Types
 
-import Data.Typeable     ( Typeable )
+import Data.Typeable     ( Typeable, cast )
 
 -- ------------------------------------------------------------------------
 -- Exception datatypes and operations
@@ -222,7 +223,11 @@ data IOException
    }
 instance Typeable IOException
 
-instance Exception IOException
+instance Exception IOException where
+    toException = SomeException
+    fromException e = case cast e of
+        Just (IOFail s) -> Just (userError s)
+        Nothing -> cast e
 
 instance Eq IOException where
   (IOError h1 e1 loc1 str1 en1 fn1) == (IOError h2 e2 loc2 str2 en2 fn2) =


Neat, isn’t it?

Now I can proceed separating some of the Foreign stuff from the IO
stuff.


Greetings,
Joachim
--
Joachim "nomeata" Breitner
Debian Developer
  [hidden email] | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: [hidden email] | http://people.debian.org/~nomeata


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: base package (Was: GHC 7.8 release?)

Ian Lynagh-2
In reply to this post by Simon Marlow-7
On Fri, Feb 15, 2013 at 02:45:19PM +0000, Simon Marlow wrote:
>
> Remember that fingerprinting is not hashing.  For fingerprinting we
> need to have a realistic expectation of no collisions.  I don't
> think FNV is suitable.
>
> I'm sure it would be possible to replace the C md5 code with some
> Haskell.  Performance *is* important here though - Typeable is in
> the inner loop of certain generic programming libraries, like SYB.

We currently just compare
    hash(str)
for equality, right? Could we instead compare
    (hash str, str)
? That would be even more correct, even if a bad/cheap hash function is
used, and would only be slower for the case where the types match
(unless you're unlucky and get a hash collision).

In fact, we may be able to arrange it so that in the equal case the
strings are normally exactly the same string, so we can do a cheap
pointer equality test (like ByteString already does) to make the equal
case fast too (falling back to actually checking the strings are equal,
if they aren't the same string).


Thanks
Ian


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package (Was: GHC 7.8 release?)

Edward Kmett-2
Comparing hash, ptr, str gives you a pretty good acceptance/rejection test. hash for the quick rejection, ptr for quick acceptance, str for accuracy. Especially since the particular fingerprints for Typeable at least are usually made up of 3 bytestrings that were just stuffed in and forgotten about.

That said, this would seem to bring ByteString or at least Ptr in at a pretty low level for the level of generality folks seem to be suddenly seeking.

-Edward

On Wed, Feb 20, 2013 at 12:12 PM, Ian Lynagh <[hidden email]> wrote:
On Fri, Feb 15, 2013 at 02:45:19PM +0000, Simon Marlow wrote:
>
> Remember that fingerprinting is not hashing.  For fingerprinting we
> need to have a realistic expectation of no collisions.  I don't
> think FNV is suitable.
>
> I'm sure it would be possible to replace the C md5 code with some
> Haskell.  Performance *is* important here though - Typeable is in
> the inner loop of certain generic programming libraries, like SYB.

We currently just compare
    hash(str)
for equality, right? Could we instead compare
    (hash str, str)
? That would be even more correct, even if a bad/cheap hash function is
used, and would only be slower for the case where the types match
(unless you're unlucky and get a hash collision).

In fact, we may be able to arrange it so that in the equal case the
strings are normally exactly the same string, so we can do a cheap
pointer equality test (like ByteString already does) to make the equal
case fast too (falling back to actually checking the strings are equal,
if they aren't the same string).


Thanks
Ian


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package

Simon Marlow-7
In reply to this post by Joachim Breitner-2
On 20/02/13 15:40, Joachim Breitner wrote:

> +-- | This exception is thrown by the 'fail' method of the 'Monad' 'IO' instance.
> +--
> +--   The Exception instance of IOException will also catch this, converting the
> +--   IOFail to a UserError, for compatibility and consistency with the Haskell
> +--   report
> +data IOFail = IOFail String
> +
> +instance Typeable IOFail -- deriving does not work without package
> +instance Show IOFail     -- name changes to GHC
> +instance Exception IOFail
> +

I like the idea of making IOFail a separate exception type.

> -instance Exception IOException
> +instance Exception IOException where
> +    toException = SomeException
> +    fromException e = case cast e of
> +        Just (IOFail s) -> Just (userError s)
> +        Nothing -> cast e

I think that should be

 > +    fromException (SomeException e) = case cast e of
 > +        Just (IOFail s) -> Just (userError s)
 > +        Nothing -> cast e

Otherwise it will typecheck but not work (hurrah for dynamic typing).

The trick is indeed neat, but only if it is possible to make IOFail
completely invisible.  If it isn't possible to make it completely
invisible, then I would prefer IOFail to be a first-class exception type
without the special trick to coerce it to IOException, and accept the
API breakage.  I don't think it's a good idea to have special magic in
the exception hierarchy - other people would start doing it too, then
we'd have a mess.

Cheers,
        Simon


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package (Was: GHC 7.8 release?)

Simon Marlow-7
In reply to this post by Ian Lynagh-2
On 20/02/13 17:12, Ian Lynagh wrote:

> On Fri, Feb 15, 2013 at 02:45:19PM +0000, Simon Marlow wrote:
>>
>> Remember that fingerprinting is not hashing.  For fingerprinting we
>> need to have a realistic expectation of no collisions.  I don't
>> think FNV is suitable.
>>
>> I'm sure it would be possible to replace the C md5 code with some
>> Haskell.  Performance *is* important here though - Typeable is in
>> the inner loop of certain generic programming libraries, like SYB.
>
> We currently just compare
>      hash(str)
> for equality, right? Could we instead compare
>      (hash str, str)
> ? That would be even more correct, even if a bad/cheap hash function is
> used, and would only be slower for the case where the types match
> (unless you're unlucky and get a hash collision).
 >
> In fact, we may be able to arrange it so that in the equal case the
> strings are normally exactly the same string, so we can do a cheap
> pointer equality test (like ByteString already does) to make the equal
> case fast too (falling back to actually checking the strings are equal,
> if they aren't the same string).

So it's not a single string, a TypeRep consists of a TyCon applied to
some arguments, which themselves are TypeReps etc.

You could do pointer equality, and maybe that would work for the common
cases.  But I don't see why we have to do that when fingerprinting works
well and we already have it.  Why add a potential performance pitfall
when we don't have to?

One other thing: it's useful to be able to use the fingerprint as an
identifier for the contents, e.g. when sending Typeable values across
the network.  If you can't do this with the fingerprint, then you need
another unique Id, which is the situation we used to have before
fingerprints.

Cheers,
        Simon


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package

Joachim Breitner-2
In reply to this post by Simon Marlow-7
Hi,

Am Donnerstag, den 21.02.2013, 09:42 +0000 schrieb Simon Marlow:
> The trick is indeed neat, but only if it is possible to make IOFail
> completely invisible.  If it isn't possible to make it completely
> invisible, then I would prefer IOFail to be a first-class exception type
> without the special trick to coerce it to IOException, and accept the
> API breakage.  I don't think it's a good idea to have special magic in
> the exception hierarchy - other people would start doing it too, then
> we'd have a mess.

great – for the purposes of splitting base I don’t care which one is
taken, as long as I know that there is some way.


I have now changed my setup so that I can modify the files without
removing others or moving them to different branches or symlinking. This
way, a simple "git diff ghc-7.6 base-split" lists all changes.
I also created a script to check whether every module is in exactly one
of the new packages.

See
https://github.com/nomeata/packages-base/blob/base-split/README.md
for a description of the setup and an explanation of the changes
applied.

Contributions welcome!

Greetings,
Joachim

--
Joachim "nomeata" Breitner
Debian Developer
  [hidden email] | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: [hidden email] | http://people.debian.org/~nomeata


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: base package

Joachim Breitner-2

Hi,

another status report about my base-splitting experiment:

The list of packages has grown to

 * base-pure: Basic stuff without `IO`, `Foreign` or floating point
   arithmetic.
 * base-st: The `ST` monad, uses base-pure.
 * base-array: Arrays, uses base-st.
 * base-float: Floating point types, uses base-array internally.
 * base-io: The `IO` monad, uses base-st.
 * base-concurrent: Currently just `MVar`, uses base-io.
 * base-foreign: Everything related to `Foreign`, uses base-io and
   base-float (for the `Storable` instances for `Double` and `Float`).

Of the (exactly!) 200 modules in base, 83 have not yet been sorted into
these or other packages:

Control/Concurrent.hs
Control/Concurrent/Chan.hs
Control/Concurrent/QSem.hs
Control/Concurrent/QSemN.hs
Control/Concurrent/SampleVar.hs
Control/Exception.hs
Control/Exception/Base.hs
Control/Monad/Instances.hs
Data/Data.hs
Data/Dynamic.hs
Data/Fixed.hs
Data/HashTable.hs
Data/Unique.hs
Data/Version.hs
Debug/Trace.hs
GHC/Conc.lhs
GHC/Conc/IO.hs
GHC/Conc/Signal.hs
GHC/Conc/Sync.lhs
GHC/Conc/Windows.hs
GHC/ConsoleHandler.hs
GHC/Constants.hs
GHC/Desugar.hs
GHC/Environment.hs
GHC/Event.hs
GHC/Event/Array.hs
GHC/Event/Clock.hsc
GHC/Event/Control.hs
GHC/Event/EPoll.hsc
GHC/Event/IntMap.hs
GHC/Event/Internal.hs
GHC/Event/KQueue.hsc
GHC/Event/Manager.hs
GHC/Event/PSQ.hs
GHC/Event/Poll.hsc
GHC/Event/Thread.hs
GHC/Event/Unique.hs
GHC/Exts.hs
GHC/GHCi.hs
GHC/Generics.hs
GHC/Handle.hs
GHC/IO.hs-boot
GHC/IO/Device.hs
GHC/IO/Exception.hs
GHC/IO/Exception.hs-boot
GHC/IO/FD.hs
GHC/IO/Handle.hs
GHC/IO/Handle.hs-boot
GHC/IO/Handle/FD.hs
GHC/IO/Handle/FD.hs-boot
GHC/IO/Handle/Internals.hs
GHC/IO/Handle/Text.hs
GHC/IO/Handle/Types.hs
GHC/IO/IOMode.hs
GHC/IOArray.hs
GHC/IOBase.hs
GHC/IP.hs
GHC/PArr.hs
GHC/Pack.lhs
GHC/Stack.hsc
GHC/Stats.hsc
GHC/TopHandler.lhs
GHC/TypeLits.hs
GHC/Windows.hs
NHC/PosixTypes.hsc
NHC/SizedTypes.hs
Numeric.hs
Prelude.hs
System/CPUTime.hsc
System/Console/GetOpt.hs
System/Environment.hs
System/Environment/ExecutablePath.hsc
System/Exit.hs
System/IO.hs
System/IO/Error.hs
System/Info.hs
System/Mem.hs
System/Mem/StableName.hs
System/Mem/Weak.hs
System/Posix/Types.hs
System/Timeout.hs
Text/Printf.hs

Inspired by a similar graph by Herbert Valerio Riedel, I tried to
visualize the current state and came up with this:
https://github.com/nomeata/packages-base/blob/base-split/graph.pdf?raw=true

It is not completely accurate due to Prelude not included in
-ddump-minimal-imports (but that shouldn’t matter as most interesting
functions of the Prelude are in base-pure). The script to
generate the dot file from *.imports is included in the branch at
https://github.com/nomeata/packages-base/tree/base-split


Next I’d need to see how entangled the system-close stuff is (File IO,
concurrency, GHC.Event.*).


Of course with too much splitting one runs in the Bane of the Orphaned
Instances – neither should base-foreign require base-float nor the other
way around, but "Storable Double" needs to be define somewhere... And
the same question will arise if Data.Date should go to a package of its
own.


Also, I notice that there is an issue with “internal” modules (mostly
GHC.something) that should not be part of some stable API, but needed to
implement packages further down. Should they just not be considered part
of the “public” (and PVP-relevant) API? Or should there be two packages,
e.g. base-pure-internal and base-pure, where the latter re-exports those
modules that are meant for public consumption?


So, what is the general opinion? Is this a way worth pursuing? Or are we
fine with the huge base and I can do different things again ;-)?

Greetings,
Joachim


--
Joachim "nomeata" Breitner
Debian Developer
  [hidden email] | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: [hidden email] | http://people.debian.org/~nomeata


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: base package

Johan Tibell-2
Hi Joachim.

Glad to see you're making progress on this. Once we're done exploring how fine-grained we can make the division we might want to pull back a bit and consider what logical groupings makes sense. For example, even if the float functionality can be split from the int functionality, I don't think that makes for a very logical grouping.

In addition, I don't think we want to say that e.g. pure data structures can't depend on the FFI. While their current implementation might not use the FFI, what if we want to use it in the future. We'd have to reshuffle the packages again.

Just my 2 cents.

-- Johan


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package

Joachim Breitner-2
Hi,

Am Freitag, den 22.02.2013, 11:38 -0800 schrieb Johan Tibell:
> In addition, I don't think we want to say that e.g. pure data
> structures can't depend on the FFI. While their current implementation
> might not use the FFI, what if we want to use it in the future. We'd
> have to reshuffle the packages again.

right, there is a tension between having just independent APIs and
having also independent implementations. My main goal is to allow
packages to specify their imports more precisely, to require less
changes as not-so-common stuff in base evolves and to make it easier for
alternative compiler/targets to implement parts of base; this would just
require providing better grouped APIs.

But if we want that while retaining the freedom to have an entangled
implementation, we are back at the "large base + specific re-exporting
packages" approach, which wasn’t particularly well received here.

Greetings,
Joachim

PS: Even with the currently explored split stuff in base-pure can use
the FFI; it could just not use the modules from the Foreign.* structure.
This may or may not be a problem. It was for the GHC.Fingeprint
implementation, as it was marshalling arrays.

--
Joachim "nomeata" Breitner
Debian Developer
  [hidden email] | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: [hidden email] | http://people.debian.org/~nomeata


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: base package

Johan Tibell-2
On Fri, Feb 22, 2013 at 3:34 PM, Joachim Breitner <[hidden email]> wrote:
right, there is a tension between having just independent APIs and
having also independent implementations. My main goal is to allow
packages to specify their imports more precisely, to require less
changes as not-so-common stuff in base evolves and to make it easier for
alternative compiler/targets to implement parts of base; this would just
require providing better grouped APIs.

But if we want that while retaining the freedom to have an entangled
implementation, we are back at the "large base + specific re-exporting
packages" approach, which wasn’t particularly well received here.

I don't know about entangled implementations. But I'd like to have a base package (e.g. your base-pure) that has a consistent set of basic data types e.g. Int, Word, Float, Double, Char, String, ByteString, Text, [a], Maybe, Either, and so forth. These are logically at the same layer. I think splitting them according to how they happen to be implemented at the moment is a misstake. It would give us a illogical and unstable layering in the long run.

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package

Ian Lynagh-2
In reply to this post by Joachim Breitner-2
On Fri, Feb 22, 2013 at 07:52:00PM +0100, Joachim Breitner wrote:
>
> Of course with too much splitting one runs in the Bane of the Orphaned
> Instances – neither should base-foreign require base-float nor the other
> way around, but "Storable Double" needs to be define somewhere...

This is no different than the question of whether the instance should be
in Foreign.Storable or GHC.Types. One has to import/depend-on the other,
and it doesn't matter which: it's an implementation issue, and doesn't
affect people importing/depending-on the modules/packages.

In this case, GHC.Types.Double is in ghc-prim, so you will presumably
need to leave the instance in Foreign.Storable in base-foreign.

> Also, I notice that there is an issue with “internal” modules (mostly
> GHC.something) that should not be part of some stable API, but needed to
> implement packages further down. Should they just not be considered part
> of the “public” (and PVP-relevant) API? Or should there be two packages,
> e.g. base-pure-internal and base-pure, where the latter re-exports those
> modules that are meant for public consumption?

If it's easy to split out the GHC.* modules, then that's probably
better. If not (e.g. because Public.A imports GHC.B, which import
Public.C, which imports GHC.D) then it's probably not worth the bother.

> So, what is the general opinion?

Looks good to me!


Thanks
Ian


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package

Ian Lynagh-2
In reply to this post by Johan Tibell-2
On Fri, Feb 22, 2013 at 11:38:04AM -0800, Johan Tibell wrote:
>
> Glad to see you're making progress on this. Once we're done exploring how
> fine-grained we can make the division we might want to pull back a bit

I definitely agree with "Once we're done". Once we have made all the
splits we might want to make, it'll be a lot easier to see the big
picture and merge packages we decide should be merged. It's a lot harder
to work the other way, as it's tricky to see what is or isn't possible.

> In addition, I don't think we want to say that e.g. pure data structures
> can't depend on the FFI. While their current implementation might not use
> the FFI, what if we want to use it in the future. We'd have to reshuffle
> the packages again.

I think the issue is what a package exports, rather than what it depends
on. Changing the package dependencies later won't affect users of the
package.


Thanks
Ian


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

RE: base package

Simon Peyton Jones
In reply to this post by Johan Tibell-2

It’s great work, thanks Joachim. 

 

I’d like to be very clear about goals, though.  I have not been following this thread closely enough, but is there a Wiki page that explains what the goals of the base-package break-up is? 

 

I believe that the driving goal is:

·        to allow changes to internals without forcing a version-bump on ‘base’, on which every package depends

But that goal needs a bit of unpacking. Suppose we divided base into six, base1, base2, base3, etc, but each was a vertical silo and every other package depended on all six.  Then nothing would be gained; bumping any of them would cause a ripple of bumps down the line.

 

Alternatively, suppose we split it into just two: ‘base’ and ‘base-internal’, where the former is just a wrapper around the latter, but presenting a stable API.  Now you’d get the stability you want.

 

I’m sure this is a vastly simplistic analysis, and those of you who have been thinking hard about this can say it more precisely than I, based on practical experience.  But I would find it very helpful to have a crystal-clear articulation of the actual goals, with concrete examples, to help orient the debate, and guide choices.

 

I assume that a non-goal is

·        split base into as many packages as possible.

though a superficial reading of the thread might suggest just that.  Indeed other things being equal, a goal should be

·        split base into as FEW packages as possible, consistent with meeting the other goals

As Johan points out, a split now could paint us into a corner later, so we should not gratuitously split things up.

 

Many thanks for working on this.

 

Simon

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Johan Tibell
Sent: 22 February 2013 19:38
To: Joachim Breitner
Cc: [hidden email]
Subject: Re: base package

 

Hi Joachim.

 

Glad to see you're making progress on this. Once we're done exploring how fine-grained we can make the division we might want to pull back a bit and consider what logical groupings makes sense. For example, even if the float functionality can be split from the int functionality, I don't think that makes for a very logical grouping.

 

In addition, I don't think we want to say that e.g. pure data structures can't depend on the FFI. While their current implementation might not use the FFI, what if we want to use it in the future. We'd have to reshuffle the packages again.

 

Just my 2 cents.

 

-- Johan

 


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package

Roman Cheplyaka-2
* Simon Peyton-Jones <[hidden email]> [2013-02-23 10:27:46+0000]
> I believe that the driving goal is:
>
> *        to allow changes to internals without forcing a version-bump
> on 'base', on which every package depends

This is a good goal.

Another goal could be to make the packages more meaningful.

I think this goal is good by itself (judging from common engineering
principles), and also would make it easier to share the code between
different implementations. (Right now JHC has its own version of many
base modules, because it cannot reuse base.)

What is common between the following modules?

  Control.Applicative
  Control.Concurrent
  Data.Foldable
  Data.STRef
  Foreign.C
  GHC.Exts
  System.Environment
  Text.ParserCombinators.ReadP

If we ignore for a second historical incidents and tricky
inter-dependencies, I think we'd all agree that by today's standard
they all logically belong to different packages. In other words, if
they didn't exist today, and a package came out that contained two or
more of the above modules, it would look really strange.

Roman

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package -- goals

Joachim Breitner-2
In reply to this post by Simon Peyton Jones
Hi,

Am Samstag, den 23.02.2013, 10:27 +0000 schrieb Simon Peyton-Jones:
> I’d like to be very clear about goals, though.  I have not been
> following this thread closely enough, but is there a Wiki page that
> explains what the goals of the base-package break-up is?  

I added a Goals section to
http://hackage.haskell.org/trac/ghc/wiki/SplitBase
 
and besides

> I believe that the driving goal is:
>
> ·       to allow changes to internals without forcing a version-bump
> on ‘base’, on which every package depends

added these two goals, which I find worthwhile to pursue:

To allow packages to be explictly about what they need

        A library that does not use the IO monad could communicate that
        just by not depending on some base-io package. Similar with the
        Foreign Function Interface or unsafe operations.
       
To allow alternative implementations/targets

        A Haskell-to-Javascript compiler will not support File IO, or
        maybe not even IO at all. It would be desirable such an
        implementation has a chance to at least provide a complete and
        API compatible base-pure package, and that one can hence
        reasonably assume that packages and libraries depending only on
        base-pure will indeed work without modification. This might be
        subsumed by fulfilling the previous goal.
       

Just by looking at the goals, the variant with a big base package that
uses all kinds of “uglyness” (FFI for pure stuff, cyclic dependencies
between API-wise unrelated stuff, lots of GHC internals used) and
re-exporting packages that have a more specific and possibly more stable
API sounds like it can provide the mentioned goals.

Iain commented this idea earlier this thread¹ with three points:

> * No-one would use the new packages unless they come with GHC;
>   e.g. not a perfect analogy, but compare the number of rev-deps
>   according to http://packdeps.haskellers.com/reverse of the various
>   *prelude* packages vs base:
>       4831 base
>          6 basic-prelude
>          8 classy-prelude
>          4 classy-prelude-conduit
>          2 custom-prelude
>          1 general-prelude
>          1 modular-prelude
>         17 numeric-prelude
>          2 prelude-extras
Hopefully the problem here (often-changing base) is big enough and the
alternative (more purpose-oriented and more stable) packages are
attractive enough to make people use the new set.

> * If it comes with GHC, it would mean us permanently maintaining the two
>   levels

True. I’m not convinced that it will be too much a burden, at least if
the reexporting packages do that on the module level.

> * base would still be an opaque blob, with too many modules and cyclic
>   imports, which makes development tricky

Does it really make development tricky? I’d rather expect it to make
development easier, because you _can_ mix, say, IO and Foreign stuff
easily and even use some of that in lower levels. If it were less tricky
to separate it, then my experiment at
https://github.com/nomeata/packages-base/tree/base-split would have been
less work...


In any case there is still the problem: What and where is the Prelude...
but maybe let’s postpone this.

Greetings,
Joachim

¹ http://www.haskell.org/pipermail/glasgow-haskell-users/2013-February/023774.html


--
Joachim "nomeata" Breitner
Debian Developer
  [hidden email] | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: [hidden email] | http://people.debian.org/~nomeata


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: base package -- goals

Simon Peyton Jones
| I added a Goals section to
| http://hackage.haskell.org/trac/ghc/wiki/SplitBase

Thanks.  But the first goal, which is the dominant one, is very unclear to me as my comments mentioned.  A description of what the problem is, and why a simple "API wrapper" approach would not solve it, would be useful.

SImon


| -----Original Message-----
| From: [hidden email] [mailto:glasgow-haskell-
| [hidden email]] On Behalf Of Joachim Breitner
| Sent: 25 February 2013 13:32
| To: [hidden email]
| Subject: Re: base package -- goals
|
| Hi,
|
| Am Samstag, den 23.02.2013, 10:27 +0000 schrieb Simon Peyton-Jones:
| > I’d like to be very clear about goals, though.  I have not been
| > following this thread closely enough, but is there a Wiki page that
| > explains what the goals of the base-package break-up is?
|
| I added a Goals section to
| http://hackage.haskell.org/trac/ghc/wiki/SplitBase
|
| and besides
|
| > I believe that the driving goal is:
| >
| > ·       to allow changes to internals without forcing a version-bump
| > on ‘base’, on which every package depends
|
| added these two goals, which I find worthwhile to pursue:
|
| To allow packages to be explictly about what they need
|
|         A library that does not use the IO monad could communicate that
|         just by not depending on some base-io package. Similar with the
|         Foreign Function Interface or unsafe operations.
|
| To allow alternative implementations/targets
|
|         A Haskell-to-Javascript compiler will not support File IO, or
|         maybe not even IO at all. It would be desirable such an
|         implementation has a chance to at least provide a complete and
|         API compatible base-pure package, and that one can hence
|         reasonably assume that packages and libraries depending only on
|         base-pure will indeed work without modification. This might be
|         subsumed by fulfilling the previous goal.
|
|
| Just by looking at the goals, the variant with a big base package that
| uses all kinds of “uglyness” (FFI for pure stuff, cyclic dependencies
| between API-wise unrelated stuff, lots of GHC internals used) and re-
| exporting packages that have a more specific and possibly more stable
| API sounds like it can provide the mentioned goals.
|
| Iain commented this idea earlier this thread¹ with three points:
|
| > * No-one would use the new packages unless they come with GHC;
| >   e.g. not a perfect analogy, but compare the number of rev-deps
| >   according to http://packdeps.haskellers.com/reverse of the various
| >   *prelude* packages vs base:
| >       4831 base
| >          6 basic-prelude
| >          8 classy-prelude
| >          4 classy-prelude-conduit
| >          2 custom-prelude
| >          1 general-prelude
| >          1 modular-prelude
| >         17 numeric-prelude
| >          2 prelude-extras
|
| Hopefully the problem here (often-changing base) is big enough and the
| alternative (more purpose-oriented and more stable) packages are
| attractive enough to make people use the new set.
|
| > * If it comes with GHC, it would mean us permanently maintaining the
| two
| >   levels
|
| True. I’m not convinced that it will be too much a burden, at least if
| the reexporting packages do that on the module level.
|
| > * base would still be an opaque blob, with too many modules and cyclic
| >   imports, which makes development tricky
|
| Does it really make development tricky? I’d rather expect it to make
| development easier, because you _can_ mix, say, IO and Foreign stuff
| easily and even use some of that in lower levels. If it were less tricky
| to separate it, then my experiment at
| https://github.com/nomeata/packages-base/tree/base-split would have been
| less work...
|
|
| In any case there is still the problem: What and where is the Prelude...
| but maybe let’s postpone this.
|
| Greetings,
| Joachim
|
| ¹ http://www.haskell.org/pipermail/glasgow-haskell-users/2013-
| February/023774.html
|
|
| --
| Joachim "nomeata" Breitner
| Debian Developer
|   [hidden email] | ICQ# 74513189 | GPG-Keyid: 4743206C
|   JID: [hidden email] | http://people.debian.org/~nomeata

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package -- goals

Ian Lynagh-2
On Mon, Feb 25, 2013 at 02:25:03PM +0000, Simon Peyton-Jones wrote:
> | I added a Goals section to
> | http://hackage.haskell.org/trac/ghc/wiki/SplitBase
>
> Thanks.  But the first goal, which is the dominant one, is very unclear to me as my comments mentioned.  A description of what the problem is, and why a simple "API wrapper" approach would not solve it, would be useful.

On the wiki page you say:

    SPJ: But that goal needs a bit of unpacking. Suppose we divided base
    into six, base1, base2, base3, etc, but each was a vertical silo and
    every other package depended on all six. Then nothing would be gained;
    bumping any of them would cause a ripple of bumps down the line.

but even if we did just divide base up into vertical silos then I don't
think most packages would depend on them all; for example, most packages
would probably not depend on file-io or concurrency.

But in any case, I'd hope we would also make some horizontal cuts, and I
expect very few packages would need to depend on ghc-io-manager etc.


Thanks
Ian


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package -- goals

Ian Lynagh-2
In reply to this post by Joachim Breitner-2
On Mon, Feb 25, 2013 at 02:31:56PM +0100, Joachim Breitner wrote:
>
> Hopefully the problem here (often-changing base) is big enough and the
> alternative (more purpose-oriented and more stable) packages are
> attractive enough to make people use the new set.

I'm pretty confident that most packages won't do more than the minimal
base bumping while importing base continues to work.

> > * base would still be an opaque blob, with too many modules and cyclic
> >   imports, which makes development tricky
>
> Does it really make development tricky? I’d rather expect it to make
> development easier, because you _can_ mix, say, IO and Foreign stuff
> easily and even use some of that in lower levels. If it were less tricky
> to separate it, then my experiment at
> https://github.com/nomeata/packages-base/tree/base-split would have been
> less work...

It's tricky to make changes to the core modules, because that generally
requires changing imports, and it's hard to see how to actually do that
without making an import loop (or without making more import loops than
are necessary).

In general there's actually a fair amount of flexibility in which way
module imports go (e.g. you can move a "Cl Ty" instance from the Cl
module to the Ty module or vice-versa), but in base it's hard to see how
to structure things best: there are approaching 200 modules, half of
which are tied up in a recursive knot, with 13 hs-boot modules (2 of
which import other hs-boot modules).

> In any case there is still the problem: What and where is the Prelude...
> but maybe let’s postpone this.

I'd put it in its own package for now, and then look at whether/what it
should be merged with later.

I'm in 2 minds about it. On the one hand, I'm sure that lots of people
won't like a single-module package that essentially everything depends
on. But on the other hand, Prelude is both magic and broad, so it would
make some sense to have it in its own package.


Thanks
Ian


_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Reply | Threaded
Open this post in threaded view
|

Re: base package -- goals

Stephen Paul Weber
Somebody claiming to be Ian Lynagh wrote:
>On Mon, Feb 25, 2013 at 02:31:56PM +0100, Joachim Breitner wrote:
>> In any case there is still the problem: What and where is the Prelude...
>> but maybe let’s postpone this.
>
>I'd put it in its own package for now, and then look at whether/what it
>should be merged with later.

Why shouldn't Prelude (and other really stable, standard modules) just live
in the `haskell2010` package?

--
Stephen Paul Weber, @singpolyma
See <http://singpolyma.net> for how I prefer to be contacted
edition right joseph

_______________________________________________
Glasgow-haskell-users mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

signature.asc (853 bytes) Download Attachment
1 ... 345678