unix package

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

unix package

Ian Lynagh

Hi all,

Currently the unix package is a mishmash of different solutions to the
portability problem. Here are all the ones I've found, along with what I
think the best action would be. If this sounds reasonable then I'll
start making patches and proposals as appropriate.

(for POSIX/SUS, see http://www.opengroup.org/onlinepubs/009695399/)

1) Values outside of the scope of the package, conditionally available:
   * System.Posix.Signals.Exts.*

Propose: Moving modules into a new package, unix-exts perhaps.

2) Things that fall back to probably-reasonable alternatives:
   * unsetEnv, setEnv
   * PATH_MAX

Propose: Leave as they are.

3) Things falling back to functionality which can cause problems:
   * The Haskell usleep uses the C usleep (marked Obsolescent) if
     nanosleep (a Timer extension) isn't available. C's usleep interacts
     badly with GHC's threaded RTS.

Propose: Leave as it is.

4) Things that raise errors at runtime if they are not available,
   but there is something we could fall back to:
   * getGroupEntryForID (getgrgid_r, thread-safe extension)
   * getGroupEntryForName (getgrnam_r, thread-safe extension)

Propose: Write fallback code.

5) Things that raise errors at runtime if they are not available,
   but as far as I can see are required to be available anyway:
   * _PC_* in System/Posix/Files.hsc
   * getUserEntryForID (getpwuid)
   * getUserEntryForName (getpwnam)
   * grBufSize (sysconf, _SC_GETGR_R_SIZE_MAX)
   * pwBufSize (sysconf, _SC_GETPW_R_SIZE_MAX)
   * sysconfWithDefault (sysconf)

Propose: Change to using them unconditionally and see if any platforms
         break.

6) Things depending on optional functionality, conditionally available:
   * setSymbolicLinkOwnerAndGroup depends on lchown, an XSI extension
     http://www.opengroup.org/onlinepubs/009695399/functions/lchown.html
   * RLIMIT_AS (ResourceTotalMemory), RLIM_SAVED_MAX, RLIM_SAVED_CUR
     (all of the rlimit stuff actually appears to be an XSI extension).

Propose: Move to unix-xsi package.

7) Things that raise errors at runtime if they are not available:
   * getAllGroupEntries (getgrent, XSI extension)
   * getAllUserEntries (getpwent, setpwent, endpwent, XSI extensions)
   * Various in DynamicLinker.Prim

Propose: Move to unix-xsi package.

(for both 6 and 7, an alternative would be to just make them
unconditionally used in the unix package; I'm not sure what platforms,
if any, this would break on).

8) Things that are implemented only for certain compilers:
   * forkProcess (GHC-only)

Propose: Write implementations for other compilers/interpreters.


Thanks
Ian

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

RE: unix package

Bayley, Alistair-2
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Ian Lynagh
>
> Currently the unix package is a mishmash of different solutions to the
> portability problem. Here are all the ones I've found, along
> with what I
> think the best action would be. If this sounds reasonable then I'll
> start making patches and proposals as appropriate.


This is outside the scope of Ian's proposal, but I was wondering...

The unix package isn't available to Windows users. Unsurprising, maybe.
But, given that it seems to be a layer over the Posix API, would it be
reasonable to request that the unix package is made available on
Windows? There is a Posix API in Windows (NT/2000/XP), although I
understand it is next to useless. There is also this:

http://sourceforge.net/projects/pw32/

and possibly some other projects which provide the necessary
functionality on Windows (cygwin? MSYS?).

Alistair
*****************************************************************
Confidentiality Note: The information contained in this message,
and any attachments, may contain confidential and/or privileged
material. It is intended solely for the person(s) or entity to
which it is addressed. Any review, retransmission, dissemination,
or taking of any action in reliance upon this information by
persons or entities other than the intended recipient(s) is
prohibited. If you received this in error, please contact the
sender and delete the material from any computer.
*****************************************************************
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: unix package

Bryan O'Sullivan
Bayley, Alistair wrote:

> The unix package isn't available to Windows users. Unsurprising, maybe.
> But, given that it seems to be a layer over the Posix API, would it be
> reasonable to request that the unix package is made available on
> Windows?

I've been thinking about just this, too.

The underlying problem is that Haskell's portable I/O-related libraries
are underpowered, particularly the stuff in System.Directory.  Try doing
any significant filesystem-related work using only the portable APIs and
you'll see what I mean.

A portable library that put a POSIX-like wrapper atop filesystem
operations would be a big win, particularly if it were distributed as a
standard library.  This is the approach that Python's standard libraries
take, and it works well.

If I had a vote on how Ian might spend his time, it would be on
something like this, which would have direct value to programmers by
addressing a significant deficit of the current standard libraries.
Splitting up the unix package seems more like an inconsequential
janitorial task.  Given finite resources, I'd prefer to tackle the task
with greater benefits.

        <b
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: unix package

Bjorn Bringert-2
On Jun 28, 2007, at 18:25 , Bryan O'Sullivan wrote:

> Bayley, Alistair wrote:
>
>> The unix package isn't available to Windows users. Unsurprising,  
>> maybe.
>> But, given that it seems to be a layer over the Posix API, would  
>> it be
>> reasonable to request that the unix package is made available on
>> Windows?
>
> I've been thinking about just this, too.
>
> The underlying problem is that Haskell's portable I/O-related  
> libraries are underpowered, particularly the stuff in  
> System.Directory.  Try doing any significant filesystem-related  
> work using only the portable APIs and you'll see what I mean.
>
> A portable library that put a POSIX-like wrapper atop filesystem  
> operations would be a big win, particularly if it were distributed  
> as a standard library.  This is the approach that Python's standard  
> libraries take, and it works well.
>
> If I had a vote on how Ian might spend his time, it would be on  
> something like this, which would have direct value to programmers  
> by addressing a significant deficit of the current standard  
> libraries. Splitting up the unix package seems more like an  
> inconsequential janitorial task.  Given finite resources, I'd  
> prefer to tackle the task with greater benefits.

I started working on this for implementing the file operations needed  
by the tar package, but didn't take it further than that. Here's what  
I have so far:

http://www.cs.chalmers.se/~bringert/darcs/unix-compat/

/Björn


_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: unix package

Duncan Coutts
In reply to this post by Bryan O'Sullivan
On Thu, 2007-06-28 at 09:25 -0700, Bryan O'Sullivan wrote:

> Bayley, Alistair wrote:
>
> > The unix package isn't available to Windows users. Unsurprising, maybe.
> > But, given that it seems to be a layer over the Posix API, would it be
> > reasonable to request that the unix package is made available on
> > Windows?
>
> I've been thinking about just this, too.
>
> The underlying problem is that Haskell's portable I/O-related libraries
> are underpowered, particularly the stuff in System.Directory.  Try doing
> any significant filesystem-related work using only the portable APIs and
> you'll see what I mean.

Yes. I've got a little Gtk2Hs demo that shows you a directory tree with
file names and file sizes. With the portable Haskell directory stuff I
can't even get the file size, I'd have to open every file and use
hFileSize.

> A portable library that put a POSIX-like wrapper atop filesystem
> operations would be a big win, particularly if it were distributed as a
> standard library.  This is the approach that Python's standard libraries
> take, and it works well.

Yes, we know that both POSIX and Win32 have operations to 'stat' a file
and get back some common attributes about it. Of course the intersection
of those attributes is not huge, but it at least contains the file size!

> If I had a vote on how Ian might spend his time, it would be on
> something like this, which would have direct value to programmers by
> addressing a significant deficit of the current standard libraries.
> Splitting up the unix package seems more like an inconsequential
> janitorial task.  Given finite resources, I'd prefer to tackle the task
> with greater benefits.

Though it's also probably more work :-)

It seems to me, one of our main problems is that the current base
package cannot depend on the posix/unix or Win32 packages and so we
cannot easily implement the portable Haskell APIs on top of the existing
posix/win32 api bindings. All the portable stuff is currently
implemented on top of C wrappers which interally use cppery to call
posix or win32 functions. So fewer people feel able to contribute
because most of the code is not Haskell.

Perhaps with splitting the base package up we have the opportunity to
have the directory & IO stuff live in a package that can depend on
either the posix/unix or win32 packages, and implement the portable IO
stuff on top of those (probably still using cpp).

Duncan

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: unix package

Ian Lynagh
In reply to this post by Bryan O'Sullivan
On Thu, Jun 28, 2007 at 09:25:44AM -0700, Bryan O'Sullivan wrote:
>
> Splitting up the unix package seems more like an inconsequential
> janitorial task.

A lot of what I do (bug triaging, approving mailing list posts, ...) is
janitorial, because it is those tasks that people don't tend to want to
do in their evenings, can't do for SoC projects, and so on.

And for what it's worth, in my opinion the task is important; there is
quite a long road to having multiple (modern) Haskell implementations,
with the same libraries, providing the same interfaces and coming with
regression and performance testsuites, but we are making progress, and
this represents one more step along the road.


Writing a windows-posix package would also be useful, but would probably
be better done by someone with a task they want it for, who is motivated
to just sit down for a weekend or two and get it written.

If I were to do it then it would keep getting put aside, as things like
the ongoing janitorial tasks above need tending to, and bugs holding up
the 6.8 release need fixing, and so on, so it would take much longer to
materialise.

Also, this is only one of dozens of libraries that it would be great to
have. We need a similar number of people, each with their own itch that
needs scratching!


Thanks
Ian

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: unix package

Esa Ilari Vuokko
In reply to this post by Duncan Coutts
On 6/30/07, Duncan Coutts <[hidden email]> wrote:
> > operations would be a big win, particularly if it were distributed as a
> > standard library.  This is the approach that Python's standard libraries
> > take, and it works well.
>
> Yes, we know that both POSIX and Win32 have operations to 'stat' a file
> and get back some common attributes about it. Of course the intersection
> of those attributes is not huge, but it at least contains the file size!

I can't help but to think this is exactly the way I don't want to see portable
file IO api.  I am fairly tired of Windows applications that misuse Windows
file rights, and wierd software that got ported from unixy-type systems that
interacts very badly because of misuse of file locking.  I don't think
difference
in APIs is trivial.  File locking and access rights work very diffrently.

> > If I had a vote on how Ian might spend his time, it would be on
> > something like this, which would have direct value to programmers by
> > addressing a significant deficit of the current standard libraries.
> > Splitting up the unix package seems more like an inconsequential
> > janitorial task.  Given finite resources, I'd prefer to tackle the task
> > with greater benefits.
>
> Though it's also probably more work :-)
>
> It seems to me, one of our main problems is that the current base
> package cannot depend on the posix/unix or Win32 packages and so we
> cannot easily implement the portable Haskell APIs on top of the existing
> posix/win32 api bindings. All the portable stuff is currently
> implemented on top of C wrappers which interally use cppery to call
> posix or win32 functions. So fewer people feel able to contribute
> because most of the code is not Haskell.

Indeed.  Also, as Win32 maintainer, I am unsure how far we are supporting
old Windowses.  When I tentavily asked some people about it, they thought
we should continue to support old Windows versions.  Unfortunately file
rights (in practice all of Security-API) is not available under Win 9x and Me.

(But I might have broken somethin anyway, I recall ghc installer having some
changes to Win32 to support old Windows versions.)

> Perhaps with splitting the base package up we have the opportunity to
> have the directory & IO stuff live in a package that can depend on
> either the posix/unix or win32 packages, and implement the portable IO
> stuff on top of those (probably still using cpp).

I hope portable IO means something that still allows access to important
details, and promotes good style.

Best regards,
Esa Ilari Vuokko
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: unix package

Ian Lynagh
On Sat, Jun 30, 2007 at 08:01:09PM +0300, Esa Ilari Vuokko wrote:
>
> Indeed.  Also, as Win32 maintainer, I am unsure how far we are supporting
> old Windowses.  When I tentavily asked some people about it, they thought
> we should continue to support old Windows versions.

Our (GHC) plan is not to support Win 9x (and Me?) from 6.8 onwards, as
we don't have machines on which test it or debug problems. I don't think
the installers that we make now install a GHC that works on Win 9x, but
I could be wrong.


Thanks
Ian

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re[2]: unix package

Bulat Ziganshin-2
In reply to this post by Duncan Coutts
Hello Duncan,

Saturday, June 30, 2007, 8:17:05 PM, you wrote:

> Yes. I've got a little Gtk2Hs demo that shows you a directory tree with
> file names and file sizes. With the portable Haskell directory stuff I
> can't even get the file size, I'd have to open every file and use
> hFileSize.

getFileSize isn't any harder to implement than getModificationTime,
the only minoor problem is that function used (lstat) will truncate
file size to low 32 bits, for full 64-bit value non-portable lstati64
should be used instead (and another type for C-level filesize too)

> Perhaps with splitting the base package up we have the opportunity to
> have the directory & IO stuff live in a package that can depend on
> either the posix/unix or win32 packages, and implement the portable IO
> stuff on top of those (probably still using cpp).

anyway file operations best should be split away from base. the
only problem is that Handle type used in definition of Exception which
is used in definition of IO monad which is used in Prelude (and some
Prelude functions need file i/o by itself)

i once proposed plan of implementing stack of file system/io libraries
http://haskell.org/haskellwiki/Library/IO , also portable base for file i/o
and memory-mapped files i/o implemented in Streams library: see
System.FD and System.MMFile modules in
http://www.haskell.org/library/StreamsBeta.tar.gz 


--
Best regards,
 Bulat                            mailto:[hidden email]

_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: unix package

Isaac Dupree
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bulat Ziganshin wrote:
> anyway file operations best should be split away from base. the
> only problem is that Handle type used in definition of Exception which
> is used in definition of IO monad which is used in Prelude (and some
> Prelude functions need file i/o by itself)

Maybe we will end up having to move Prelude out of base (and all base
modules import explicitly what they want; we could have a small prelude
for them if there are some things that can only be gotten from the
Prelude ( (.), ($), seq... ?), or make real modules that export those.)

And there is a long-term plan for fixing Exception to be an extensible
single-inheritance hierarchy ... maybe ghc 6.10 ... at which point
low-level refactorings will be MUCH more possible (I tried once...
Exception was one problem, avoiding orphan instances, another...)

Isaac
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGh7TFHgcxvIWYTTURAomPAKCxLrNULfBLMWwcWmJRoVqVKiA9/ACaA8vz
ZOwtpjK7Zasrvflfwwkq+aM=
=H2Uc
-----END PGP SIGNATURE-----
_______________________________________________
Libraries mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/libraries