Proposal: Add a splitBy / splitOn in Data.List

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Proposal: Add a splitBy / splitOn in Data.List

Saurabh Nanda
This has certainly been discussed before. A quick Google search turned up the following past discussions:
Is there anything blocking this discussion & implementation? Anything that can be done to unblock it?

-- Saurabh.


_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Add a splitBy / splitOn in Data.List

Edward Kmett-2
The main thing that prevented it from going into base is the number of subtleties about what precisely it means to properly "split" something.

Most languages make fairly arbitrary calls on topics such as:

* Do you split on list elements (e.g. ',') or list of elements, so you can multi-character delimiters ", "? What about multiple types of thing that are all delimiters, e.g. any whitespace character?
* What do you do with the delimiters?
* What happens with runs of delimiters? 
* What about initial or final runs of delimiters (e.g. leading spaces)?

The end result was that a split package was written by Brent Yorgey back in 2008 or so that rather comprehensively covers the design space, and it was incorporated into the Haskell Platform.

On Thu, Nov 1, 2018 at 1:34 PM Saurabh Nanda <[hidden email]> wrote:
This has certainly been discussed before. A quick Google search turned up the following past discussions:
Is there anything blocking this discussion & implementation? Anything that can be done to unblock it?

-- Saurabh.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Add a splitBy / splitOn in Data.List

Elliot Cameron-2
Despite these subtleties, I must confess I've often wanted to whip up a quick script and been frustrated that these functions are missing from base. For example using Haskell as a sed/awk alternative can be pleasant *if* the functions you need are in base. What's more, in many years I've only really wanted one or two versions of this.

What if we added the most flexible of versions and included only that? This version would accept multicharacter delimiters, always throw them away, and always produce a new entry in the result for every occurrence of the delimiter. If you don't want the empty entries, you can filter. If you don't want leading, you can dropWhile. If you want the delimiters back, you can map. This seems like a nice trade-off for just being available in base.

On Fri, Nov 2, 2018, 1:51 AM Edward Kmett <[hidden email] wrote:
The main thing that prevented it from going into base is the number of subtleties about what precisely it means to properly "split" something.

Most languages make fairly arbitrary calls on topics such as:

* Do you split on list elements (e.g. ',') or list of elements, so you can multi-character delimiters ", "? What about multiple types of thing that are all delimiters, e.g. any whitespace character?
* What do you do with the delimiters?
* What happens with runs of delimiters? 
* What about initial or final runs of delimiters (e.g. leading spaces)?

The end result was that a split package was written by Brent Yorgey back in 2008 or so that rather comprehensively covers the design space, and it was incorporated into the Haskell Platform.

On Thu, Nov 1, 2018 at 1:34 PM Saurabh Nanda <[hidden email]> wrote:
This has certainly been discussed before. A quick Google search turned up the following past discussions:
Is there anything blocking this discussion & implementation? Anything that can be done to unblock it?

-- Saurabh.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Add a splitBy / splitOn in Data.List

Vanessa McHale

cabal now has the ability to be used for [scripting](https://github.com/haskell/cabal/pull/5483#issuecomment-409633079) which I think addresses your use case (at least, it's easier than forking base...).

On 11/2/18 6:38 AM, Elliot Cameron wrote:
Despite these subtleties, I must confess I've often wanted to whip up a quick script and been frustrated that these functions are missing from base. For example using Haskell as a sed/awk alternative can be pleasant *if* the functions you need are in base. What's more, in many years I've only really wanted one or two versions of this.

What if we added the most flexible of versions and included only that? This version would accept multicharacter delimiters, always throw them away, and always produce a new entry in the result for every occurrence of the delimiter. If you don't want the empty entries, you can filter. If you don't want leading, you can dropWhile. If you want the delimiters back, you can map. This seems like a nice trade-off for just being available in base.

On Fri, Nov 2, 2018, 1:51 AM Edward Kmett <[hidden email] wrote:
The main thing that prevented it from going into base is the number of subtleties about what precisely it means to properly "split" something.

Most languages make fairly arbitrary calls on topics such as:

* Do you split on list elements (e.g. ',') or list of elements, so you can multi-character delimiters ", "? What about multiple types of thing that are all delimiters, e.g. any whitespace character?
* What do you do with the delimiters?
* What happens with runs of delimiters? 
* What about initial or final runs of delimiters (e.g. leading spaces)?

The end result was that a split package was written by Brent Yorgey back in 2008 or so that rather comprehensively covers the design space, and it was incorporated into the Haskell Platform.

On Thu, Nov 1, 2018 at 1:34 PM Saurabh Nanda <[hidden email]> wrote:
This has certainly been discussed before. A quick Google search turned up the following past discussions:
Is there anything blocking this discussion & implementation? Anything that can be done to unblock it?

-- Saurabh.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
--



Vanessa McHale
Functional Compiler Engineer | Chicago, IL

Website: www.iohk.io
Twitter: @vamchale
PGP Key ID: 4209B7B5

Input
          Output

Twitter Github LinkedIn


This e-mail and any file transmitted with it are confidential and intended solely for the use of the recipient(s) to whom it is addressed. Dissemination, distribution, and/or copying of the transmission by anyone other than the intended recipient(s) is prohibited. If you have received this transmission in error please notify IOHK immediately and delete it from your system. E-mail transmissions cannot be guaranteed to be secure or error free. We do not accept liability for any loss, damage, or error arising from this transmission

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Add a splitBy / splitOn in Data.List

Theodore Lief Gannon
In reply to this post by Elliot Cameron-2
If you accept more than one delimiter but drop them, you've lost info about which one caused each break and can't map them back. It's more generic to keep them, since you can still filter.

On Fri, Nov 2, 2018, 4:39 AM Elliot Cameron <[hidden email] wrote:
Despite these subtleties, I must confess I've often wanted to whip up a quick script and been frustrated that these functions are missing from base. For example using Haskell as a sed/awk alternative can be pleasant *if* the functions you need are in base. What's more, in many years I've only really wanted one or two versions of this.

What if we added the most flexible of versions and included only that? This version would accept multicharacter delimiters, always throw them away, and always produce a new entry in the result for every occurrence of the delimiter. If you don't want the empty entries, you can filter. If you don't want leading, you can dropWhile. If you want the delimiters back, you can map. This seems like a nice trade-off for just being available in base.

On Fri, Nov 2, 2018, 1:51 AM Edward Kmett <[hidden email] wrote:
The main thing that prevented it from going into base is the number of subtleties about what precisely it means to properly "split" something.

Most languages make fairly arbitrary calls on topics such as:

* Do you split on list elements (e.g. ',') or list of elements, so you can multi-character delimiters ", "? What about multiple types of thing that are all delimiters, e.g. any whitespace character?
* What do you do with the delimiters?
* What happens with runs of delimiters? 
* What about initial or final runs of delimiters (e.g. leading spaces)?

The end result was that a split package was written by Brent Yorgey back in 2008 or so that rather comprehensively covers the design space, and it was incorporated into the Haskell Platform.

On Thu, Nov 1, 2018 at 1:34 PM Saurabh Nanda <[hidden email]> wrote:
This has certainly been discussed before. A quick Google search turned up the following past discussions:
Is there anything blocking this discussion & implementation? Anything that can be done to unblock it?

-- Saurabh.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Add a splitBy / splitOn in Data.List

Elliot Cameron-2
I didn't realize cabal now supported scripting. I suppose that addresses a large number of my use cases for having this.

I didn't mean choosing different delimiters but only a single multielement delimiter, albeit that is also not flexible. If we also had a multicharacter replace function then a single-element split would be more tolerable.

I'm still in favor of providing one or two of the most common, most flexible versions of this just to help newcomers from other languages that have these functions in their standard libraries, but my opinion is not very strongly held.

On Fri, Nov 2, 2018, 8:18 AM Theodore Lief Gannon <[hidden email] wrote:
If you accept more than one delimiter but drop them, you've lost info about which one caused each break and can't map them back. It's more generic to keep them, since you can still filter.

On Fri, Nov 2, 2018, 4:39 AM Elliot Cameron <[hidden email] wrote:
Despite these subtleties, I must confess I've often wanted to whip up a quick script and been frustrated that these functions are missing from base. For example using Haskell as a sed/awk alternative can be pleasant *if* the functions you need are in base. What's more, in many years I've only really wanted one or two versions of this.

What if we added the most flexible of versions and included only that? This version would accept multicharacter delimiters, always throw them away, and always produce a new entry in the result for every occurrence of the delimiter. If you don't want the empty entries, you can filter. If you don't want leading, you can dropWhile. If you want the delimiters back, you can map. This seems like a nice trade-off for just being available in base.

On Fri, Nov 2, 2018, 1:51 AM Edward Kmett <[hidden email] wrote:
The main thing that prevented it from going into base is the number of subtleties about what precisely it means to properly "split" something.

Most languages make fairly arbitrary calls on topics such as:

* Do you split on list elements (e.g. ',') or list of elements, so you can multi-character delimiters ", "? What about multiple types of thing that are all delimiters, e.g. any whitespace character?
* What do you do with the delimiters?
* What happens with runs of delimiters? 
* What about initial or final runs of delimiters (e.g. leading spaces)?

The end result was that a split package was written by Brent Yorgey back in 2008 or so that rather comprehensively covers the design space, and it was incorporated into the Haskell Platform.

On Thu, Nov 1, 2018 at 1:34 PM Saurabh Nanda <[hidden email]> wrote:
This has certainly been discussed before. A quick Google search turned up the following past discussions:
Is there anything blocking this discussion & implementation? Anything that can be done to unblock it?

-- Saurabh.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Add a splitBy / splitOn in Data.List

Dan Burton
What about just adding Data.List.Split to base?

-- Dan Burton


On Fri, Nov 2, 2018 at 8:56 AM Elliot Cameron <[hidden email]> wrote:
I didn't realize cabal now supported scripting. I suppose that addresses a large number of my use cases for having this.

I didn't mean choosing different delimiters but only a single multielement delimiter, albeit that is also not flexible. If we also had a multicharacter replace function then a single-element split would be more tolerable.

I'm still in favor of providing one or two of the most common, most flexible versions of this just to help newcomers from other languages that have these functions in their standard libraries, but my opinion is not very strongly held.

On Fri, Nov 2, 2018, 8:18 AM Theodore Lief Gannon <[hidden email] wrote:
If you accept more than one delimiter but drop them, you've lost info about which one caused each break and can't map them back. It's more generic to keep them, since you can still filter.

On Fri, Nov 2, 2018, 4:39 AM Elliot Cameron <[hidden email] wrote:
Despite these subtleties, I must confess I've often wanted to whip up a quick script and been frustrated that these functions are missing from base. For example using Haskell as a sed/awk alternative can be pleasant *if* the functions you need are in base. What's more, in many years I've only really wanted one or two versions of this.

What if we added the most flexible of versions and included only that? This version would accept multicharacter delimiters, always throw them away, and always produce a new entry in the result for every occurrence of the delimiter. If you don't want the empty entries, you can filter. If you don't want leading, you can dropWhile. If you want the delimiters back, you can map. This seems like a nice trade-off for just being available in base.

On Fri, Nov 2, 2018, 1:51 AM Edward Kmett <[hidden email] wrote:
The main thing that prevented it from going into base is the number of subtleties about what precisely it means to properly "split" something.

Most languages make fairly arbitrary calls on topics such as:

* Do you split on list elements (e.g. ',') or list of elements, so you can multi-character delimiters ", "? What about multiple types of thing that are all delimiters, e.g. any whitespace character?
* What do you do with the delimiters?
* What happens with runs of delimiters? 
* What about initial or final runs of delimiters (e.g. leading spaces)?

The end result was that a split package was written by Brent Yorgey back in 2008 or so that rather comprehensively covers the design space, and it was incorporated into the Haskell Platform.

On Thu, Nov 1, 2018 at 1:34 PM Saurabh Nanda <[hidden email]> wrote:
This has certainly been discussed before. A quick Google search turned up the following past discussions:
Is there anything blocking this discussion & implementation? Anything that can be done to unblock it?

-- Saurabh.

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Add a splitBy / splitOn in Data.List

Henning Thielemann

On Fri, 2 Nov 2018, Dan Burton wrote:

> What about just adding Data.List.Split to base?

... and then splitting 'base'? :-)
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Add a splitBy / splitOn in Data.List

Elliot Cameron-2
Ah in the context of splitting base this seems like a backward move. The solution must really be to have tooling that can pull in libraries with minimal friction.

On Fri, Nov 2, 2018 at 10:47 AM Henning Thielemann <[hidden email]> wrote:

On Fri, 2 Nov 2018, Dan Burton wrote:

> What about just adding Data.List.Split to base?

... and then splitting 'base'? :-)

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Add a splitBy / splitOn in Data.List

Dan Burton
If and when base is split, then just include Data.List.Split with whatever package all the other List stuff gets put in. My point is, this module should live in the same package where the other list functions live.

I'm in favor of splitting base, but things should not be so broken up to the extreme of having a package just for left-pad. It is possible to find middle ground.

-- Dan Burton


On Fri, Nov 2, 2018 at 10:49 AM Elliot Cameron <[hidden email]> wrote:
Ah in the context of splitting base this seems like a backward move. The solution must really be to have tooling that can pull in libraries with minimal friction.

On Fri, Nov 2, 2018 at 10:47 AM Henning Thielemann <[hidden email]> wrote:

On Fri, 2 Nov 2018, Dan Burton wrote:

> What about just adding Data.List.Split to base?

... and then splitting 'base'? :-)

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Add a splitBy / splitOn in Data.List

Bryan Richter-2
+1 to adding a single function that splits a list by a multi-element delimiter, e.g. the hypothetical

>>> *Data.List.split [a, b] [c, a, b, d, a, b, e, a]
[[c], [d], [e, a]]

The split package seems to heavyweight for base (I know I'd always have to look up the differences between splitOn, split, chop, and divvy), and more sophisticated needs should probably be filled by a special-purpose parser.

I would even say it might make sense to just restrict the function to Strings, unless there is widespread need for supporting Lists in general.

On Fri, Nov 2, 2018 at 4:42 PM Dan Burton <[hidden email]> wrote:
If and when base is split, then just include Data.List.Split with whatever package all the other List stuff gets put in. My point is, this module should live in the same package where the other list functions live.

I'm in favor of splitting base, but things should not be so broken up to the extreme of having a package just for left-pad. It is possible to find middle ground.

-- Dan Burton


On Fri, Nov 2, 2018 at 10:49 AM Elliot Cameron <[hidden email]> wrote:
Ah in the context of splitting base this seems like a backward move. The solution must really be to have tooling that can pull in libraries with minimal friction.

On Fri, Nov 2, 2018 at 10:47 AM Henning Thielemann <[hidden email]> wrote:

On Fri, 2 Nov 2018, Dan Burton wrote:

> What about just adding Data.List.Split to base?

... and then splitting 'base'? :-)
_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
[hidden email]
http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries