GSoC: Hackage 2.0

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

GSoC: Hackage 2.0

Matthew Gruen
Hi Haskellers,

I'm Matt Gruen (Gracenotes in #haskell), and the Hackage 2.0 SoC
project at <http://hackage.haskell.org/trac/summer-of-code/ticket/1587>
really piqued my interest. It seems doable, in a summer, to make the
new hackage-server more-than-deployment-ready as well as clearing out
some items in the hackage bug tracker[0]; so, I've been working on a
proposal. In this email I'd like to consolidate my mental notes for
haskell-cafe and formulate a roadmap towards a more social Hackage.

The most vital part is getting hackage-server
<http://code.haskell.org/hackage-server/> to a state where it can be
switched in place of hackage-scripts
<http://darcs.haskell.org/hackage-scripts/>, and doing it properly,
organizing the code so it can be extended painlessly in the future.
Duncan Coutts, Antoine Latter, and others have done some great work on
it in the past few years. I've been using Haskell for 1.33 of those
years. I think I could become fluent in the codebases after a week of
dedicated study, although even the hackage-server and hackage-scripts
repositories don't include some components such as the current build
report script.

For putting the 2.0 in Hackage 2.0, any interface changes should help
the library users and the library writers/uploaders without hurting
either of them. Hackage should contain more of the right kind of
information. Statistics help everyone, and they're a pretty good gauge
on the direction of Hackage as a whole. Package popularity contents
are one form of this. Reverse dependencies and even dependency
graphs[1] are great, if I can integrate and expand Roel van Dijk's
work[2].

There should also be some space on package pages, or on pages a link
away from them, for users to contribute information and suggestions.
Coders can explain why or why not the package met their needs, as a
sort of informal bug/enhancement tracking service. Another helpful
flavor of information is package relationships beyond dependencies:
'Deprecated in favor of Foo', 'a fork of Foo', or 'does blah instead
of Foo's plugh functionality'. This will help people find the 'right'
package among look-alikes. There could also be links to illustrative
usage examples for getting started. Happstack's state management will
be a good match for updating and serving this information in real
time.

There's also a need for a more interactive form of package
documentation, but this should strengthen relationships with existing
tools like Haddock and Cabal, not bypass the tools. For example,
adding a changelog[3] or making Haddock's declaration-by-declaration
commentary more wiki-like[4]. Changelogs seem to be within the scope
of Hackage 2.0, integrating with Cabal; Haddock wikification might not
be, perhaps deserving a separate student-summer session of its own.
These can improve the package page and documentation subtrees.

More generally, how can library users find the package they want?
Categories themselves are great, but a tag system could identify and
group specific package functionality. There could be sorting by
ratings and reviews (4/5 lambdas!). Metadata searches, like those
Sascha Böhme implemented in SoC 2007[5], could be integrated. It's not
always obvious which ideas will help and which won't see good returns,
which makes it all the more important to bring hackage-server to a
state where future extensions can be easily written, submitted and
deployed. That's the goal here.

On the technical side, I realize I'd need to spend a not-insignificant
amount of time on a user account system, dealing with authentication
and related issues. One additional bit of functionality to manage is
the hackage build system, which is used to ensure that packages build
and to generate documentation. When building depends on FFI or
OS-specific bindings, specific versions of other packages, compiler
choice or compiler version choice, including language extensions, this
is not trivial. One of two good routes is running cabal server-side to
generate build reports and alleviate the miscompiling a smidge. Given
that cabal install dependency calculation seems to be up in the air
still, the current Setup-running script might end up staying for the
moment, with some basic integrity checking (no cycles or other
impossible dependency scenarios). The other not-mutually-exclusive
approach is accepting failed build reports from users as a web
service[6] to generate a matrix of the platforms that seem to
encounter the most trouble.

Ultimately, it's important that I, or whoever ends up doing the
project, plans for it to benefit you guys. What do you think?

—Matt Gruen

PS: how do you feel about augmenting the .cabal format to provide more
information for Hackage?

[0] Active Tickets by Component, scroll down to HackageDB.
http://hackage.haskell.org/trac/hackage/report/9?USER=anonymous&page=3
[1] Re: ANN: Reverse Dependencies in Hackage (demo).
http://www.haskell.org/pipermail/haskell-cafe/2009-October/067847.html
[2] Show reverse dependencies. Includes a patch for hackage-scripts.
http://hackage.haskell.org/trac/hackage/ticket/576
[3] Add changelog feature to hackagedb.
http://hackage.haskell.org/trac/hackage/ticket/299
[4] Reddit comment. The other comments are good too.
http://www.reddit.com/r/haskell/comments/8bylw/ask_haskell_reddit_how_can_we_improve_the/c08tc0q
[5] Hackage Web Interface, Doc-Browser.
http://code.google.com/soc/2007/haskell/appinfo.html?csaid=D505A03B8B64C3BF
[6] cabal-install should report build results to hackage server.
http://hackage.haskell.org/trac/hackage/ticket/184
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: GSoC: Hackage 2.0

Matthew Gruen
On Wed, Apr 7, 2010 at 12:40 AM, Matthew Gruen <[hidden email]> wrote:
> Hi Haskellers,
>
> <snip>

Oh, heh, I apologize if that was more of a wall of text than I had
realized. The above wasn't a project proposal itself, more the result
of some brainstorming and some research. If you have the time to read
it, I'd really appreciate your feedback.

In fewer words, what kinds of features would benefit the community
most for Hackage?

/Matt
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: GSoC: Hackage 2.0

Maciej Piechotka
In reply to this post by Matthew Gruen
On Wed, 2010-04-07 at 00:40 -0400, Matthew Gruen wrote:

> Hi Haskellers,
>
> I'm Matt Gruen (Gracenotes in #haskell), and the Hackage 2.0 SoC
> project at <http://hackage.haskell.org/trac/summer-of-code/ticket/1587>
> really piqued my interest. It seems doable, in a summer, to make the
> new hackage-server more-than-deployment-ready as well as clearing out
> some items in the hackage bug tracker[0]; so, I've been working on a
> proposal. In this email I'd like to consolidate my mental notes for
> haskell-cafe and formulate a roadmap towards a more social Hackage.
>
> The most vital part is getting hackage-server
> <http://code.haskell.org/hackage-server/> to a state where it can be
> switched in place of hackage-scripts
> <http://darcs.haskell.org/hackage-scripts/>, and doing it properly,
> organizing the code so it can be extended painlessly in the future.
> Duncan Coutts, Antoine Latter, and others have done some great work on
> it in the past few years. I've been using Haskell for 1.33 of those
> years. I think I could become fluent in the codebases after a week of
> dedicated study, although even the hackage-server and hackage-scripts
> repositories don't include some components such as the current build
> report script.
>
> For putting the 2.0 in Hackage 2.0, any interface changes should help
> the library users and the library writers/uploaders without hurting
> either of them. Hackage should contain more of the right kind of
> information. Statistics help everyone, and they're a pretty good gauge
> on the direction of Hackage as a whole. Package popularity contents
> are one form of this. Reverse dependencies and even dependency
> graphs[1] are great, if I can integrate and expand Roel van Dijk's
> work[2].
>
> There should also be some space on package pages, or on pages a link
> away from them, for users to contribute information and suggestions.
> Coders can explain why or why not the package met their needs, as a
> sort of informal bug/enhancement tracking service. Another helpful
> flavor of information is package relationships beyond dependencies:
> 'Deprecated in favor of Foo', 'a fork of Foo', or 'does blah instead
> of Foo's plugh functionality'. This will help people find the 'right'
> package among look-alikes. There could also be links to illustrative
> usage examples for getting started. Happstack's state management will
> be a good match for updating and serving this information in real
> time.
>
I guess 'works also with B in version X.Y.Z' is also. Most of the above
changes should not be IMHO in cabal (sorry for answering PS here).
Especially 'not maintained anymore' and 'does not build on recent
GHC' ;)

> There's also a need for a more interactive form of package
> documentation, but this should strengthen relationships with existing
> tools like Haddock and Cabal, not bypass the tools. For example,
> adding a changelog[3] or making Haddock's declaration-by-declaration
> commentary more wiki-like[4]. Changelogs seem to be within the scope
> of Hackage 2.0, integrating with Cabal; Haddock wikification might not
> be, perhaps deserving a separate student-summer session of its own.
> These can improve the package page and documentation subtrees.
>

As we are with Haddock - 'rebuild' button if package failed to build?
For some reasons some packages fails to build on hackage (as yi). I'm
not sure if it will work however. At least uploading docs should be
possible.

Regards


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Matthew Gruen
On Wed, Apr 7, 2010 at 9:02 AM, Maciej Piechotka <[hidden email]> wrote:

> I guess 'works also with B in version X.Y.Z' is also. Most of the above
> changes should not be IMHO in cabal (sorry for answering PS here).
> Especially 'not maintained anymore' and 'does not build on recent
> GHC' ;)
>
> <quote>
>
> As we are with Haddock - 'rebuild' button if package failed to build?
> For some reasons some packages fails to build on hackage (as yi). I'm
> not sure if it will work however. At least uploading docs should be
> possible.
>
> Regards
>

More flexibility for building times is a good thing, but people
shouldn't able to DDOS the server by compiling, since it's a rather
CPU-intensive thing, or by building documentation. Now, it looks like
yi fails because data-accessor-template fails because of TH versioning
issues (seemingly). I like the ideas of docs you can upload to get
around these sorts of problems, and the ability to add links to other
docs and tutorials. People could edit Hackage to reference
haskellwiki, community.haskell.org, and other sites/services.

I see what you mean about the changes being segregated from cabal, and
I agree here. There's a set of information that's useful both for the
web interface and for cabal-install (which is what Hackage currently
displays, although perhaps not enough of it), and that should stay in
the .cabal files. But, other information that's user-editable on
Hackage is a plus.

/Matt
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Antoine Latter-2
On Thu, Apr 8, 2010 at 6:38 PM, Matthew Gruen <[hidden email]> wrote:

> On Wed, Apr 7, 2010 at 9:02 AM, Maciej Piechotka <[hidden email]> wrote:
>> I guess 'works also with B in version X.Y.Z' is also. Most of the above
>> changes should not be IMHO in cabal (sorry for answering PS here).
>> Especially 'not maintained anymore' and 'does not build on recent
>> GHC' ;)
>>
>> <quote>
>>
>> As we are with Haddock - 'rebuild' button if package failed to build?
>> For some reasons some packages fails to build on hackage (as yi). I'm
>> not sure if it will work however. At least uploading docs should be
>> possible.
>>
>> Regards
>>
>
> More flexibility for building times is a good thing, but people
> shouldn't able to DDOS the server by compiling, since it's a rather
> CPU-intensive thing, or by building documentation. Now, it looks like
> yi fails because data-accessor-template fails because of TH versioning
> issues (seemingly). I like the ideas of docs you can upload to get
> around these sorts of problems, and the ability to add links to other
> docs and tutorials. People could edit Hackage to reference
> haskellwiki, community.haskell.org, and other sites/services.
>
> I see what you mean about the changes being segregated from cabal, and
> I agree here. There's a set of information that's useful both for the
> web interface and for cabal-install (which is what Hackage currently
> displays, although perhaps not enough of it), and that should stay in
> the .cabal files. But, other information that's user-editable on
> Hackage is a plus.
>

One thing in the branch over in http://code.haskell.org/hackage-server
 is the ability for package maintainers to upload documentation to the
server. This way we're not tying the ability of the server doing a
build-check to the ability to host documentation for a package.

Antoine
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Matthew Gruen
On Thu, Apr 8, 2010 at 10:58 PM, Antoine Latter <[hidden email]> wrote:
> One thing in the branch over in http://code.haskell.org/hackage-server
>  is the ability for package maintainers to upload documentation to the
> server. This way we're not tying the ability of the server doing a
> build-check to the ability to host documentation for a package.
>
> Antoine
>

Antoine, hi, I'm still absorbing the great things that have been done
in hackage-server so far. I saw some documentation replacement logic
in the code, but don't see a web interface for it. Is there one I'm
missing?

—Matt
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Antoine Latter-2
On Thu, Apr 8, 2010 at 11:07 PM, Matthew Gruen <[hidden email]> wrote:

> On Thu, Apr 8, 2010 at 10:58 PM, Antoine Latter <[hidden email]> wrote:
>> One thing in the branch over in http://code.haskell.org/hackage-server
>>  is the ability for package maintainers to upload documentation to the
>> server. This way we're not tying the ability of the server doing a
>> build-check to the ability to host documentation for a package.
>>
>> Antoine
>>
>
> Antoine, hi, I'm still absorbing the great things that have been done
> in hackage-server so far. I saw some documentation replacement logic
> in the code, but don't see a web interface for it. Is there one I'm
> missing?
>

There's nothing so useful! The feature is definitely missing proper
tool support ,- at the moment there is only a POST request to a
particular address.

Antoine
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: GSoC: Hackage 2.0

Duncan Coutts-4
In reply to this post by Matthew Gruen
On Wed, 2010-04-07 at 00:40 -0400, Matthew Gruen wrote:
> Hi Haskellers,
>
> I'm Matt Gruen (Gracenotes in #haskell), and the Hackage 2.0 SoC
> project at <http://hackage.haskell.org/trac/summer-of-code/ticket/1587>
> really piqued my interest. It seems doable, in a summer, to make the
> new hackage-server more-than-deployment-ready as well as clearing out
> some items in the hackage bug tracker[0]; so, I've been working on a
> proposal. In this email I'd like to consolidate my mental notes for
> haskell-cafe and formulate a roadmap towards a more social Hackage.

Great.

> The most vital part is getting hackage-server
> <http://code.haskell.org/hackage-server/> to a state where it can be
> switched in place of hackage-scripts
> <http://darcs.haskell.org/hackage-scripts/>, and doing it properly,
> organizing the code so it can be extended painlessly in the future.

Yes. I should warn you that I've become increasingly keen on the latter
aspect recently. :-)

> For putting the 2.0 in Hackage 2.0, any interface changes should help
> the library users and the library writers/uploaders without hurting
> either of them.

Yes, there can sometimes be a bit of a tradoff between users and
uploaders. With some proposed features we have to be careful not to
annoy one group or the other.

> Hackage should contain more of the right kind of information.
> Statistics help everyone, and they're a pretty good gauge on the
> direction of Hackage as a whole. Package popularity contents are one
> form of this. Reverse dependencies and even dependency graphs[1] are
> great, if I can integrate and expand Roel van Dijk's work[2].

Yep, reverse deps are totally doable and really useful. Number of
reverse deps, combined with number of downloads is probably a pretty
good popularity metric.

> There should also be some space on package pages, or on pages a link
> away from them, for users to contribute information and suggestions.
> Coders can explain why or why not the package met their needs, as a
> sort of informal bug/enhancement tracking service.

Yeah, that's where we've got to be careful. Many packages already have
bug trackers and maintainers do not necessarily want yet another website
to have to cover to see where users are complaining. I think a user
commenting system is probably one of the most tricky bits to design,
because of the social aspects. There are issues like not duplicating
existing mailing lists / bug trackers / wikis and trying to keep
information relevant as new releases come out (eg imaging a comment
saying "this package is no good because it does not have feature X" and
yet the current release has feature X).

My suggestion is to put this feature further down the TODO list.

> Another helpful flavor of information is package relationships beyond
> dependencies: 'Deprecated in favor of Foo', 'a fork of Foo'

Yes, deprecation is important. We currently have some support for that,
but it's not very good or easy for maintainers to use.

> There's also a need for a more interactive form of package
> documentation, but this should strengthen relationships with existing
> tools like Haddock and Cabal, not bypass the tools. For example,
> adding a changelog[3] or making Haddock's declaration-by-declaration
> commentary more wiki-like[4]. Changelogs seem to be within the scope
> of Hackage 2.0, integrating with Cabal; Haddock wikification might not
> be, perhaps deserving a separate student-summer session of its own.
> These can improve the package page and documentation subtrees.

Yes, I'd suggest looking at the changelog issue but probably not wiki
haddock editing. That would indeed be cool but is a rather bigger scope.

> More generally, how can library users find the package they want?

Search! Metrics!

> Categories themselves are great, but a tag system could identify and
> group specific package functionality. There could be sorting by
> ratings and reviews (4/5 lambdas!). Metadata searches, like those
> Sascha Böhme implemented in SoC 2007[5], could be integrated. It's not
> always obvious which ideas will help and which won't see good returns,
> which makes it all the more important to bring hackage-server to a
> state where future extensions can be easily written, submitted and
> deployed. That's the goal here.

Again, I suspect this is a feature too far for a GSoC. If we can build
the infrastructure which makes adding such features easier then the
project would be a success.

> On the technical side, I realize I'd need to spend a not-insignificant
> amount of time on a user account system, dealing with authentication
> and related issues. One additional bit of functionality to manage is
> the hackage build system, which is used to ensure that packages build
> and to generate documentation. When building depends on FFI or
> OS-specific bindings, specific versions of other packages, compiler
> choice or compiler version choice, including language extensions, this
> is not trivial. One of two good routes is running cabal server-side to
> generate build reports and alleviate the miscompiling a smidge. Given
> that cabal install dependency calculation seems to be up in the air
> still, the current Setup-running script might end up staying for the
> moment, with some basic integrity checking (no cycles or other
> impossible dependency scenarios). The other not-mutually-exclusive
> approach is accepting failed build reports from users as a web
> service[6] to generate a matrix of the platforms that seem to
> encounter the most trouble.

Yes, I think build reporting is probably more important at the moment
than user comments or integration with hayoo/hoogle.

> Ultimately, it's important that I, or whoever ends up doing the
> project, plans for it to benefit you guys. What do you think?

So as I mentioned at the beginning, one of the most important aspects I
think is to get the architecture right. The hackage server will
increasingly become a huge database of useful information and it is
important that that information can be easily got at by other tools and
systems. My suggestion is to use a RESTful approach and to make
information available in human and machine readable formats so that it
is usable as a website and also by other specialised clients.

Another aspect is the internal architecture. We want to be able to add
(and remove) features relatively easily without everything getting
entangled together. Partly we want this for smoother feature growth (or
feature replacement) and partly because it is quite likely that we will
want slightly different sets of features in: a test/staging server, the
central server, package mirror servers, in-house team servers.

Duncan

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Job Vranish-2
In reply to this post by Matthew Gruen
I vote for adding a feature that would let people post comments/code snippets to the documentation of other peoples packages :)

It would be even nicer if you could post comments to individual haskell definitions on the haddock page, and then hide most of them by default under an expander of some sort.

I've often spent time trying to figure out how poorly documented function(s) on someone else's package worked. Once I've figured it out, I usually have a nice little example or explanation that I could post to save other people the same trouble. Having an easy way to do this would be nice.

Basically any collaborative/wikish enhancements to the documentation on hackage packages would make me happy :)

- Job

On Wed, Apr 7, 2010 at 4:43 AM, Matthew Gruen <[hidden email]> wrote:
On Wed, Apr 7, 2010 at 12:40 AM, Matthew Gruen <[hidden email]> wrote:
> Hi Haskellers,
>
> <snip>

Oh, heh, I apologize if that was more of a wall of text than I had
realized. The above wasn't a project proposal itself, more the result
of some brainstorming and some research. If you have the time to read
it, I'd really appreciate your feedback.

In fewer words, what kinds of features would benefit the community
most for Hackage?

/Matt
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Ivan Lazar Miljenovic
Job Vranish <[hidden email]> writes:
> I vote for adding a feature that would let people post comments/code
> snippets to the documentation of other peoples packages :)

You mean turn every hackage project page into a mini wiki?

> It would be even nicer if you could post comments to individual haskell
> definitions on the haddock page, and then hide most of them by default under
> an expander of some sort.

Rather than, you know, providing the maintainer with a patch with some
improved documentation?

--
Ivan Lazar Miljenovic
[hidden email]
IvanMiljenovic.wordpress.com
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Job Vranish-2


On Fri, Apr 9, 2010 at 9:46 AM, Ivan Lazar Miljenovic <[hidden email]> wrote:
Job Vranish <[hidden email]> writes:
> I vote for adding a feature that would let people post comments/code
> snippets to the documentation of other peoples packages :)

You mean turn every hackage project page into a mini wiki?

Yep.
 
> It would be even nicer if you could post comments to individual haskell
> definitions on the haddock page, and then hide most of them by default under
> an expander of some sort.

Rather than, you know, providing the maintainer with a patch with some
improved documentation?


This is often more difficult than it sounds. The biggest obstacle to this approach is that a new hackage version of the package must to be uploaded to update the documentation and the authors (me included) tend to prefer to push new packages only when there are significant changes.

Steps involved currently:
0. pull down package source to build manually
1. add documentation/code snippet to source
2. build haddock documentation
3. debug bad/ugly syntax / missing line breaks that break haddock
4. generate a patch
5. email patch to author
6. wait a week for the author to actually get around to applying the patch to whatever repository the source resides
7. wait several weeks for the author to release the next version of the package

Steps involved with mini wiki:
0. add [code] [/code] tags (or whatever)
1. copy
2. paste
3. submit

I think making this process easier would greatly increase the community involvement in the generation of documentation and improve the quality of the documentation as a whole.

I would imaging that this would not be a trivial task, but I think even something super simple (like what they have for the php documentation) would be much better than nothing.

- Job
 


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Edward Kmett-2
On Fri, Apr 9, 2010 at 10:21 AM, Job Vranish <[hidden email]> wrote:
On Fri, Apr 9, 2010 at 9:46 AM, Ivan Lazar Miljenovic <[hidden email]> wrote:
Job Vranish <[hidden email]> writes:
> I vote for adding a feature that would let people post comments/code
> snippets to the documentation of other peoples packages :)
You mean turn every hackage project page into a mini wiki?
Yep.

How would such annotations/snippets/changes react to the next release of the package? Would they be per-package? per version?

-Edward Kmett

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Malcolm Wallace
In reply to this post by Job Vranish-2
> > It would be even nicer if you could post comments to individual  
> haskell
> > definitions on the haddock page, and then hide most of them by  
> default under
> > an expander of some sort.
>
> Rather than, you know, providing the maintainer with a patch with some
> improved documentation?


How much cooler it would be, if the wiki-like comment on Hackage could  
automatically be converted into a darcs/git/whatever patch, and mailed  
to the package author/maintainer by Hackage itself.

Regards,
     Malcolm

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Job Vranish-2
In reply to this post by Edward Kmett-2


On Fri, Apr 9, 2010 at 10:31 AM, Edward Kmett <[hidden email]> wrote:
On Fri, Apr 9, 2010 at 10:21 AM, Job Vranish <[hidden email]> wrote:
On Fri, Apr 9, 2010 at 9:46 AM, Ivan Lazar Miljenovic <[hidden email]> wrote:
Job Vranish <[hidden email]> writes:
> I vote for adding a feature that would let people post comments/code
> snippets to the documentation of other peoples packages :)
You mean turn every hackage project page into a mini wiki?
Yep.

How would such annotations/snippets/changes react to the next release of the package? Would they be per-package? per version?

-Edward Kmett

Yeah that's the sticky part.

I think I would make comments only apply only to the version of the package they were submitted to, and then make it the package maintainers responsibility to potentially update the hard documentation with the more useful comments when he/she releases the next version.

This keeps the comments up to date, and helps prevent things from getting too clogged up with junk comments.

It would also be nice to provide an easy way for maintainers to copy comments from older versions to newer ones, but this is a bit more tricky to implement.

- Job

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Job Vranish-2
In reply to this post by Malcolm Wallace


On Fri, Apr 9, 2010 at 10:46 AM, Malcolm Wallace <[hidden email]> wrote:
> It would be even nicer if you could post comments to individual haskell
> definitions on the haddock page, and then hide most of them by default under
> an expander of some sort.

Rather than, you know, providing the maintainer with a patch with some
improved documentation?


How much cooler it would be, if the wiki-like comment on Hackage could automatically be converted into a darcs/git/whatever patch, and mailed to the package author/maintainer by Hackage itself.

This would indeed be awesome :)

Though I think I would prefer to select, from a list of comments, which ones I would like to include, and then click the "Download darcs/git/whatever patch" button. (rather than get hit by emails )

- Job

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Matthew Gruen
On Fri, Apr 9, 2010 at 10:59 AM, Job Vranish <[hidden email]> wrote:

>
>
> On Fri, Apr 9, 2010 at 10:46 AM, Malcolm Wallace
> <[hidden email]> wrote:
>> How much cooler it would be, if the wiki-like comment on Hackage could
>> automatically be converted into a darcs/git/whatever patch, and mailed to
>> the package author/maintainer by Hackage itself.
>>
> This would indeed be awesome :)
>
> Though I think I would prefer to select, from a list of comments, which ones
> I would like to include, and then click the "Download darcs/git/whatever
> patch" button. (rather than get hit by emails )
>
> - Job
>

That's what this proposal is, suggested a year ago:

http://www.reddit.com/r/haskell/comments/8bylw/ask_haskell_reddit_how_can_we_improve_the/c08tc0q

The wiki way to go about it where changes can still be sent upstream
is to make patches for every documentation change (using a simple
editing interface, not the HTML source), and update Hackage's Haddock
tree with the patch. The author can then apply the patches. If he
doesn't patch anything but submits a new version of the source, the
patches will just be reapplied again, with prompts to resolve
conflicts which he or others can resolve. Complicated, but I think it
would work.

/Matt
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Nick Bowler-3
In reply to this post by Job Vranish-2
On 10:21 Fri 09 Apr     , Job Vranish wrote:

> On Fri, Apr 9, 2010 at 9:46 AM, Ivan Lazar Miljenovic <
> [hidden email]> wrote:
>
> > Job Vranish <[hidden email]> writes:
> > > I vote for adding a feature that would let people post comments/code
> > > snippets to the documentation of other peoples packages :)
> >
> > You mean turn every hackage project page into a mini wiki?
> >
> > Yep.

My worry with this is that users will fill carefully written
documentation with irreverent nonsense or, worse, factual errors.
Moderation seems necessary.

> > > It would be even nicer if you could post comments to individual haskell
> > > definitions on the haddock page, and then hide most of them by default
> > > under an expander of some sort.
> >
> > Rather than, you know, providing the maintainer with a patch with some
> > improved documentation?
> >
> This is often more difficult than it sounds. The biggest obstacle to this
> approach is that a new hackage version of the package must to be uploaded to
> update the documentation and the authors (me included) tend to prefer to
> push new packages only when there are significant changes.

It seems to me that the solution to this particular problem is to allow
package maintainers to publish updated documentation separately from new
packages.

> Steps involved currently:
> 0. pull down package source to build manually
> 1. add documentation/code snippet to source
> 2. build haddock documentation
> 3. debug bad/ugly syntax / missing line breaks that break haddock
> 4. generate a patch
> 5. email patch to author
> 6. wait a week for the author to actually get around to applying the patch
> to whatever repository the source resides
> 7. wait several weeks for the author to release the next version of the
> package

I suspect that most maintainers are amenable to simple emails containing
change requests where documentation is concerned ("please change the
first sentence of bazify's documentation to ..."),  which means you can
skip steps 0 through 4.

> Steps involved with mini wiki:
> 0. add [code] [/code] tags (or whatever)
> 1. copy
> 2. paste
> 3. submit
>
> I think making this process easier would greatly increase the community
> involvement in the generation of documentation and improve the quality of
> the documentation as a whole.
>
> I would imaging that this would not be a trivial task, but I think even
> something super simple (like what they have for the php documentation) would
> be much better than nothing.

PHP's comments are a fine example of what I *don't* want to see
polluting my documentation.  There is very little signal to be found
amongst that noise.

> On Fri, Apr 9, 2010 at 10:46 AM, Malcolm Wallace <[hidden email]> wrote:
> > How much cooler it would be, if the wiki-like comment on Hackage could
> > automatically be converted into a darcs/git/whatever patch, and mailed to
> > the package author/maintainer by Hackage itself.
> >
> > This would indeed be awesome :)
>
> Though I think I would prefer to select, from a list of comments, which ones
> I would like to include, and then click the "Download darcs/git/whatever
> patch" button. (rather than get hit by emails )

The resulting patches will be patently useless, because the only sort of
changes they can make is to append bullet points to existing documentation.
Unless you are proposing that users can perform any change whatsoever on
hackage?

--
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Stephen Tetley-2
Hello all

I support the immediate aims of Matthew Gruen's proposal and hope it
gets adopted as a SoC project, but for the 'web2.0' aspects I largely
agree with Nick Bowler.

As a package author, checking disparate wiki pages to help people is
more work than answering emails. Also, if people have install problems
with (other peoples) packages on Windows and post a problem report to
the Cafe, I'll often have a look to see if I can help as it isn't much
trouble provided the dependency depth is near 0. I certainly won't be
able to do that if there is a switch over to wiki for reporting
problems.


Best wishes

Stephen
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: GSoC: Hackage 2.0

Matthew Gruen
In reply to this post by Matthew Gruen
The proposal as I submitted it is here:

http://docs.google.com/View?docid=0Afa5MxwyB_zYZGhjanNrdjNfMjkzZjloOWNienY&pageview=1&hgd=1&hl=en

And it might need further revision as I talk to Duncan and the
community. The advanced social features wouldn't get deployed by the
end of the summer, but by that point the project would have structural
integrity enough to write and release the features relatively easily,
as well as act as a web service for their functionality. About
concerns of excessive Web 2.0-ness: I agree that some of the features
should be optional, particularly ones which duplicate other formal and
informal bug-tracking systems. Ones which provide information to rank
and navigate packages effectively are more essential. There's a lot to
learn from other projects with similar systems and Haskellers'
experience with those.

Cheers,
Matt Gruen
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: Re: GSoC: Hackage 2.0

Don Stewart-2
wikigracenotes:

> The proposal as I submitted it is here:
>
> http://docs.google.com/View?docid=0Afa5MxwyB_zYZGhjanNrdjNfMjkzZjloOWNienY&pageview=1&hgd=1&hl=en
>
> And it might need further revision as I talk to Duncan and the
> community. The advanced social features wouldn't get deployed by the
> end of the summer, but by that point the project would have structural
> integrity enough to write and release the features relatively easily,
> as well as act as a web service for their functionality. About
> concerns of excessive Web 2.0-ness: I agree that some of the features
> should be optional, particularly ones which duplicate other formal and
> informal bug-tracking systems. Ones which provide information to rank
> and navigate packages effectively are more essential. There's a lot to
> learn from other projects with similar systems and Haskellers'
> experience with those.

Very pragmatic! Thanks for submitting a proposal!
_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
12