|
Hello,
This is to announce a provisional webpage to summarise hackage.haskell.org. http://www.fremissant.net/portackage It is a lot of data, so will be a bit slow to load. Columns can be sorted by clicking the headers, but this will also take some seconds on a typical client. The initial sort is by module name. Clicking a table row links to the project page on hackage. Although I believe this is a complete list of all packages/modules in hackage, it is probably still buggy, so if you see anything amiss please let me know. Also, a subsequent version in the near future will update automatically based on the http://hackage.haskell.org/packages/archive/recent.rss RSS feed, but for now it's based on a recent snapshot. Haskell code which prepares the page will be released after a few more days testing and refinement. Some previous work [1] provided a list of all modules, but is not being maintained, and I was unable to rearouse the author's interest. Feature requests and suggestions are welcome! Kind Reg'ds, Andrew Seniuk [1] http://folk.ntnu.no/hammar/explore-hackage/ _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
Jonathan,
On Thu, Apr 26, 2012 at 4:13 PM, Jonathan Daugherty <[hidden email]> wrote: >> Although I believe this is a complete list of all packages/modules in >> hackage, it is probably still buggy, so if you see anything amiss >> please let me know. > > It might not be showing the most recent releases of everything. As an > example, it shows a much older version of my vty-ui library (1.0) and > its metadata, instead of the latest version, 1.5. As a result the > version, module list, description, homepage, etc., are not current. Thank-you, I'd realised this issue but forgot to address it. It is due to a false assumption on my part that the .cabal entries in the 00-index.tar file were in increasing version order. I'll address this problem today. Cheers, Andrew _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
Should be fixed in that respect now, thanks again for pointing it out.
On Thu, Apr 26, 2012 at 4:23 PM, Andrew Seniuk <[hidden email]> wrote: > Jonathan, > > On Thu, Apr 26, 2012 at 4:13 PM, Jonathan Daugherty <[hidden email]> wrote: >>> Although I believe this is a complete list of all packages/modules in >>> hackage, it is probably still buggy, so if you see anything amiss >>> please let me know. >> >> It might not be showing the most recent releases of everything. As an >> example, it shows a much older version of my vty-ui library (1.0) and >> its metadata, instead of the latest version, 1.5. As a result the >> version, module list, description, homepage, etc., are not current. > > Thank-you, I'd realised this issue but forgot to address it. It is > due to a false assumption on my part that the .cabal entries in the > 00-index.tar file were in increasing version order. > > I'll address this problem today. > > Cheers, > Andrew _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
In reply to this post by Andrew Seniuk
Somewhat lighter, using summarised module lists. A more sophistocated
UI with per-column filtering is in the works. I invite your comments. Andrew On Thu, Apr 26, 2012 at 4:05 PM, Andrew Seniuk <[hidden email]> wrote: > Hello, > > This is to announce a provisional webpage to summarise hackage.haskell.org. > > http://www.fremissant.net/portackage > > It is a lot of data, so will be a bit slow to load. Columns can be > sorted by clicking the headers, but this will also take some seconds > on a typical client. The initial sort is by module name. Clicking a > table row links to the project page on hackage. > > Although I believe this is a complete list of all packages/modules in > hackage, it is probably still buggy, so if you see anything amiss > please let me know. Also, a subsequent version in the near future will > update automatically based on the > http://hackage.haskell.org/packages/archive/recent.rss RSS feed, but > for now it's based on a recent snapshot. Haskell code which prepares > the page will be released after a few more days testing and > refinement. > > Some previous work [1] provided a list of all modules, but is not > being maintained, and I was unable to rearouse the author's interest. > > Feature requests and suggestions are welcome! > > Kind Reg'ds, > Andrew Seniuk > > [1] http://folk.ntnu.no/hammar/explore-hackage/ _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
Excuse the delay, I have no home internet.
> On Fri, Apr 27, 2012 at 1:16 AM, <[hidden email]> wrote: > Great idea. Maybe the format can start as compactly as possible, > i.e. 1 entry per line to make it easy to browse as many entries > as possible in one page. > > It looks as though disabling just the dependencies would go a > long way to accomplish that. Yep, and in fact the version I uploaded while reading your message has the deps column elided (and one or two others). Enforcing single-line rows is a but a few tweaks away. The thing is, the module namespace was the main motivator for this project, so I'm also seeing whether I can get away with a single view, package-oriented as presently, and still give good focus to the modules names. Soon I hope to include "accordion" views of the module namespace, with cookie-based (?) persistent state so user's collapse/expand tree state is preserved. > I'd like to vote to make the synopsis to be the second column. > > will each column type have an enable/disable ? I like the idea (although that one wasn't in the upload). Hopefully when I explore the JavaScript UI tools a bit more, the columns (key structure) will be totally configurable by drag-and-drop, with columns sizable, and keys selected from a listbox. (?) stuff like that > it's probably best if a click on a specific target, say the actual > name of the package, sends you to the hackage entry instead of a > click anywhere on the row. I find myself clicking on the page > by accident and it sending me off to some random hackage entry. :) That's a feature! But seriously, you're right, and as a bonus the links can be specific to the columns (package -> hackage, modules -> API pages, homepages -> hps, maintainer -> sendto [1]). (A 2-finger or Mod- click anywhere on a row might still be nice, but a bare tap is admittedly way too accident prone with touch input.) > I really like the idea - makes your hackage surfing > much more enjoyable ! > > Brian And thank-you for sharing your thoughts! I'm committed to improving it, until the ideas run out. > On Fri, Apr 27, 2012 at 4:12 AM, Christoph Breitkopf <[hidden email]> wrote: > this already looks great. So here's my wish-list, in case you have > too much time on yours hands ;-) Thanks! Yes I suppose I do have too much time on my hands, but the pain of that is conterbalanced by all the excitement. > Quality/popularity indicators (I'm aware that some of these are > real work and better provided by hackage itself): > - used by how many other packages (number of edges to that package > in the dependency graph of all packages) > - haddock coverage, test coverage, etc. > - like, +1, whatnot button > - link to issue tracker, if there is one If there's interest, this could become integrated with hackage at some point in the future. Next thing is to get a "last upload" field which can replace the version field. The codependency graph :) would be a nice metric to add, I'll look into it. I remember seeing a package that does this someplace on hackage. The +1 button is a good one! In short I'll try to implement all these ideas, thanks to you both for the encouraging comments. -Andrew [1] I omitted the maintainer column, thinking its inclusion might be harmful, making the emails so easy for a bot to collect. _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
Hi Derek,
Thanks for your comments. Yes, I am probably going to focus on making it work as a web service and stay away from platform-specific mobile apps. Not been thinking mobile much at all, since the quantities of data are taxing even to my laptop, and also I'm inclined to believe people are mostly at a laptop or desktop when coding Haskell. However, I'm planning to try server-side sorting and filtering, and compare performance. No DB is used on the server yet, but it would be simple enough I suppose -- all this is quite new territory for me. Also my shared hosting resource limits may become a problem if this gets a lot of use. On the other hand, was planning to use persistence (cookies i guess?) to save the data on the client, so that patches based on Hackage RSS could be applied incrementally. Also, I think the sorting could be faster by sending per-column lists of indices. But maybe it is just the browser rendering which is taking all the time? I'll take this opportunity to announce a few improvements: - shift-click a row to see the Hackage page (bare click does nothing) - per-column filtering widgets Presently working on getting the "last updated" column, this information is not in a .cabal file but I'm hoping it is in the TAR metadata... The link again: http://www.fremissant.net/portackage A. _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
> On Apr 27, 2012 6:18 PM, "Andrew Seniuk" <[hidden email]> wrote:
> Oh, the idea wasn't that you would make the app, but that by providing a web > service interface you would make it easier for me or anyone else to make an > app or other alternative interface. It would also reduce bandwidth usage > over screen scraping. Yes, well as I mentioned in the part of my response you didn't quote, I agree it would be desireable, and am working on it, but my motivation was to provide people with something immediately useful. Once I get something reasonably efficient worked out, I'll look at uploading the Haskell code to Hackage, providing JSON data, etc.. Meanwhile if someone wanted to work on their own app or interface, it's not too hard to get CSV from the table. I made a mistake and the link was down for about 12 hours. It's back now, with a number of improvements and up-to-date hackage info. It is an expensive page to run, you may get "Script not responding" pop-ups depending on your hardware, but I think it's worth it. http://fremissant.net/portackage Kind Reg'ds, Andrew _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
Great work! Thanks.
Does it include only packages without executables ? Eg I see hledger-lib but not the hledger or gist packages. It would be nice to have all of hackage there. Best - Simon _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
First let me mention, there is now a complete modules tree with API
links (in a very preliminary state), at http://fremissant.net/portackage/modules.php and the package view is still http://fremissant.net/portackage On Mon, Apr 30, 2012 at 11:35 AM, Simon Michael <[hidden email]> wrote: > Great work! Thanks. > > Does it include only packages without executables ? Eg I see > hledger-lib but not the hledger or gist packages. It would be nice > to have all of hackage there. Simon, thank you. And, yes, as a temporary measure, I elided the packages that expose no modules, to try to tame the slowness a bit. I could provide them on a similar, separate page, until figure out some optimisations. Derek, if you're reading, sorry if I seemed defensive there before. :) Okay, this bagel is screaming "Eat Me!!"... Kind Reg'ds, Andrew On Mon, Apr 30, 2012 at 11:35 AM, Simon Michael <[hidden email]> wrote: > Great work! Thanks. > > Does it include only packages without executables ? Eg I see hledger-lib but > not the hledger or gist packages. It would be nice to have all of hackage > there. > > Best - Simon > > > _______________________________________________ > Haskell-Cafe mailing list > [hidden email] > http://www.haskell.org/mailman/listinfo/haskell-cafe _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
...and the "binary" packages (those exposing no modules) are now up at
http://fremissant.net/portackage/binary.php Not nice to have them in a separate table, but for now that's the case. :( So the three links in summary: module tree: http://fremissant.net/portackage/modules.php library packages: http://fremissant.net/portackage/binary.php other packages: http://fremissant.net/portackage/portackage.php On Mon, Apr 30, 2012 at 1:42 PM, Andrew Seniuk <[hidden email]> wrote: > First let me mention, there is now a complete modules tree with API > links (in a very preliminary state), at > http://fremissant.net/portackage/modules.php and the package view is > still http://fremissant.net/portackage > > On Mon, Apr 30, 2012 at 11:35 AM, Simon Michael <[hidden email]> wrote: >> Great work! Thanks. >> >> Does it include only packages without executables ? Eg I see >> hledger-lib but not the hledger or gist packages. It would be nice >> to have all of hackage there. > > Simon, thank you. And, yes, as a temporary measure, I elided the > packages that expose no modules, to try to tame the slowness a bit. I > could provide them on a similar, separate page, until figure out some > optimisations. > > Derek, if you're reading, sorry if I seemed defensive there before. :) > > Okay, this bagel is screaming "Eat Me!!"... > > Kind Reg'ds, > Andrew > > On Mon, Apr 30, 2012 at 11:35 AM, Simon Michael <[hidden email]> wrote: >> Great work! Thanks. >> >> Does it include only packages without executables ? Eg I see hledger-lib but >> not the hledger or gist packages. It would be nice to have all of hackage >> there. >> >> Best - Simon >> >> >> _______________________________________________ >> Haskell-Cafe mailing list >> [hidden email] >> http://www.haskell.org/mailman/listinfo/haskell-cafe _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
Oops, the three links in summary are:
module tree: http://fremissant.net/portackage/modules.php library packages: http://fremissant.net/portackage/portackage.php other packages: http://fremissant.net/portackage/binary.php _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
Hi ___,
Thank you for your suggestion to provide JSON, which seconds Derek's earlier suggestion. I have anonymised your name since you wrote to me but didn't CC the list. :) As much as I wish the web interface was up to snuff, it is really too slow. While I continue to try to resolve that, there is something like a JSON feed available now. I'm new to these things and may be doing it wrong. Any advice would be appreciated, either in accelerating the web interface or in providing the JSON. I have no idea whether it parses or not. JSON Schema: http://www.fremissant.net/portackage/packages.schema.json.txt Hopefully compressed JSON data: http://www.fremissant.net/portackage/packages.php Raw JSON data in case the compressed doesn't work: http://www.fremissant.net/portackage/packages.json The hackage data updated as of today (June 14). For now this is only the packages which expose modules (i.e. not including binary packages). No particular reason, I just ran out of time this afternoon and thought I'd post what I've got. The fields present in the JSON are those available on the HTML table version. There are several dozen other fields which come up more or less infrequently in the Hackage packages, and they could all be included. The complete list of modules is included (rather than the summary versions used in the HTML table version). A next step would be for the server to incrementally update the JSON data based on the Hackage RSS feed. Kind Regards, Andrew On Wed, Jun 13, 2012 at 7:58 AM, ___ wrote: > Hi Andrew > > Portackage is a great site for browsing packages, but it's really > really slow for filtering per column. What about offering an > additional non-javascript version of the pages -- searching with plain > text in a browser works quite well. > > On 4/30/12, Andrew Seniuk <[hidden email]> wrote: >> Oops, the three links in summary are: >> >> module tree: http://fremissant.net/portackage/modules.php >> library packages: http://fremissant.net/portackage/portackage.php >> other packages: http://fremissant.net/portackage/binary.php >> >> _______________________________________________ >> Haskell-Cafe mailing list >> [hidden email] >> http://www.haskell.org/mailman/listinfo/haskell-cafe >> > > > -- > ___ _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
I discovered http://jsonlint.com/ and got it to validate. :)
On Thu, Jun 14, 2012 at 6:18 PM, Andrew Seniuk <[hidden email]> wrote: > Hi ___, > > Thank you for your suggestion to provide JSON, which seconds Derek's > earlier suggestion. I have anonymised your name since you wrote to me > but didn't CC the list. :) > > As much as I wish the web interface was up to snuff, it is really too > slow. While I continue to try to resolve that, there is something like > a JSON feed available now. I'm new to these things and may be doing it > wrong. Any advice would be appreciated, either in accelerating the web > interface or in providing the JSON. I have no idea whether it parses > or not. > > JSON Schema: > http://www.fremissant.net/portackage/packages.schema.json.txt > > Hopefully compressed JSON data: > http://www.fremissant.net/portackage/packages.php > > Raw JSON data in case the compressed doesn't work: > http://www.fremissant.net/portackage/packages.json > > The hackage data updated as of today (June 14). > > For now this is only the packages which expose modules (i.e. not > including binary packages). No particular reason, I just ran out of > time this afternoon and thought I'd post what I've got. > > The fields present in the JSON are those available on the HTML table > version. There are several dozen other fields which come up more or > less infrequently in the Hackage packages, and they could all be > included. The complete list of modules is included (rather than the > summary versions used in the HTML table version). > > A next step would be for the server to incrementally update the JSON > data based on the Hackage RSS feed. > > Kind Regards, > Andrew > > On Wed, Jun 13, 2012 at 7:58 AM, ___ wrote: >> Hi Andrew >> >> Portackage is a great site for browsing packages, but it's really >> really slow for filtering per column. What about offering an >> additional non-javascript version of the pages -- searching with plain >> text in a browser works quite well. >> >> On 4/30/12, Andrew Seniuk <[hidden email]> wrote: >>> Oops, the three links in summary are: >>> >>> module tree: http://fremissant.net/portackage/modules.php >>> library packages: http://fremissant.net/portackage/portackage.php >>> other packages: http://fremissant.net/portackage/binary.php >>> >>> _______________________________________________ >>> Haskell-Cafe mailing list >>> [hidden email] >>> http://www.haskell.org/mailman/listinfo/haskell-cafe >>> >> >> >> -- >> ___ _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
|
Now all packages, including those without exposed modules, are
included. Also, I've attempted to parse the author lists into JSON arrays. Due to the free format of many fields in the .cabal package descriptions, including author list, there are many exceptional cases where they are ill-parsed, but at least the JSON file is well-formed and validates at jsonlint.com. Once more: JSON Schema: http://www.fremissant.net/portackage/packages.schema.json.txt Hopefully compressed JSON data: http://www.fremissant.net/portackage/packages.php Raw JSON data in case the compressed doesn't work: http://www.fremissant.net/portackage/packages.json The hackage data updated as of yesterday (June 14). Will work on daily incremental updates from Hackage RSS feed soon. -Andrew On Thu, Jun 14, 2012 at 7:05 PM, Andrew Seniuk <[hidden email]> wrote: > I discovered http://jsonlint.com/ and got it to validate. :) > > On Thu, Jun 14, 2012 at 6:18 PM, Andrew Seniuk <[hidden email]> wrote: >> Hi ___, >> >> Thank you for your suggestion to provide JSON, which seconds Derek's >> earlier suggestion. I have anonymised your name since you wrote to me >> but didn't CC the list. :) >> >> As much as I wish the web interface was up to snuff, it is really too >> slow. While I continue to try to resolve that, there is something like >> a JSON feed available now. I'm new to these things and may be doing it >> wrong. Any advice would be appreciated, either in accelerating the web >> interface or in providing the JSON. I have no idea whether it parses >> or not. >> >> JSON Schema: >> http://www.fremissant.net/portackage/packages.schema.json.txt >> >> Hopefully compressed JSON data: >> http://www.fremissant.net/portackage/packages.php >> >> Raw JSON data in case the compressed doesn't work: >> http://www.fremissant.net/portackage/packages.json >> >> The hackage data updated as of today (June 14). >> >> For now this is only the packages which expose modules (i.e. not >> including binary packages). No particular reason, I just ran out of >> time this afternoon and thought I'd post what I've got. >> >> The fields present in the JSON are those available on the HTML table >> version. There are several dozen other fields which come up more or >> less infrequently in the Hackage packages, and they could all be >> included. The complete list of modules is included (rather than the >> summary versions used in the HTML table version). >> >> A next step would be for the server to incrementally update the JSON >> data based on the Hackage RSS feed. >> >> Kind Regards, >> Andrew >> >> On Wed, Jun 13, 2012 at 7:58 AM, ___ wrote: >>> Hi Andrew >>> >>> Portackage is a great site for browsing packages, but it's really >>> really slow for filtering per column. What about offering an >>> additional non-javascript version of the pages -- searching with plain >>> text in a browser works quite well. >>> >>> On 4/30/12, Andrew Seniuk <[hidden email]> wrote: >>>> Oops, the three links in summary are: >>>> >>>> module tree: http://fremissant.net/portackage/modules.php >>>> library packages: http://fremissant.net/portackage/portackage.php >>>> other packages: http://fremissant.net/portackage/binary.php >>>> >>>> _______________________________________________ >>>> Haskell-Cafe mailing list >>>> [hidden email] >>>> http://www.haskell.org/mailman/listinfo/haskell-cafe >>>> >>> >>> >>> -- >>> ___ _______________________________________________ Haskell-Cafe mailing list [hidden email] http://www.haskell.org/mailman/listinfo/haskell-cafe |
| Powered by Nabble | Edit this page |
