Speculation, OT: Program a Spreadsheet

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Speculation, OT: Program a Spreadsheet

trent shipley
I have a vision for a spreadsheet that is programmable as a spreadsheet itself. (Excel, for example, is programmable in VBA, but VBA requires switching modes to the VBA environment and using a non-spreadsheet idiom. Excel is not self-programmable as a spreadsheet.)

First there one function in a cell or array can reference other cells and arrays, including those that contain functions themselves. Second, IF(boolean,then,else) functions provide for selection. 

Third, the Function local scope property for a sheet in a workbook, the PARAMETER() function, and the RETURN() function allow the spreadsheet programmer to write new functions, including recursive functions.

* Is a spreadsheet you can program from the spreadsheet a reasonable goal?
* Has it been done?
(I don't have the academic CS chops to know what keywords to use, and how search CS publications in English.)

I have written a 30 page essay on Scriptsheets, but the essence boils down to the above. (If you, for some inexplicable reason, actually want the draft of the essay I'd be more than happy to share it with the understanding that you would provide feedback).  I call the proposal Scriptsheets.

So the plan is to take something like GNUmeric or LibreOffice Calc and graft on a primitive function sheet interpreter.

It would be natural to use C++, but the astute will note that a spreadsheet basically does not rewrite cells (unless you use a circular reference), so I'd also like to use a functional language, maybe Haskel.

* Would using a functional language as a basic language of the project save effort and intellectual load?

In the longer term I'd like as much of the spreadsheet programmable as a spreadsheet to be written to run on the JVM.

As near as I can tell near future Java and typed functional languages, include the following options:

Eta,
Frege,
Kotlin and,
Scala.

Note that a spreadsheet needs to give the satisfaction of immediate results, or failing immediate results, the sensation of actively working, so if the language could be interpreted that would be a huge help.

* Which combination of typed, compiled, interpreted, FOSS functional language that runs on the JVM, JAVA, Haskell, C++, C, used in that order of preference, makes the most sense for the Java compatible functional language at the top of the preference hierarchy? 

Note also, that I have only the equivalent of an AA degree from a CIS, not a CS, perspective, so the odds are the whole idea is vaporware, unless I can determine feasibility and desirability, then sweet talk real developers to help out.

[hidden email] (junk email address)

See also the spreadsheet at http://orgmode.org/  it's cool.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Sergiu Ivanov-2
Hello Trent,

Thus quoth  trent shipley  on Sun Nov 19 2017 at 08:05 (+0100):
>
> I have a vision for a spreadsheet that is programmable as a spreadsheet
> itself.
[...]
> * Is a spreadsheet you can program from the spreadsheet a reasonable
> goal?

What kind of use cases do you imagine for your programmable spreadsheet?
"Reasonable" will usually depend on your context.

The idea in itself is definitely interesting, but you really should
define the application context to make it clearer.

> * Has it been done?
> (I don't have the academic CS chops to know what keywords to use, and how
> search CS publications in English.)

I haven't heard of anything like that expressly, but then the functions
in conventional spreadsheets are technically sufficiently powerful to
express any computation.

The question therefore comes back again to what your application would
be: you could design your spreadsheet around it.

> It would be natural to use C++, but the astute will note that a spreadsheet
> basically does not rewrite cells (unless you use a circular reference), so
> I'd also like to use a functional language, maybe Haskel.

Since you want to write functions in cells, you are probably better off
using a language with a compact syntax.  Haskell is one of them, but not
the only one.

Using C++ looks in no way natural _to me_; "natural" will also depend on
your use case ;-)

> Note that a spreadsheet needs to give the satisfaction of immediate
> results, or failing immediate results, the sensation of actively working,
> so if the language could be interpreted that would be a huge help.

Since your spreadsheet code will be heavily modularised, you could
probably do some very granular (therefore fast) compilation.

> Note also, that I have only the equivalent of an AA degree from a CIS, not
> a CS, perspective, so the odds are the whole idea is vaporware, unless I
> can determine feasibility and desirability, then sweet talk real developers
> to help out.

Having or not a degree is usually rather weakly (if not negatively)
correlated with thinking or being capable of getting things done :-) So
don't worry, go ahead and think big; there will always be people around
to try and shoot down the hot-air balloons of your imagination for you.

--
Sergiu

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

trent shipley
Middle posting follows

On Sun, Nov 19, 2017 at 3:40 AM Sergiu Ivanov <[hidden email]> wrote:
Hello Trent,

Thus quoth  trent shipley  on Sun Nov 19 2017 at 08:05 (+0100):
>
> I have a vision for a spreadsheet that is programmable as a spreadsheet
> itself.
[...]
> * Is a spreadsheet you can program from the spreadsheet a reasonable
> goal?

What kind of use cases do you imagine for your programmable spreadsheet?
"Reasonable" will usually depend on your context.

The idea in itself is definitely interesting, but you really should
define the application context to make it clearer.

I really haven't thought of specific use cases, but I have thought in general terms about what you might do with a natively programmable spreadsheet.

  1. It would be an interesting exercise in itself, and if it hadn't been done before, might be worth a paper. (I submit that this is actually a real use case.)
  2. It would be more elegant that the current solution to a programmable spreadsheet, which is a spreadsheet that is almost, but not quite programmable, with a helper imperative language and programming mode stuck onto the base spreadsheet.
  3. You could build a programmable spreadsheet on steroids with this idea, and I have some other interesting ideas, like "virtual spreadsheets" that would allow spreadsheets to extend into n dimensions (but I digress).
  4. So engineers, scientists, economists, and finance might have niche uses for a (n-dimensional) programmable spreadsheet on steroids--though I don't know in detail what those would be.
  5. I am a trained social scientist (anthropology) I know it is not uncommon for social scientists to have to deal with large, multidimensional datasets, and analyze the same. A multidimensional programmable spreadsheet might help a lot with that.
  6. I have heard that finance wizards often use Excel+VBA to do modeling. They might like a consistent, single language programming environment that offers more power and type safety that Excel + VBA. It would also need to be FAST if it is to be used for something like market analysis or automated trading.
Finding plausible use cases is also hindered by both a version of an existing spreadsheet enabling native functions, or a more full featured Scriptsheets product both being vaporware.
 
> * Has it been done?
> (I don't have the academic CS chops to know what keywords to use, and how
> search CS publications in English.)

I haven't heard of anything like that expressly, but then the functions
in conventional spreadsheets are technically sufficiently powerful to
express any computation.

The question therefore comes back again to what your application would
be: you could design your spreadsheet around it.

> It would be natural to use C++, but the astute will note that a spreadsheet
> basically does not rewrite cells (unless you use a circular reference), so
> I'd also like to use a functional language, maybe Haskel.

Since you want to write functions in cells, you are probably better off
using a language with a compact syntax.  Haskell is one of them, but not
the only one.

Using C++ looks in no way natural _to me_; "natural" will also depend on
your use case ;-)

The main argument for C++ is that if you are "reusing" existing code from GNUmeric or LibreOffice Calc none of that is in a functional language. It will tend to be in C++ (and maybe Java for Calc). That's an unfortunate fact if you want to pilot the central idea on a FOSS spreadsheet as a proof of concept. 

Also, if speed is of the essence, my impression is that C++ and C come to the fore.

 
> Note that a spreadsheet needs to give the satisfaction of immediate
> results, or failing immediate results, the sensation of actively working,
> so if the language could be interpreted that would be a huge help.

Since your spreadsheet code will be heavily modularised, you could
probably do some very granular (therefore fast) compilation.

> Note also, that I have only the equivalent of an AA degree from a CIS, not
> a CS, perspective, so the odds are the whole idea is vaporware, unless I
> can determine feasibility and desirability, then sweet talk real developers
> to help out.

Having or not a degree is usually rather weakly (if not negatively)
correlated with thinking or being capable of getting things done :-) So
don't worry, go ahead and think big; there will always be people around
to try and shoot down the hot-air balloons of your imagination for you.

--
Sergiu

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Joachim Durchholz
In reply to this post by trent shipley
Am 19.11.2017 um 08:05 schrieb trent shipley:
> * Is a spreadsheet you can program from the spreadsheet a reasonable goal?

I.e. use the same programming language for cells formulae and scripts?
Yes, that's very much reasonable.

> * Has it been done?

Not in any mainstream spreadsheet. Which boils down to two: Excel and
Open/Libre Office Calc.
Excel offers VBA.
Calc is actually language-agnostic and uses URLs and XML to tie things
together. It currently supports BeanShell (Java-without-types, it
seems), Java, JS, Python, and OO Basic.

> So the plan is to take something like GNUmeric or LibreOffice Calc and
> graft on a primitive function sheet interpreter.

The main point is that you'd have to replace the formular language.
I do not think that Calc was made for that.

> It would be natural to use C++,

You'd instantly kill adoption with that. Only a minimal part of the Calc
user base is even remotely capable of coding formulae in C++, and even
of these, a substantial fraction would be able but unwilling.

 > but the astute will note that a
> spreadsheet basically does not rewrite cells (unless you use a circular
> reference), so I'd also like to use a functional language, maybe Haskel.

You don't want to inline the formulae of other cells anyway, because
then the calculation will be done twice: Once to fill the referenced
cell, and once as part of the referencing cell.

So for reference cycles, you'll rely on whatever the spreadsheet is
already doing to deal with them.

> * Would using a functional language as a basic language of the project
> save effort and intellectual load?

That depends on whether you're talking about the implementation language
or the cell/macro language.

For the implementation language, you'll save the most time by using
whatever you already know. Unless the project is going to last longer
than, say, two years. And if you plan on getting other people to join
the project, you'll want the language with the largest pool of
interested and able people, which is essentially guesswork but I'd
avoid, say, the VBA or PHP crowd ;-)

For the user-facing language, do whatever is easy to use for a
non-programmer. Haskell should work fine, but prepare to collect
references to tutorials, and what does *not* work fine is performance
predictions, particularly not for nonprogrammers. This probably means
you need strict evaluation, which means even if it looks like Haskell
it's going to be an entirely different language.

> In the longer term I'd like as much of the spreadsheet programmable as a
> spreadsheet to be written to run on the JVM.
> As near as I can tell near future Java and typed functional languages,
> include the following options:
>
> Eta,
> Frege,
> Kotlin and,
> Scala.

I don't know any of these well enough to make any recommendations.

However, for the user-facing language, you need this:
1) As easy to learn as possible.
2) Scales well to a few thousand lines of code. The learning curve must
not have bumps along that road, because with every bump, a substantial
fraction of the user base will be deterred from progressing to more
complicated tasks.
3) You'll need good support for large-scale programming if you want to
enable "just calling" into third-party modules (which would be pretty
appealing to people who use Calc for nontrivial stuff). The JVM excels
at this, BTW, though if you don't use Java, a substantial fraction of
Maven modules will be awkward to use.

For (3), you'll need static typing.
For (1) - and arguably (2) and (3) as well - you need something that is
excellent at type inference.

Type inference does not work well for updatable data structures, so you
will want something that excels at handling immutable data. Which
essentially rules out C++ or any other imperative language.
The type system is a real problem. For many real-world situations you
need dependent types, but these have complicated error situations so
newbies will usually be unable to deal with it, which means a bump in
the learning curve. I don't know of a good way to deal with that, it's
just on the list of things that I routinely check if somebody asks me to
take a look at his great new language :-) (Haskell people excel at
bending its type system to simulate all kinds of things, and I wouldn't
be surprised if nobody had tried to achieve most dependent-type benefits
from Haskell's type system; however, people using such a type framework
will need to know the Haskell type system and the internals of the
framework to make sense of any error messages that come out of a type
bug, so this is a variant of programmer's golf, not something you want
for newbies and learners.)

The JVM would be desirable, but typical .jar modules make heavy use of
mutable data. So you need something that's alien to mutable data types
but not incompatible with then; that's a relative fine line that the
language design would have to strike.
I'd probably use a language that allows mutables but disallows aliases
to them. Clean does this via the type system, other strategies might be
work. However, given that arbitrary modules from the JVM ecosystem might
throw aliased updateable references left and right, all guarantees are
off as soon as a computation relies on data provided from a JVM module,
so maybe it's still not worth using that. (Java folks have been thinking
about "value types", i.e. immutables, for a while now, but I don't see
that coming any year soon.)

Evalutation strategy is another issue.
Non-strict has some nice properties, in particular you don't need to
differentiate between an expression and its value. However, nonstrict is
difficult to control performance-wise, and I still see people struggle
if they see that their code is unexpectedly slow.

> Note that a spreadsheet needs to give the satisfaction of immediate
> results, or failing immediate results, the sensation of actively
> working, so if the language could be interpreted that would be a huge help.

+1

> * Which combination of typed, compiled, interpreted, FOSS functional
> language that runs on the JVM, JAVA, Haskell, C++, C, used in that order
> of preference, makes the most sense for the Java compatible functional
> language at the top of the preference hierarchy?

C++ and C aren't worth it if you plan to go for the JVM.
Don't know if there's a useful JVM port of Haskell.

You *can* interface with OO using XML descriptors and such, so the JVM
isn't your only option. You could even use (or invent) a language that
compiles to binary, with LLVM that has become a realistic option.
However, plan to invest some time into understanding LLVM.

> Note also, that I have only the equivalent of an AA degree from a CIS,
> not a CS, perspective, so the odds are the whole idea is vaporware,
> unless I can determine feasibility and desirability,

I suspect that no existing language fits the bill.
Given your background, you'll need somebody with language design
experience; language design is hard because of so many conflicting goals.

 > then sweet talk
> real developers to help out.

That's a good plan :-)
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

trent shipley
In reply to this post by trent shipley
ASCII examples (view in monospaced font):

NormalSheet
-------------------------
  | A               | B |
-------------------------
1 | 1               |   |
-------------------------
2 | 2               |   |
--------------------------
3 | 3               |   |
--------------------------
4 | 4               |   |
-------------------------
5 | =SUM(A1:A4)     |   | (answer is 10)
-------------------------
6 | =myFunction(A5) |   |
-------------------------

==================================
A new sheet
==================================

myFunction (it has the "function sheet" property)
----------------------------
  |  A                 | B |
----------------------------
1 | =PARAMETER(double) |   |
----------------------------
2 | = A1 * A1          |   |
----------------------------
3 | =RETURN(A2)        |   |
----------------------------

Copy to CSV

NormalSheet

1
2
3
4
5
=SUM(A1:A5)

=MY_FUNCTION(A6)

Copy second sheet to CSV

MY_FUNCTION (a function sheet)

=PARAMETER(double)

=A1*A1

=RETURN(A3)

On Sun, Nov 19, 2017 at 12:05 AM trent shipley <[hidden email]> wrote:
I have a vision for a spreadsheet that is programmable as a spreadsheet itself. (Excel, for example, is programmable in VBA, but VBA requires switching modes to the VBA environment and using a non-spreadsheet idiom. Excel is not self-programmable as a spreadsheet.)

First there one function in a cell or array can reference other cells and arrays, including those that contain functions themselves. Second, IF(boolean,then,else) functions provide for selection. 

Third, the Function local scope property for a sheet in a workbook, the PARAMETER() function, and the RETURN() function allow the spreadsheet programmer to write new functions, including recursive functions.

* Is a spreadsheet you can program from the spreadsheet a reasonable goal?
* Has it been done?
(I don't have the academic CS chops to know what keywords to use, and how search CS publications in English.)

I have written a 30 page essay on Scriptsheets, but the essence boils down to the above. (If you, for some inexplicable reason, actually want the draft of the essay I'd be more than happy to share it with the understanding that you would provide feedback).  I call the proposal Scriptsheets.

So the plan is to take something like GNUmeric or LibreOffice Calc and graft on a primitive function sheet interpreter.

It would be natural to use C++, but the astute will note that a spreadsheet basically does not rewrite cells (unless you use a circular reference), so I'd also like to use a functional language, maybe Haskel.

* Would using a functional language as a basic language of the project save effort and intellectual load?

In the longer term I'd like as much of the spreadsheet programmable as a spreadsheet to be written to run on the JVM.

As near as I can tell near future Java and typed functional languages, include the following options:

Eta,
Frege,
Kotlin and,
Scala.

Note that a spreadsheet needs to give the satisfaction of immediate results, or failing immediate results, the sensation of actively working, so if the language could be interpreted that would be a huge help.

* Which combination of typed, compiled, interpreted, FOSS functional language that runs on the JVM, JAVA, Haskell, C++, C, used in that order of preference, makes the most sense for the Java compatible functional language at the top of the preference hierarchy? 

Note also, that I have only the equivalent of an AA degree from a CIS, not a CS, perspective, so the odds are the whole idea is vaporware, unless I can determine feasibility and desirability, then sweet talk real developers to help out.

[hidden email] (junk email address)

See also the spreadsheet at http://orgmode.org/  it's cool.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

trent shipley
In reply to this post by Joachim Durchholz
Middle posting follows:

First there are a few things going on.

There is the language or languages used by the developers of the spreadsheet product. There is the language of the spreadsheet itself. There are the programming languages used by the developers of the spreadsheet (say accounting power users), like VBA.

Case A:
So for Excel:

Implementation Language: MS C++ (my guess), 
Spreadsheet Language: Excel, 
Built-in Language VBA. (Excel HAS a programming language and tool.)

Case B:
For Emacs OrgMode spreadsheets (a weird little spreadsheet in a tool whose main purpose is hardly being a spreadsheet.)

Imlementation Language: Emacs Lisp (my guess),
Spreadsheet languages: Emacs Calc, and Emacs Lisp,
Built-in Language: I'm not sure it has one. Given the fact that you can write ELisp in the cells, it may not need one. However, it also doesn't look like a full-blown 3d spreadsheet.

Case C:
For the proposed Scriptsheets product:

Implementation Languages: Frege (?), Java, Haskell, C++, C, in that order.
Spreadsheet languages: Scriptsheets,
Bulit-in language: Scriptsheets. (Scriptsheets IS a programming language and tool.)

Note that a spreadsheet product can have programming languages, it can be a programming language, and it can interpret (or very quickly compile) programming statements in a language such as ELisp or Haskell within the cells of the spreadsheet itself. The features are not mutually exclusive.

I meant to imply that I was interested in Case C, and I believe Joachim understood me to be interested in Case B (but not Emacs OrgMode as a specific example of Case B).

Middle posting follows.


On Sun, Nov 19, 2017 at 5:07 AM Joachim Durchholz <[hidden email]> wrote:
Am 19.11.2017 um 08:05 schrieb trent shipley:
> * Is a spreadsheet you can program from the spreadsheet a reasonable goal?

I.e. use the same programming language for cells formulae and scripts?
Yes, that's very much reasonable.

That is possible.

However, I am interested in programming scripts in spreadsheet, not say, programming formula in Haskell.
 

> * Has it been done?

Not in any mainstream spreadsheet. Which boils down to two: Excel and
Open/Libre Office Calc.
Excel offers VBA.
Calc is actually language-agnostic and uses URLs and XML to tie things
together. It currently supports BeanShell (Java-without-types, it
seems), Java, JS, Python, and OO Basic.

> So the plan is to take something like GNUmeric or LibreOffice Calc and
> graft on a primitive function sheet interpreter.

The main point is that you'd have to replace the formular language.
I do not think that Calc was made for that.

> It would be natural to use C++,

You'd instantly kill adoption with that. Only a minimal part of the Calc
user base is even remotely capable of coding formulae in C++, and even
of these, a substantial fraction would be able but unwilling.

Agreed. I was not thinking of programming cells in C++. I was thinking of moving Scriptsheets from vaporware to proof of concept and beyond using C++ by the PROJECT developers (in large measure to appropriate existing FOSS code).

 
 > but the astute will note that a
> spreadsheet basically does not rewrite cells (unless you use a circular
> reference), so I'd also like to use a functional language, maybe Haskel.

You don't want to inline the formulae of other cells anyway, because
then the calculation will be done twice: Once to fill the referenced
cell, and once as part of the referencing cell.

So for reference cycles, you'll rely on whatever the spreadsheet is
already doing to deal with them.

Exactly, at least until your forked product goes from, say, version 1.0 to version 2.0.
 
> * Would using a functional language as a basic language of the project
> save effort and intellectual load?

That depends on whether you're talking about the implementation language
or the cell/macro language.

I was talking about implementation language.

 
For the implementation language, you'll save the most time by using
whatever you already know. Unless the project is going to last longer
than, say, two years. And if you plan on getting other people to join
the project, you'll want the language with the largest pool of
interested and able people, which is essentially guesswork but I'd
avoid, say, the VBA or PHP crowd ;-)

Would there be any advantage over preferring a functional over an imperative language given that Scriptsheets (as a full blown product) would itself be a functional language.
 
For the user-facing language, do whatever is easy to use for a
non-programmer. Haskell should work fine, but prepare to collect
references to tutorials, and what does *not* work fine is performance
predictions, particularly not for nonprogrammers. This probably means
you need strict evaluation, which means even if it looks like Haskell
it's going to be an entirely different language.

> In the longer term I'd like as much of the spreadsheet programmable as a
> spreadsheet to be written to run on the JVM.
> As near as I can tell near future Java and typed functional languages,
> include the following options:
>
> Eta,
> Frege,
> Kotlin and,
> Scala.

I don't know any of these well enough to make any recommendations.

However, for the user-facing language, you need this:
1) As easy to learn as possible.
2) Scales well to a few thousand lines of code. The learning curve must
not have bumps along that road, because with every bump, a substantial
fraction of the user base will be deterred from progressing to more
complicated tasks.
3) You'll need good support for large-scale programming if you want to
enable "just calling" into third-party modules (which would be pretty
appealing to people who use Calc for nontrivial stuff). The JVM excels
at this, BTW, though if you don't use Java, a substantial fraction of
Maven modules will be awkward to use.

For (3), you'll need static typing.
For (1) - and arguably (2) and (3) as well - you need something that is
excellent at type inference.

Type inference does not work well for updatable data structures, so you
will want something that excels at handling immutable data. Which
essentially rules out C++ or any other imperative language.
The type system is a real problem. For many real-world situations you
need dependent types, but these have complicated error situations so
newbies will usually be unable to deal with it, which means a bump in
the learning curve. I don't know of a good way to deal with that, it's
just on the list of things that I routinely check if somebody asks me to
take a look at his great new language :-) (Haskell people excel at
bending its type system to simulate all kinds of things, and I wouldn't
be surprised if nobody had tried to achieve most dependent-type benefits
from Haskell's type system; however, people using such a type framework
will need to know the Haskell type system and the internals of the
framework to make sense of any error messages that come out of a type
bug, so this is a variant of programmer's golf, not something you want
for newbies and learners.)

The JVM would be desirable, but typical .jar modules make heavy use of
mutable data. So you need something that's alien to mutable data types
but not incompatible with then; that's a relative fine line that the
language design would have to strike.
I'd probably use a language that allows mutables but disallows aliases
to them. Clean does this via the type system, other strategies might be
work. However, given that arbitrary modules from the JVM ecosystem might
throw aliased updateable references left and right, all guarantees are
off as soon as a computation relies on data provided from a JVM module,
so maybe it's still not worth using that. (Java folks have been thinking
about "value types", i.e. immutables, for a while now, but I don't see
that coming any year soon.)

Evalutation strategy is another issue.
Non-strict has some nice properties, in particular you don't need to
differentiate between an expression and its value. However, nonstrict is
difficult to control performance-wise, and I still see people struggle
if they see that their code is unexpectedly slow.

> Note that a spreadsheet needs to give the satisfaction of immediate
> results, or failing immediate results, the sensation of actively
> working, so if the language could be interpreted that would be a huge help.

+1

> * Which combination of typed, compiled, interpreted, FOSS functional
> language that runs on the JVM, JAVA, Haskell, C++, C, used in that order
> of preference, makes the most sense for the Java compatible functional
> language at the top of the preference hierarchy?

C++ and C aren't worth it if you plan to go for the JVM.
Don't know if there's a useful JVM port of Haskell.


I know Hadoop in in Java, except where speed is of essence, then Hadoop uses C++ or sometimes C. I figure the same would be true for this project.

For Haskell on JVM see Eta and Frege. Frege is the older project. Neither seem to be particularly mature.
 
You *can* interface with OO using XML descriptors and such, so the JVM
isn't your only option. You could even use (or invent) a language that
compiles to binary, with LLVM that has become a realistic option.
However, plan to invest some time into understanding LLVM.


 
> Note also, that I have only the equivalent of an AA degree from a CIS,
> not a CS, perspective, so the odds are the whole idea is vaporware,
> unless I can determine feasibility and desirability,

I suspect that no existing language fits the bill.
Given your background, you'll need somebody with language design
experience; language design is hard because of so many conflicting goals.

 > then sweet talk
> real developers to help out.

That's a good plan :-)

Do you know any gullible developers with the chops?

Which do you see as more promising a real GUI spreadsheet for Case B or one for Case C? Or would it make sense to be ambitious and try to do both in the same project using a common spreadsheet core backend and largely shared front end?
 
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

J. Garrett Morris-5
In reply to this post by trent shipley
On Sun, Nov 19, 2017 at 9:41 AM, trent shipley <[hidden email]> wrote:

> ==================================
> A new sheet
> ==================================
>
> myFunction (it has the "function sheet" property)
> ----------------------------
>   |  A                 | B |
> ----------------------------
> 1 | =PARAMETER(double) |   |
> ----------------------------
> 2 | = A1 * A1          |   |
> ----------------------------
> 3 | =RETURN(A2)        |   |
> ----------------------------

You might find Simon PJ's classic paper on Excel as a functional
language relevant:

https://www.microsoft.com/en-us/research/publication/a-user-centred-approach-to-functions-in-excel/

 /g
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Joachim Durchholz
In reply to this post by trent shipley
> There is the language or languages used by the developers of the
> spreadsheet product. There is the language of the spreadsheet itself.

Not sure what you mean with "language of the spreadsheet iself".
The formula language?

> There are the programming languages used by the developers of the
> spreadsheet (say accounting power users), like VBA.

> Note that a spreadsheet product can have programming languages, it can
> be a programming language,

A spreadsheet is not a programming language, so you must be meaning
something different, but I have no idea what.

> On Sun, Nov 19, 2017 at 5:07 AM Joachim Durchholz <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Am 19.11.2017 um 08:05 schrieb trent shipley:
>      > * Is a spreadsheet you can program from the spreadsheet a
>     reasonable goal?
>
>     I.e. use the same programming language for cells formulae and scripts?
>     Yes, that's very much reasonable.
>
>
> That is possible.
>
> However, I am interested in programming scripts in spreadsheet, not say,
> programming formula in Haskell.

I have no idea what your plan is, then.

>      > * Would using a functional language as a basic language of the
>     project
>      > save effort and intellectual load?
>
>     That depends on whether you're talking about the implementation language
>     or the cell/macro language.
>
> I was talking about implementation language.

If you plan to create a new spreadsheet program, that's a huge project.
Even if you stick to cells and formulae and skip everything on the
presentation side.

>     For the implementation language, you'll save the most time by using
>     whatever you already know. Unless the project is going to last longer
>     than, say, two years. And if you plan on getting other people to join
>     the project, you'll want the language with the largest pool of
>     interested and able people, which is essentially guesswork but I'd
>     avoid, say, the VBA or PHP crowd ;-)
>
> Would there be any advantage over preferring a functional over an
> imperative language given that Scriptsheets (as a full blown product)
> would itself be a functional language.

The choice of the programming language for building a new product is
pretty independent of the product itself. This is because the logic that
you see (cell calculations) is just a very small part of the overall code.
For a new spreadsheet program, look for a language that has a good GUI
library, for example, that's going to be a much larger timesaver than
language semantics that matches the application domain.

> I know Hadoop in in Java, except where speed is of essence, then Hadoop
> uses C++ or sometimes C. I figure the same would be true for this project.

Only once the project has seen several person-decades of work.

>       > then sweet talk
>      > real developers to help out.
>
>     That's a good plan :-)
>
>
> Do you know any gullible developers with the chops?

Actually I'd be interested in something like this if I didn't have my
own project ;-P

> Which do you see as more promising a real GUI spreadsheet for Case B or
> one for Case C? Or would it make sense to be ambitious and try to do
> both in the same project using a common spreadsheet core backend and
> largely shared front end?

I haven't really understood what you're trying to do, so I can'd advise.

Regards,
Jo
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Simon  Thompson
In reply to this post by trent shipley
Trent, 


You might be interested in this article on Interactive Visual Functional Programming, from Keith Hanna (formerly of University of Kent), from ICFP 2002


Abstract: An interactive graphical environment for supporting the development and use of Haskell applications programs is described. The environment, named Vital, is particularly intended for supporting the open-ended, incremental development style often preferred by non-specialist users in which successive steps of program development are motivated and informed by results so far obtained.Significant features of Vital include: the graphical display of data structures in a format defined by a datatype-indexed stylesheet, the way that evaluation of (possibly infinite) values is demand-driven by the action of the user scrolling around an unbounded workspace, and support for copy-and-paste graphical editing of data structures. This latter allows, for example, the user to modify a complex data structure by point-and-click operations, or to create (by functional evaluation) a regular data structure and then edit values or expressions into it. The effect of each editing operation is immediately reflected in the Haskell program source code.

Simon


On 19 Nov 2017, at 07:05, trent shipley <[hidden email]> wrote:

I have a vision for a spreadsheet that is programmable as a spreadsheet itself. (Excel, for example, is programmable in VBA, but VBA requires switching modes to the VBA environment and using a non-spreadsheet idiom. Excel is not self-programmable as a spreadsheet.)

First there one function in a cell or array can reference other cells and arrays, including those that contain functions themselves. Second, IF(boolean,then,else) functions provide for selection. 

Third, the Function local scope property for a sheet in a workbook, the PARAMETER() function, and the RETURN() function allow the spreadsheet programmer to write new functions, including recursive functions.

* Is a spreadsheet you can program from the spreadsheet a reasonable goal?
* Has it been done?
(I don't have the academic CS chops to know what keywords to use, and how search CS publications in English.)

I have written a 30 page essay on Scriptsheets, but the essence boils down to the above. (If you, for some inexplicable reason, actually want the draft of the essay I'd be more than happy to share it with the understanding that you would provide feedback).  I call the proposal Scriptsheets.

So the plan is to take something like GNUmeric or LibreOffice Calc and graft on a primitive function sheet interpreter.

It would be natural to use C++, but the astute will note that a spreadsheet basically does not rewrite cells (unless you use a circular reference), so I'd also like to use a functional language, maybe Haskel.

* Would using a functional language as a basic language of the project save effort and intellectual load?

In the longer term I'd like as much of the spreadsheet programmable as a spreadsheet to be written to run on the JVM.

As near as I can tell near future Java and typed functional languages, include the following options:

Eta,
Frege,
Kotlin and,
Scala.

Note that a spreadsheet needs to give the satisfaction of immediate results, or failing immediate results, the sensation of actively working, so if the language could be interpreted that would be a huge help.

* Which combination of typed, compiled, interpreted, FOSS functional language that runs on the JVM, JAVA, Haskell, C++, C, used in that order of preference, makes the most sense for the Java compatible functional language at the top of the preference hierarchy? 

Note also, that I have only the equivalent of an AA degree from a CIS, not a CS, perspective, so the odds are the whole idea is vaporware, unless I can determine feasibility and desirability, then sweet talk real developers to help out.

[hidden email] (junk email address)

See also the spreadsheet at http://orgmode.org/  it's cool.
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

Simon Thompson | Professor of Logic and Computation 
School of Computing | University of Kent | Canterbury, CT2 7NF, UK
[hidden email] | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Olaf Klinke
In reply to this post by trent shipley
Hello Trent,

let me suggest to step back and think about why people use spreadsheets. They provide access to data analysis functions, statistics and graphing that is otherwise missing (at least on) Windows systems. Serious number crunching is not done in spreadsheets, though. There is R, Matlab and the like. You should not try to comptete with that. The big selling point (in my eyes) of spreadsheets over scripting languages like R is that the data as well as the formulae and intermediate results are, well, spread out in front of your eyes. That is e.g. desirable when you need someone else to understand what you did and how you arrived at the answer given. But is it important that the intermediate result resides in cells C2 to C13? Certainly not.
I wrote a small financial application in Haskell that would otherwise be done in spreadsheets. Haskell provided, via its type system, extra sanity checks and much more that spreadsheets don't provide. However, I often wished that there was a way to disassemble the computation into logical bits that can be formatted neatly and be presented to a human inspector. Haskell provides ways to do this, e.g. by mixing Haskell and LaTeX, or writing a small DSL with two interpreters, one for calculations, one for layout.
So instead of enhancing a spreadsheet format I'd rather have a scripting language with a workbook file format à la Mathematica, where comments, code and results of computations are displayed together. Changing a formula triggers change in the results. (The workbooks I've seen don't do that.) Input data and intermediate results can be hidden or expanded via mouse click. Results can be traced back through the workbook's formulae up to the input data. (The latter is not always easy in spreadsheets.) One would need to research whether such a language exists. If not, I'd be happy to help develop it.

Olaf

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

trent shipley
Hi Olaf,

I must say you haven't convinced me that there is no use for a spreadsheet functional language. Nor have you convinced that important, mission critical work is not done in spreadsheets.

However, what you want certainly seems worthwhile.  Where can I see an example of what Mathematica does. It costs the Earth, and I've never played with it.

For simplicity I will assume serial code, not parallel.

Code is linear in almost all programming models. Code flows left to right and down the page (screen, at least in English). 

So what I imagine you want is something like a persistent code tracer, that by default reruns the code as changes are made.

      | Input | Code      | Result | Comment | LaTeX  | Etc  | Many Possible Columns
----------------------------------------------------------------------------
0001   | input1 | expression1 | result1 | comment1 | formula1 | etc1 |
-----------------------------------------------------------------------
And so on to the end of the program.

Like so?
A tabular view of the program and it's execution. You can program in the table. The table auto-updates when you change stuff. Works best with LOTS of screen real estate.

That would be kinda cool.

You might want to refine your idea and start another thread about that.  It's cool, but it's not a functional spreadsheet like in https://www.microsoft.com/en-us/research/publication/a-user-centred-approach-to-functions-in-excel/ and some of the article's citations, which is very much what I'm talking about. It looks like I could have plagiarized most of my ideas from that article, even though I've never read it before. (I was aware of comparisons of spreadsheets to functional programming before.) 


On Sun, Nov 19, 2017 at 1:39 PM Olaf Klinke <[hidden email]> wrote:
Hello Trent,

let me suggest to step back and think about why people use spreadsheets. They provide access to data analysis functions, statistics and graphing that is otherwise missing (at least on) Windows systems. Serious number crunching is not done in spreadsheets, though. There is R, Matlab and the like. You should not try to comptete with that. The big selling point (in my eyes) of spreadsheets over scripting languages like R is that the data as well as the formulae and intermediate results are, well, spread out in front of your eyes. That is e.g. desirable when you need someone else to understand what you did and how you arrived at the answer given. But is it important that the intermediate result resides in cells C2 to C13? Certainly not.
I wrote a small financial application in Haskell that would otherwise be done in spreadsheets. Haskell provided, via its type system, extra sanity checks and much more that spreadsheets don't provide. However, I often wished that there was a way to disassemble the computation into logical bits that can be formatted neatly and be presented to a human inspector. Haskell provides ways to do this, e.g. by mixing Haskell and LaTeX, or writing a small DSL with two interpreters, one for calculations, one for layout.
So instead of enhancing a spreadsheet format I'd rather have a scripting language with a workbook file format à la Mathematica, where comments, code and results of computations are displayed together. Changing a formula triggers change in the results. (The workbooks I've seen don't do that.) Input data and intermediate results can be hidden or expanded via mouse click. Results can be traced back through the workbook's formulae up to the input data. (The latter is not always easy in spreadsheets.) One would need to research whether such a language exists. If not, I'd be happy to help develop it.

Olaf


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Brandon Allbery
On Sun, Nov 19, 2017 at 4:50 PM, trent shipley <[hidden email]> wrote:
I must say you haven't convinced me that there is no use for a spreadsheet functional language. Nor have you convinced that important, mission critical work is not done in spreadsheets.

I am pretty sure you're talking past each other at this point. Step back and take a deep breath, both of you.

--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

trent shipley
Hi Brandon, Olaf,

I did not mean to be a jerk. I think Olaf has an itch and a good idea of how to scratch it, and it could be the basis for a great project. I've never used Mathematica, so I have a hard time visualizing what he is driving at. My apologies if I was inappropriate. 

It is reasonable to ask whether there would be a reasonable user base for a functional programming language/spreadsheet. My intuition is that there would be among more advanced spreadsheet users. So that is obviously a point of disagreement, but that is OK. We should probably debate that.

I also think he is talking about a different thing than I am, and what this thread has been apart thus far. It is a Bad Thing (in the Winnie the Pooh sense) when threads turn into fibers. I think Olaf's idea/itch is actually quite good, and deserves a thread of its own.

With apologies,

Trent.


On Sun, Nov 19, 2017 at 4:38 PM Brandon Allbery <[hidden email]> wrote:
On Sun, Nov 19, 2017 at 4:50 PM, trent shipley <[hidden email]> wrote:
I must say you haven't convinced me that there is no use for a spreadsheet functional language. Nor have you convinced that important, mission critical work is not done in spreadsheets.

I am pretty sure you're talking past each other at this point. Step back and take a deep breath, both of you.


--
brandon s allbery kf8nh                               sine nomine associates
[hidden email]                                  [hidden email]
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Pierpaolo Bernardi
People interested in implementing spreadsheets could give a look at
"Spreadsheet Implementation Technology: Basics and Extensions", Peter
Sestoft, MIT Press.

It's the only book on the topic, as far as I know, and it's an
interesting reading (even if they use C#, but we can't have
everything).
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Dominic Steinitz-2
In reply to this post by trent shipley
Here’s a presentation about a functional language embedded in Excel which I used when I worked at an investment bank

https://www.youtube.com/watch?v=uICBMTeIl2U

http://cufp.org/2009/fmd-functional-development-excel.html

As far as I know the code is still in use (I recall there was something like 400 users in the bank) and was never open-sourced but it may provide some ideas given it was pretty successful internally.

Dominic Steinitz
[hidden email]
http://idontgetoutmuch.wordpress.com

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Sergiu Ivanov-2
In reply to this post by trent shipley
Hello Trent,

Thus quoth  trent shipley  on Sun Nov 19 2017 at 12:12 (+0100):

> On Sun, Nov 19, 2017 at 3:40 AM Sergiu Ivanov <[hidden email]> wrote:
>> Thus quoth  trent shipley  on Sun Nov 19 2017 at 08:05 (+0100):
>> >
>> > I have a vision for a spreadsheet that is programmable as a spreadsheet
>> > itself.
>> [...]
>> > * Is a spreadsheet you can program from the spreadsheet a reasonable
>> > goal?
>>
>> What kind of use cases do you imagine for your programmable spreadsheet?
>> "Reasonable" will usually depend on your context.
>
> I really haven't thought of specific use cases, but I have thought in
> general terms about what you might do with a natively programmable
> spreadsheet.
>
>    1. It would be an interesting exercise in itself, and if it hadn't been
>    done before, might be worth a paper. (I submit that this is actually a real
>    use case.)
That's a good and a fun use case, but it's probably not going to attract
a lot of developers/maintainers.

>    2. It would be more elegant that the current solution to a programmable
>    spreadsheet, which is a spreadsheet that is almost, but not quite
>    programmable, with a helper imperative language and programming mode stuck
>    onto the base spreadsheet.

Well, as Olaf points out in another thread (I think), the fact that
conventional spreadsheets are not completely programmable is also an
advantage: you know what your spreadsheet _cannot_ do, which helps with
debugging.

I do admit that's not an argument against programmable spreadsheets,
though.

>    3. You could build a programmable spreadsheet on steroids with this
>    idea, and I have some other interesting ideas, like "virtual spreadsheets"
>    that would allow spreadsheets to extend into n dimensions (but I digress).
>    4. So engineers, scientists, economists, and finance might have niche
>    uses for a (n-dimensional) programmable spreadsheet on steroids--though I
>    don't know in detail what those would be.
>    5. I am a trained social scientist (anthropology) I know it is not
>    uncommon for social scientists to have to deal with large, multidimensional
>    datasets, and analyze the same. A multidimensional programmable spreadsheet
>    might help a lot with that.
Multidimensionality seems to be a really important point in your
project.  On the other hand, without having any specialised knowledge in
spreadsheet-crunching, I'm tempted to suppose that what these scientist
need is quite below Turing completeness, once multidimensionality is
provided.

Do you think a semi-programmable (as in LibreOffice) multidimensional
spreadsheet may cut it for them?

>    6. I have heard that finance wizards often use Excel+VBA to do modeling.
>    They might like a consistent, single language programming environment that
>    offers more power and type safety that Excel + VBA. It would also need to
>    be FAST if it is to be used for something like market analysis or automated
>    trading.

Your argument is probably based on way more experience than mine, but
personally I don't at all aspire to have a homogeneous language
environment.  I actually like the separation in Excel: formulas for
almost any operations; VBA when real muscle-power is inevitable.

That was my two-cent addition to the brainstorming :-)

> Finding plausible use cases is also hindered by both a version of an
> existing spreadsheet enabling native functions, or a more full featured
> Scriptsheets product both being vaporware.

Not necessarily, as you showed in the message I am replying to :-)

My reasoning behind suggesting to consider potential applications is
that the abstract model you are mulling over is very general and can be
implemented in various quite different ways—e.g., LibreOffice, Scilab,
and the R language all focus on matrix processing (more or less), but
they are so different that people seem to agree they are not really
directly comparable, and thus not conflicting.

Now, I may be mistaken and you may have a very clear idea of an
implementation, in which case my suggestions are simply redundant.

>> > It would be natural to use C++
>>
>> Using C++ looks in no way natural _to me_; "natural" will also depend on
>> your use case ;-)
>>
>
> The main argument for C++ is that if you are "reusing" existing code from
> GNUmeric or LibreOffice Calc none of that is in a functional language. It
> will tend to be in C++ (and maybe Java for Calc).

Are you talking about the language for spreadsheet programming or about
the implementation language for your project?

If we are in the former case, the implementation language need not
necessarily be the same as the language of the spreadsheet, like in the
HotSpot JVM which seems to be written in C++ [0] even though it runs
tons of other languages or Excel itself, which I doubt is written in
VBA.

> Also, if speed is of the essence, my impression is that C++ and C come to
> the fore.

Yes, but overoptimisation along one criterion among many is also an
issue.  The trade-off between faster code or code easier to maintain is
a difficult one.

But then you're are right, C++ and C are often quite in the lead in what
concerns speed (even though some tweaks may bring other programming
languages quite close [1]).

--
Sergiu

[0] https://en.wikipedia.org/wiki/HotSpot

[1] http://paulspontifications.blogspot.fr/2013/01/when-haskell-is-faster-than-c.html

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Neil Mayhew
In reply to this post by Olaf Klinke

On 2017-11-19 01:39 PM, Olaf Klinke wrote:

So instead of enhancing a spreadsheet format I’d rather have a scripting language with a workbook file format à la Mathematica, where comments, code and results of computations are displayed together. Changing a formula triggers change in the results. (The workbooks I’ve seen don’t do that.) Input data and intermediate results can be hidden or expanded via mouse click. Results can be traced back through the workbook’s formulae up to the input data. (The latter is not always easy in spreadsheets.)

Olaf, have you looked at Jupyter at all? This now has a backend for Haskell and supports many of the things you mention.


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

trent shipley
In reply to this post by Sergiu Ivanov-2
Middle posting follows.

On Mon, Nov 20, 2017 at 7:23 AM Sergiu Ivanov <[hidden email]> wrote:
Hello Trent,

Thus quoth  trent shipley  on Sun Nov 19 2017 at 12:12 (+0100):
> On Sun, Nov 19, 2017 at 3:40 AM Sergiu Ivanov <[hidden email]> wrote:
>> Thus quoth  trent shipley  on Sun Nov 19 2017 at 08:05 (+0100):
>> >
>> > I have a vision for a spreadsheet that is programmable as a spreadsheet
>> > itself.
>> [...]
>> > * Is a spreadsheet you can program from the spreadsheet a reasonable
>> > goal?
>>
>> What kind of use cases do you imagine for your programmable spreadsheet?
>> "Reasonable" will usually depend on your context.
>
> I really haven't thought of specific use cases, but I have thought in
> general terms about what you might do with a natively programmable
> spreadsheet.
>
>    1. It would be an interesting exercise in itself, and if it hadn't been
>    done before, might be worth a paper. (I submit that this is actually a real
>    use case.)

That's a good and a fun use case, but it's probably not going to attract
a lot of developers/maintainers.

No it wouldn't I suppose. It would be a way to get it started, and to explore new possibilities, but the academic use case certainly won't produce stability. Actually a few fully functional spreadsheets have been developed, mostly to explore the spreadsheet as a visual programming tool, or human computer interaction. "Spreadsheet Implementation Technology: Basics and Extensions", Peter
Sestoft, MIT Press. may provide fully functional spreadsheets for hacking (maybe based on Forms/3 http://web.engr.oregonstate.edu/~burnett/Forms3/forms3.html, at least in part. Forms/3 was M Burrnett's spreadsheet for HCI research.)
 

>    2. It would be more elegant that the current solution to a programmable
>    spreadsheet, which is a spreadsheet that is almost, but not quite
>    programmable, with a helper imperative language and programming mode stuck
>    onto the base spreadsheet.

Well, as Olaf points out in another thread (I think), the fact that
conventional spreadsheets are not completely programmable is also an
advantage: you know what your spreadsheet _cannot_ do, which helps with
debugging.

I do admit that's not an argument against programmable spreadsheets,
though.

>    3. You could build a programmable spreadsheet on steroids with this
>    idea, and I have some other interesting ideas, like "virtual spreadsheets"
>    that would allow spreadsheets to extend into n dimensions (but I digress).
>    4. So engineers, scientists, economists, and finance might have niche
>    uses for a (n-dimensional) programmable spreadsheet on steroids--though I
>    don't know in detail what those would be.
>    5. I am a trained social scientist (anthropology) I know it is not
>    uncommon for social scientists to have to deal with large, multidimensional
>    datasets, and analyze the same. A multidimensional programmable spreadsheet
>    might help a lot with that.

Multidimensionality seems to be a really important point in your
project.  On the other hand, without having any specialised knowledge in
spreadsheet-crunching, I'm tempted to suppose that what these scientist
need is quite below Turing completeness, once multidimensionality is
provided.


I should be more careful with the distinction between multidimensional spreadsheets and n-dimensional spreadsheets. Multidimensional spreadsheets like Lotus Improv and Lighthouse Design's Quantrix (based on Improv and ran on Steve Job's NextStep) were spreadsheets with a sophisticated way to model data that was called "multidimensional", that is not what I am referring to. 

Assume an additional level to the spreadsheet's organization between a workbook and a sheet called a "virtual workbook". A root (file) workbook can contain virtual workbooks or sheets. A virtual workbook can also contain virtual work books or sheets. Sheets are 3-dimensional, and then you're done, but virtual workbooks can be nested as deeply as desired, so a spreadsheet with virtual workbooks could model n-dimensions.

Also, if sheets can be marked as being functions, virtual workbooks can support libraries of functions, or objects, if you want such things.
 
Do you think a semi-programmable (as in LibreOffice) multidimensional
spreadsheet may cut it for them?

Might.

It might not for finance.


See also:

I have to admit I haven't poked around as much as I should before passing these references on.
 

>    6. I have heard that finance wizards often use Excel+VBA to do modeling.
>    They might like a consistent, single language programming environment that
>    offers more power and type safety that Excel + VBA. It would also need to
>    be FAST if it is to be used for something like market analysis or automated
>    trading.

Your argument is probably based on way more experience than mine, but
personally I don't at all aspire to have a homogeneous language
environment.  I actually like the separation in Excel: formulas for
almost any operations; VBA when real muscle-power is inevitable.

That was my two-cent addition to the brainstorming :-)

S Jones, M Burnett, and A Blackwell called this (disparagingly) "the trap door model". It has also been called the "bolt on" model. It's not that bad. It does require two sets of knowledge, and rather separate skill sets.  I'm really not that hostile to it. It works, but I do like to have more than one way to do things. A fully functional spreadsheet would be a nice alternative.

Programmers have to learn multiple languages, and most wind up liking it. They also have favorite languages. LibreOffice Calc has about four bolt-on languages. As I recall, none of them are a functional programming language. Maybe a few programmers should get together and add one as a sub-project.
 
> Finding plausible use cases is also hindered by both a version of an
> existing spreadsheet enabling native functions, or a more full featured
> Scriptsheets product both being vaporware.

Not necessarily, as you showed in the message I am replying to :-)

My reasoning behind suggesting to consider potential applications is
that the abstract model you are mulling over is very general and can be
implemented in various quite different ways—e.g., LibreOffice, Scilab,
and the R language all focus on matrix processing (more or less), but
they are so different that people seem to agree they are not really
directly comparable, and thus not conflicting.

Now, I may be mistaken and you may have a very clear idea of an
implementation, in which case my suggestions are simply redundant.

If implementation means "how to program it", I have no idea.

If implementation means refining a proposal, and leading into a functional spec, then, yes, I am working on that. Up to now mostly in my own head, which is why I'm seeking input, validation, and criticism. Depending, I'll go back and refine what I have.
 
>> > It would be natural to use C++
>>
>> Using C++ looks in no way natural _to me_; "natural" will also depend on
>> your use case ;-)
>>
>
> The main argument for C++ is that if you are "reusing" existing code from
> GNUmeric or LibreOffice Calc none of that is in a functional language. It
> will tend to be in C++ (and maybe Java for Calc).

Are you talking about the language for spreadsheet programming or about
the implementation language for your project?

Implementation.
 
If we are in the former case, the implementation language need not
necessarily be the same as the language of the spreadsheet, like in the
HotSpot JVM which seems to be written in C++ [0] even though it runs
tons of other languages or Excel itself, which I doubt is written in
VBA.

As far as I know, Access, Excel, PowerPoint, and Word, are all implemented in Microsoft C++.
 
> Also, if speed is of the essence, my impression is that C++ and C come to
> the fore.

Yes, but overoptimisation along one criterion among many is also an
issue.  The trade-off between faster code or code easier to maintain is
a difficult one.

Financiers' market trading code is notorious for being time critical, they are trying to beat other trading companies to the market.
 

But then you're are right, C++ and C are often quite in the lead in what
concerns speed (even though some tweaks may bring other programming
languages quite close [1]).

--
Sergiu

[0] https://en.wikipedia.org/wiki/HotSpot

[1] http://paulspontifications.blogspot.fr/2013/01/when-haskell-is-faster-than-c.html

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Joachim Durchholz
> I should be more careful with the distinction between multidimensional
> spreadsheets and n-dimensional spreadsheets. Multidimensional
> spreadsheets like Lotus Improv and Lighthouse Design's Quantrix (based
> on Improv and ran on Steve Job's NextStep) were spreadsheets with a
> sophisticated way to model data that was called "multidimensional", that
> is not what I am referring to.
>
> Assume an additional level to the spreadsheet's organization between a
> workbook and a sheet called a "virtual workbook". A root (file) workbook
> can contain virtual workbooks or sheets. A virtual workbook can also
> contain virtual work books or sheets. Sheets are 3-dimensional, and then
> you're done, but virtual workbooks can be nested as deeply as desired,
> so a spreadsheet with virtual workbooks could model n-dimensions.

Which one it is that you are interested in?
"Three-dimensional" as in "we have row, column, and layer", or as in "we
have hierarchical workbooks"?

> Also, if sheets can be marked as being functions, virtual workbooks can
> support libraries of functions, or objects, if you want such things.

Sometimes I think Excel etc. got it all wrong.
People routinely reserve areas inside the spreadsheet for different
tasks: input, intermediate results, output. Increasing the size of each
area becomes a pain.

I'd want to have a spreadsheet where it's easy to define multiple sheets
on the same "table". Extending the input sheet with another row
automatically extends the output sheet by a row (and if you have rollup
then we get more than one row). Intermediate sheets could be replaced by
a function definition.

It's a bit of an ergonomic challenge; functions are much more abstract,
and users will want to see how the results of applying a function to an
input area look like. So the mental model might be that we still have
intermediate sheets, but with just a single function, and the
intermediate sheets can be create, destroyed, shown and hidden easily.

The other challenge would be dealing with headers. Headers are
important, but for the intermediate we have just the function so it
needs to provide the headers. I.e. the function result would have to be
a named tuple, and if you want multiline headers, we get a hierarchical
named tuple.
Manipulating the column order would also manipulate the tuple
definition, i.e. you'd have to think about how GUI manipulation affects
the function definition.

There's pivot tables, so the difference between row and column isn't as
clear-cut as the above assumes.

There are a few more things to consider. It's firmly in the GUI
ergonomics area.
For this kind of stuff, you don't care whether the programming language
evaluation semantics matches the problem domain (spreadsheet functional
semantics is easy to do in any language, it was done in Assembler with
Lotus 1-2-3), you care whether it is easy to prototype GUI approaches,
which means GUI libraries that make it easy to create new widgets.

> Programmers have to learn multiple languages, and most wind up liking
> it. They also have favorite languages. LibreOffice Calc has about four
> bolt-on languages. As I recall, none of them are a functional
> programming language. Maybe a few programmers should get together and
> add one as a sub-project.

The problem is that any additional language needs to be installed. So
either the new language needs to be integrated in the LO project, or it
is not and anybody who passes on a spreadsheet in the new language needs
to tell the recipients how to install the plugin for the language.

Maybe that's the reason why only the four languages exist.

> Financiers' market trading code is notorious for being time critical,
> they are trying to beat other trading companies to the market.

Spreadsheets are irrelevant to high-speed trading, they'd be too slow
anyway.
I am working for a company that operates a stock exchange, and rumor has
it that the reaction times are down to 12 milli(!)seconds. Maybe it's
not reaction time but ping time and the computation latency is on top of
those 12 ms, but on a 4-GHz machine, 12 ms is 12,000 clock cycles, which
is a pretty tight timescale to process dozens or maybe thousands of
orders that have happened and post your own trade.

>     But then you're are right, C++ and C are often quite in the lead in what
>     concerns speed (even though some tweaks may bring other programming
>     languages quite close [1]).

I think one of the important factors is how much coder effort is needed
to get that kind of performance.
The trade-off seems to be how much coder effort is needed to get the
code correct.

 From what I hear:
C++ gives easy and predictable performance (if you code for that), while
being somewhat unpredictable on correctness.
Haskell excels at correctness, gives reasonable but not very predictable
performance.
Java is predictable with correctness though not excellent, and
predictable with performance though not excellent.
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Speculation, OT: Program a Spreadsheet

Olaf Klinke
In reply to this post by trent shipley
> Olaf, have you looked at Jupyter <http://jupyter.org/> at all? This now
> has a backend for Haskell and supports many of the things you mention.

Neil,

I must have forgotten about Jupyter. That is exactly what I was takling about! I have become quite a fan of literate programming. Now I admit that Jupyter looks quite different from spreadsheets, so wont' fit the bill of what Trent was originally asking for. As I tried to express earlier, the strength of spreadseets in my eyes is the somewhat explicit data flow. (You need to highlight a cell to see its formula.)
Hence I envision a Jupyter with data flow made explicit. You start by defining your input data. Single values, lists, matrices, whatever. You can define some functions, which will act as edges of your data flow graph. Then you combine these functions and apply to the data. For each (f.g)(x), all of x, g(x) and (f.g)(x) become nodes of a graph with (f.g)(x) hyperlinked to g(x) which in turn gets hyperlinked to x. Thus you can explore backwards from the result and see how the answer was derived.
Of course a nicely written Jupyter notebook needs no hyperlinks. One can leave enough explanation in the markup cells. But not everyone is trained to read and understand such mixture of text and code.

There might be an applicative functor (a variant of 'State SomeGraph') for what I'm after. Once I've defined it, I'll post it here.

Olaf
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
12