Re: Accelerating Automatic Differentiation

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Automatic Differentiation

MIMUW
Hi Charlie

It certainly looks like exciting project, but the bar is currently placed very high.
TensorFlow package not only provides automatic differentiation for whole programs, but also optimizes data processing both on GPU, and reading to achieve large batches.
This field has a lot of hot developments, so You would either need to propose something unique to Haskell, or You risk being outclassed by PyTorch and TensorFlow bindings
  Maybe Dominic suggests something too.
 Cheers
      Michal

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Automatic Differentiation

Charles Blake

Thanks for the response Michal,

Yes, this did cross my mind - and I wouldn't be expecting to outperform those frameworks in the timeframe available! I assumed that the reason that this project was suggested was perhaps:

a) there is some intrinsic value in implementing these algorithms natively in haskell (hence why the 'ad' library was developed in the first place), so that those who want to use parallel automatic differentiation / the machine learning algorithms built on top of it can do so without leaving the haskell ecosystem,

and b) because the challenges involved in implementing parallel ad in a purely functional language are a little different to those involved in doing so in OO/imperative languages - so it might be interesting from that angle as well?

So perhaps my aim would no be to do something unique, but rather to do something that has already done well in other languages, but has not yet been provided as a haskell library. Does this sound like a reasonable approach or do I need to find a slightly more unique angle?

Thanks,
Charlie


From: Michal J Gajda <[hidden email]>
Sent: 24 March 2018 16:56:35
To: Dominic Steinitz; Marco Zocca; [hidden email]; Charles Blake; [hidden email]
Subject: Re: Accelerating Automatic Differentiation
 
Hi Charlie

It certainly looks like exciting project, but the bar is currently placed very high.
TensorFlow package not only provides automatic differentiation for whole programs, but also optimizes data processing both on GPU, and reading to achieve large batches.
This field has a lot of hot developments, so You would either need to propose something unique to Haskell, or You risk being outclassed by PyTorch and TensorFlow bindings
  Maybe Dominic suggests something too.
 Cheers
      Michal

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Automatic Differentiation

Dominic Steinitz-2
The list of mentors for this project looks great to me. I am not sure if I can add much other than I think this is a nice project. Perhaps it would be best to get the advice of some of the mentors?

For some very simple tests with an ODE solver, I concluded that accelerate can perform at least as well as Julia. It would certainly be very helpful to be able to get Jacobians for ODE solving and for other applications.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 24 Mar 2018, at 17:20, Charles Blake <[hidden email]> wrote:

Thanks for the response Michal,

Yes, this did cross my mind - and I wouldn't be expecting to outperform those frameworks in the timeframe available! I assumed that the reason that this project was suggested was perhaps:

a) there is some intrinsic value in implementing these algorithms natively in haskell (hence why the 'ad' library was developed in the first place), so that those who want to use parallel automatic differentiation / the machine learning algorithms built on top of it can do so without leaving the haskell ecosystem,

and b) because the challenges involved in implementing parallel ad in a purely functional language are a little different to those involved in doing so in OO/imperative languages - so it might be interesting from that angle as well?

So perhaps my aim would no be to do something unique, but rather to do something that has already done well in other languages, but has not yet been provided as a haskell library. Does this sound like a reasonable approach or do I need to find a slightly more unique angle?

Thanks,
Charlie


From: Michal J Gajda <[hidden email]>
Sent: 24 March 2018 16:56:35
To: Dominic Steinitz; Marco Zocca; [hidden email]; Charles Blake; [hidden email]
Subject: Re: Accelerating Automatic Differentiation
 
Hi Charlie

It certainly looks like exciting project, but the bar is currently placed very high.
TensorFlow package not only provides automatic differentiation for whole programs, but also optimizes data processing both on GPU, and reading to achieve large batches.
This field has a lot of hot developments, so You would either need to propose something unique to Haskell, or You risk being outclassed by PyTorch and TensorFlow bindings
  Maybe Dominic suggests something too.
 Cheers
      Michal


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Automatic Differentiation

MIMUW
As a mentor I would say it is certainly possible to outperform exsting mega-solutions in some narrow domain.
Just as I did with hPDB https://hackage.haskell.org/package/hPDB
But it requires a lot of skill and patiece.

Please proceed with this project with the current list of mentors.
I think me and Dominic have already declared committment.

You might also start by making a table of best competing solutions in other. languages, their respective strengths, and ways that we can possibly improve on them!

Where do You keep Your application draft? Ideally it should be a shared space where you can add mentors as co-editors.

Cheers
Michal
On Sun, 25 Mar 2018 at 02:32, <[hidden email]> wrote:
The list of mentors for this project looks great to me. I am not sure if I can add much other than I think this is a nice project. Perhaps it would be best to get the advice of some of the mentors?

For some very simple tests with an ODE solver, I concluded that accelerate can perform at least as well as Julia. It would certainly be very helpful to be able to get Jacobians for ODE solving and for other applications.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 24 Mar 2018, at 17:20, Charles Blake <[hidden email]> wrote:

Thanks for the response Michal,

Yes, this did cross my mind - and I wouldn't be expecting to outperform those frameworks in the timeframe available! I assumed that the reason that this project was suggested was perhaps:

a) there is some intrinsic value in implementing these algorithms natively in haskell (hence why the 'ad' library was developed in the first place), so that those who want to use parallel automatic differentiation / the machine learning algorithms built on top of it can do so without leaving the haskell ecosystem,

and b) because the challenges involved in implementing parallel ad in a purely functional language are a little different to those involved in doing so in OO/imperative languages - so it might be interesting from that angle as well?

So perhaps my aim would no be to do something unique, but rather to do something that has already done well in other languages, but has not yet been provided as a haskell library. Does this sound like a reasonable approach or do I need to find a slightly more unique angle?

Thanks,
Charlie


From: Michal J Gajda <[hidden email]>
Sent: 24 March 2018 16:56:35
To: Dominic Steinitz; Marco Zocca; [hidden email]; Charles Blake; [hidden email]
Subject: Re: Accelerating Automatic Differentiation
 
Hi Charlie

It certainly looks like exciting project, but the bar is currently placed very high.
TensorFlow package not only provides automatic differentiation for whole programs, but also optimizes data processing both on GPU, and reading to achieve large batches.
This field has a lot of hot developments, so You would either need to propose something unique to Haskell, or You risk being outclassed by PyTorch and TensorFlow bindings
  Maybe Dominic suggests something too.
 Cheers
      Michal

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Automatic Differentiation

MIMUW
Given that Marco did not confirm, I am just now confirming that we can get you mentored by  Mikhail Baikov as the third mentor (beside Dominic and me).
Both me (Michal) and Mikhail are performance optimization experts (I am in parsers, and data analytics, Mikhail is in real-time systems, he has his own top-notch serialization library - Beamable that outperforms Cereal in both data size and speed). Dominic is expert in numerical computing (ODEs and Julia, among other things).

I believe that with these three excellent mentors you have very good chance to make outstanding contribution.
We just make sure that you prep application by 27th deadline.
--
  Cheers
    Michal

On Sun, Mar 25, 2018 at 6:32 AM Michal J Gajda <[hidden email]> wrote:
As a mentor I would say it is certainly possible to outperform exsting mega-solutions in some narrow domain.
Just as I did with hPDB https://hackage.haskell.org/package/hPDB
But it requires a lot of skill and patiece.

Please proceed with this project with the current list of mentors.
I think me and Dominic have already declared committment.

You might also start by making a table of best competing solutions in other. languages, their respective strengths, and ways that we can possibly improve on them!

Where do You keep Your application draft? Ideally it should be a shared space where you can add mentors as co-editors.

Cheers
Michal
On Sun, 25 Mar 2018 at 02:32, <[hidden email]> wrote:
The list of mentors for this project looks great to me. I am not sure if I can add much other than I think this is a nice project. Perhaps it would be best to get the advice of some of the mentors?

For some very simple tests with an ODE solver, I concluded that accelerate can perform at least as well as Julia. It would certainly be very helpful to be able to get Jacobians for ODE solving and for other applications.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 24 Mar 2018, at 17:20, Charles Blake <[hidden email]> wrote:

Thanks for the response Michal,

Yes, this did cross my mind - and I wouldn't be expecting to outperform those frameworks in the timeframe available! I assumed that the reason that this project was suggested was perhaps:

a) there is some intrinsic value in implementing these algorithms natively in haskell (hence why the 'ad' library was developed in the first place), so that those who want to use parallel automatic differentiation / the machine learning algorithms built on top of it can do so without leaving the haskell ecosystem,

and b) because the challenges involved in implementing parallel ad in a purely functional language are a little different to those involved in doing so in OO/imperative languages - so it might be interesting from that angle as well?

So perhaps my aim would no be to do something unique, but rather to do something that has already done well in other languages, but has not yet been provided as a haskell library. Does this sound like a reasonable approach or do I need to find a slightly more unique angle?

Thanks,
Charlie


From: Michal J Gajda <[hidden email]>
Sent: 24 March 2018 16:56:35
To: Dominic Steinitz; Marco Zocca; [hidden email]; Charles Blake; [hidden email]
Subject: Re: Accelerating Automatic Differentiation
 
Hi Charlie

It certainly looks like exciting project, but the bar is currently placed very high.
TensorFlow package not only provides automatic differentiation for whole programs, but also optimizes data processing both on GPU, and reading to achieve large batches.
This field has a lot of hot developments, so You would either need to propose something unique to Haskell, or You risk being outclassed by PyTorch and TensorFlow bindings
  Maybe Dominic suggests something too.
 Cheers
      Michal

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Automatic Differentiation

Dominic Steinitz-2
Hi Michal,

I didn’t volunteer to be a mentor for this. The project already lists:

Mentor: Fritz Henglein, Gabriele Keller, Trevor McDonell, Edward Kmett, Sacha Sokoloski

I doubt there is much I could add to such an illustrious list.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 25 Mar 2018, at 06:11, Michal J Gajda <[hidden email]> wrote:

Given that Marco did not confirm, I am just now confirming that we can get you mentored by  Mikhail Baikov as the third mentor (beside Dominic and me).
Both me (Michal) and Mikhail are performance optimization experts (I am in parsers, and data analytics, Mikhail is in real-time systems, he has his own top-notch serialization library - Beamable that outperforms Cereal in both data size and speed). Dominic is expert in numerical computing (ODEs and Julia, among other things).

I believe that with these three excellent mentors you have very good chance to make outstanding contribution.
We just make sure that you prep application by 27th deadline.
--
  Cheers
    Michal

On Sun, Mar 25, 2018 at 6:32 AM Michal J Gajda <[hidden email]> wrote:
As a mentor I would say it is certainly possible to outperform exsting mega-solutions in some narrow domain.
Just as I did with hPDB https://hackage.haskell.org/package/hPDB
But it requires a lot of skill and patiece.

Please proceed with this project with the current list of mentors.
I think me and Dominic have already declared committment.

You might also start by making a table of best competing solutions in other. languages, their respective strengths, and ways that we can possibly improve on them!

Where do You keep Your application draft? Ideally it should be a shared space where you can add mentors as co-editors.

Cheers
Michal
On Sun, 25 Mar 2018 at 02:32, <[hidden email]> wrote:
The list of mentors for this project looks great to me. I am not sure if I can add much other than I think this is a nice project. Perhaps it would be best to get the advice of some of the mentors?

For some very simple tests with an ODE solver, I concluded that accelerate can perform at least as well as Julia. It would certainly be very helpful to be able to get Jacobians for ODE solving and for other applications.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 24 Mar 2018, at 17:20, Charles Blake <[hidden email]> wrote:

Thanks for the response Michal,

Yes, this did cross my mind - and I wouldn't be expecting to outperform those frameworks in the timeframe available! I assumed that the reason that this project was suggested was perhaps:

a) there is some intrinsic value in implementing these algorithms natively in haskell (hence why the 'ad' library was developed in the first place), so that those who want to use parallel automatic differentiation / the machine learning algorithms built on top of it can do so without leaving the haskell ecosystem,

and b) because the challenges involved in implementing parallel ad in a purely functional language are a little different to those involved in doing so in OO/imperative languages - so it might be interesting from that angle as well?

So perhaps my aim would no be to do something unique, but rather to do something that has already done well in other languages, but has not yet been provided as a haskell library. Does this sound like a reasonable approach or do I need to find a slightly more unique angle?

Thanks,
Charlie


From: Michal J Gajda <[hidden email]>
Sent: 24 March 2018 16:56:35
To: Dominic Steinitz; Marco Zocca; [hidden email]; Charles Blake; [hidden email]
Subject: Re: Accelerating Automatic Differentiation
 
Hi Charlie

It certainly looks like exciting project, but the bar is currently placed very high.
TensorFlow package not only provides automatic differentiation for whole programs, but also optimizes data processing both on GPU, and reading to achieve large batches.
This field has a lot of hot developments, so You would either need to propose something unique to Haskell, or You risk being outclassed by PyTorch and TensorFlow bindings
  Maybe Dominic suggests something too.
 Cheers
      Michal


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Automatic Differentiation

MIMUW
Great! If they all agreed, we can just add Mikhail, and done?
On Sun, 25 Mar 2018 at 15:17, <[hidden email]> wrote:
Hi Michal,

I didn’t volunteer to be a mentor for this. The project already lists:

Mentor: Fritz Henglein, Gabriele Keller, Trevor McDonell, Edward Kmett, Sacha Sokoloski

I doubt there is much I could add to such an illustrious list.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 25 Mar 2018, at 06:11, Michal J Gajda <[hidden email]> wrote:

Given that Marco did not confirm, I am just now confirming that we can get you mentored by  Mikhail Baikov as the third mentor (beside Dominic and me).
Both me (Michal) and Mikhail are performance optimization experts (I am in parsers, and data analytics, Mikhail is in real-time systems, he has his own top-notch serialization library - Beamable that outperforms Cereal in both data size and speed). Dominic is expert in numerical computing (ODEs and Julia, among other things).

I believe that with these three excellent mentors you have very good chance to make outstanding contribution.
We just make sure that you prep application by 27th deadline.
--
  Cheers
    Michal

On Sun, Mar 25, 2018 at 6:32 AM Michal J Gajda <[hidden email]> wrote:
As a mentor I would say it is certainly possible to outperform exsting mega-solutions in some narrow domain.
Just as I did with hPDB https://hackage.haskell.org/package/hPDB
But it requires a lot of skill and patiece.

Please proceed with this project with the current list of mentors.
I think me and Dominic have already declared committment.

You might also start by making a table of best competing solutions in other. languages, their respective strengths, and ways that we can possibly improve on them!

Where do You keep Your application draft? Ideally it should be a shared space where you can add mentors as co-editors.

Cheers
Michal
On Sun, 25 Mar 2018 at 02:32, <[hidden email]> wrote:
The list of mentors for this project looks great to me. I am not sure if I can add much other than I think this is a nice project. Perhaps it would be best to get the advice of some of the mentors?

For some very simple tests with an ODE solver, I concluded that accelerate can perform at least as well as Julia. It would certainly be very helpful to be able to get Jacobians for ODE solving and for other applications.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 24 Mar 2018, at 17:20, Charles Blake <[hidden email]> wrote:

Thanks for the response Michal,

Yes, this did cross my mind - and I wouldn't be expecting to outperform those frameworks in the timeframe available! I assumed that the reason that this project was suggested was perhaps:

a) there is some intrinsic value in implementing these algorithms natively in haskell (hence why the 'ad' library was developed in the first place), so that those who want to use parallel automatic differentiation / the machine learning algorithms built on top of it can do so without leaving the haskell ecosystem,

and b) because the challenges involved in implementing parallel ad in a purely functional language are a little different to those involved in doing so in OO/imperative languages - so it might be interesting from that angle as well?

So perhaps my aim would no be to do something unique, but rather to do something that has already done well in other languages, but has not yet been provided as a haskell library. Does this sound like a reasonable approach or do I need to find a slightly more unique angle?

Thanks,
Charlie


From: Michal J Gajda <[hidden email]>
Sent: 24 March 2018 16:56:35
To: Dominic Steinitz; Marco Zocca; [hidden email]; Charles Blake; [hidden email]
Subject: Re: Accelerating Automatic Differentiation
 
Hi Charlie

It certainly looks like exciting project, but the bar is currently placed very high.
TensorFlow package not only provides automatic differentiation for whole programs, but also optimizes data processing both on GPU, and reading to achieve large batches.
This field has a lot of hot developments, so You would either need to propose something unique to Haskell, or You risk being outclassed by PyTorch and TensorFlow bindings
  Maybe Dominic suggests something too.
 Cheers
      Michal


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Automatic Differentiation

Charles Blake

Thanks so much for the feedback - I really appreciate the attention you guys have given me. My draft proposal can be found here: https://docs.google.com/document/d/1bJqoqVsW5pvXJmdkfn3EXY1QKEiTXxQkl2BmREAtBsQ/edit?usp=sharing

If you have the chance to look at it and are interested, any feedback would be valuable!


In terms of mentors for the project, I'm not entirely familiar with the process involved. I've contacted the mentors originally listed for the project with a copy of my draft proposal, but of course I appreciate your input as well and if there is the option of others joining to mentor the project then that sounds fantastic.


Charlie


From: Michal J Gajda <[hidden email]>
Sent: 25 March 2018 09:08:54
To: [hidden email]
Cc: Charles Blake; Mikhail Baykov; [hidden email]; [hidden email]
Subject: Re: Accelerating Automatic Differentiation
 
Great! If they all agreed, we can just add Mikhail, and done?
On Sun, 25 Mar 2018 at 15:17, <[hidden email]> wrote:
Hi Michal,

I didn’t volunteer to be a mentor for this. The project already lists:

Mentor: Fritz Henglein, Gabriele Keller, Trevor McDonell, Edward Kmett, Sacha Sokoloski

I doubt there is much I could add to such an illustrious list.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 25 Mar 2018, at 06:11, Michal J Gajda <[hidden email]> wrote:

Given that Marco did not confirm, I am just now confirming that we can get you mentored by  Mikhail Baikov as the third mentor (beside Dominic and me).
Both me (Michal) and Mikhail are performance optimization experts (I am in parsers, and data analytics, Mikhail is in real-time systems, he has his own top-notch serialization library - Beamable that outperforms Cereal in both data size and speed). Dominic is expert in numerical computing (ODEs and Julia, among other things).

I believe that with these three excellent mentors you have very good chance to make outstanding contribution.
We just make sure that you prep application by 27th deadline.
--
  Cheers
    Michal

On Sun, Mar 25, 2018 at 6:32 AM Michal J Gajda <[hidden email]> wrote:
As a mentor I would say it is certainly possible to outperform exsting mega-solutions in some narrow domain.
Just as I did with hPDB https://hackage.haskell.org/package/hPDB
But it requires a lot of skill and patiece.

Please proceed with this project with the current list of mentors.
I think me and Dominic have already declared committment.

You might also start by making a table of best competing solutions in other. languages, their respective strengths, and ways that we can possibly improve on them!

Where do You keep Your application draft? Ideally it should be a shared space where you can add mentors as co-editors.

Cheers
Michal
On Sun, 25 Mar 2018 at 02:32, <[hidden email]> wrote:
The list of mentors for this project looks great to me. I am not sure if I can add much other than I think this is a nice project. Perhaps it would be best to get the advice of some of the mentors?

For some very simple tests with an ODE solver, I concluded that accelerate can perform at least as well as Julia. It would certainly be very helpful to be able to get Jacobians for ODE solving and for other applications.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 24 Mar 2018, at 17:20, Charles Blake <[hidden email]> wrote:

Thanks for the response Michal,

Yes, this did cross my mind - and I wouldn't be expecting to outperform those frameworks in the timeframe available! I assumed that the reason that this project was suggested was perhaps:

a) there is some intrinsic value in implementing these algorithms natively in haskell (hence why the 'ad' library was developed in the first place), so that those who want to use parallel automatic differentiation / the machine learning algorithms built on top of it can do so without leaving the haskell ecosystem,

and b) because the challenges involved in implementing parallel ad in a purely functional language are a little different to those involved in doing so in OO/imperative languages - so it might be interesting from that angle as well?

So perhaps my aim would no be to do something unique, but rather to do something that has already done well in other languages, but has not yet been provided as a haskell library. Does this sound like a reasonable approach or do I need to find a slightly more unique angle?

Thanks,
Charlie


From: Michal J Gajda <[hidden email]>
Sent: 24 March 2018 16:56:35
To: Dominic Steinitz; Marco Zocca; [hidden email]; Charles Blake; [hidden email]
Subject: Re: Accelerating Automatic Differentiation
 
Hi Charlie

It certainly looks like exciting project, but the bar is currently placed very high.
TensorFlow package not only provides automatic differentiation for whole programs, but also optimizes data processing both on GPU, and reading to achieve large batches.
This field has a lot of hot developments, so You would either need to propose something unique to Haskell, or You risk being outclassed by PyTorch and TensorFlow bindings
  Maybe Dominic suggests something too.
 Cheers
      Michal


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Automatic Differentiation

Trevor McDonell
Regarding TensorFlow (and PyTorch, etc), it is important to note that TF is designed specifically for the deep-learning style of machine learning, but automatic differentiation itself is a much more generally useful and finds applications in physics, finance, optimisation... not just machine learning. So, I think there is value in implementing AD from a more general parallel array library like Accelerate.

I'm generally happy to add mentors. If you don't want to sign up officially, I'm sure we can just add you to the discussion board / email thread / whatever, and you can still contribute to the discussion.

Cheers,
-Trev

P.S. sorry for not replying sooner, I managed to lock myself out of my office on Friday evening, so didn't have my laptop over the weekend.


On Sun, 25 Mar 2018 at 23:42 Charles Blake <[hidden email]> wrote:

Thanks so much for the feedback - I really appreciate the attention you guys have given me. My draft proposal can be found here: https://docs.google.com/document/d/1bJqoqVsW5pvXJmdkfn3EXY1QKEiTXxQkl2BmREAtBsQ/edit?usp=sharing

If you have the chance to look at it and are interested, any feedback would be valuable!


In terms of mentors for the project, I'm not entirely familiar with the process involved. I've contacted the mentors originally listed for the project with a copy of my draft proposal, but of course I appreciate your input as well and if there is the option of others joining to mentor the project then that sounds fantastic.


Charlie


From: Michal J Gajda <[hidden email]>
Sent: 25 March 2018 09:08:54
To: [hidden email]
Cc: Charles Blake; Mikhail Baykov; [hidden email]; [hidden email]

Subject: Re: Accelerating Automatic Differentiation
Great! If they all agreed, we can just add Mikhail, and done?
On Sun, 25 Mar 2018 at 15:17, <[hidden email]> wrote:
Hi Michal,

I didn’t volunteer to be a mentor for this. The project already lists:

Mentor: Fritz Henglein, Gabriele Keller, Trevor McDonell, Edward Kmett, Sacha Sokoloski

I doubt there is much I could add to such an illustrious list.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 25 Mar 2018, at 06:11, Michal J Gajda <[hidden email]> wrote:

Given that Marco did not confirm, I am just now confirming that we can get you mentored by  Mikhail Baikov as the third mentor (beside Dominic and me).
Both me (Michal) and Mikhail are performance optimization experts (I am in parsers, and data analytics, Mikhail is in real-time systems, he has his own top-notch serialization library - Beamable that outperforms Cereal in both data size and speed). Dominic is expert in numerical computing (ODEs and Julia, among other things).

I believe that with these three excellent mentors you have very good chance to make outstanding contribution.
We just make sure that you prep application by 27th deadline.
--
  Cheers
    Michal

On Sun, Mar 25, 2018 at 6:32 AM Michal J Gajda <[hidden email]> wrote:
As a mentor I would say it is certainly possible to outperform exsting mega-solutions in some narrow domain.
Just as I did with hPDB https://hackage.haskell.org/package/hPDB
But it requires a lot of skill and patiece.

Please proceed with this project with the current list of mentors.
I think me and Dominic have already declared committment.

You might also start by making a table of best competing solutions in other. languages, their respective strengths, and ways that we can possibly improve on them!

Where do You keep Your application draft? Ideally it should be a shared space where you can add mentors as co-editors.

Cheers
Michal
On Sun, 25 Mar 2018 at 02:32, <[hidden email]> wrote:
The list of mentors for this project looks great to me. I am not sure if I can add much other than I think this is a nice project. Perhaps it would be best to get the advice of some of the mentors?

For some very simple tests with an ODE solver, I concluded that accelerate can perform at least as well as Julia. It would certainly be very helpful to be able to get Jacobians for ODE solving and for other applications.

Dominic Steinitz
Twitter: @idontgetoutmuch



On 24 Mar 2018, at 17:20, Charles Blake <[hidden email]> wrote:

Thanks for the response Michal,

Yes, this did cross my mind - and I wouldn't be expecting to outperform those frameworks in the timeframe available! I assumed that the reason that this project was suggested was perhaps:

a) there is some intrinsic value in implementing these algorithms natively in haskell (hence why the 'ad' library was developed in the first place), so that those who want to use parallel automatic differentiation / the machine learning algorithms built on top of it can do so without leaving the haskell ecosystem,

and b) because the challenges involved in implementing parallel ad in a purely functional language are a little different to those involved in doing so in OO/imperative languages - so it might be interesting from that angle as well?

So perhaps my aim would no be to do something unique, but rather to do something that has already done well in other languages, but has not yet been provided as a haskell library. Does this sound like a reasonable approach or do I need to find a slightly more unique angle?

Thanks,
Charlie


From: Michal J Gajda <[hidden email]>
Sent: 24 March 2018 16:56:35
To: Dominic Steinitz; Marco Zocca; [hidden email]; Charles Blake; [hidden email]
Subject: Re: Accelerating Automatic Differentiation
 
Hi Charlie

It certainly looks like exciting project, but the bar is currently placed very high.
TensorFlow package not only provides automatic differentiation for whole programs, but also optimizes data processing both on GPU, and reading to achieve large batches.
This field has a lot of hot developments, so You would either need to propose something unique to Haskell, or You risk being outclassed by PyTorch and TensorFlow bindings
  Maybe Dominic suggests something too.
 Cheers
      Michal

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: Accelerating Automatic Differentiation

Tom Ellis
On Sun, Mar 25, 2018 at 11:57:53PM +0000, Trevor McDonell wrote:
> Regarding TensorFlow (and PyTorch, etc), it is important to note that TF is
> designed specifically for the deep-learning style of machine learning, but
> automatic differentiation itself is a much more generally useful and finds
> applications in physics, finance, optimisation... not just machine
> learning.

It's true that TF and PyTorch are designed for deep learning applications
but is there really anything presenting one using them for the other
applications you mention?
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.