How do I capture raw URL path segments? (Equivalent of `Capture` for unencoded strings, e.g. URLs)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

How do I capture raw URL path segments? (Equivalent of `Capture` for unencoded strings, e.g. URLs)

Daniel Gasienica
Hi,

First of all, kudos for all the great work on Servant! I’ve attempted to learn Haskell several times and discovering Servant (+ its tutorial) is what finally made me persevere :)

Here’s my question: Imagine you have a link shortener service and want to let your users submit URLs via http://shorten.me/<url>, e.g. http://shorten.me/http://example.com. How can I use Servant to capture that URL after the `/`? I tried using `Capture` but it (obviously) fails due to the `//` in the unescaped URL, as it thinks those slashes are path segment separators. On the other hand, submitting URI encoded URLs works, e.g. http://shorten.me/http%3A%2F%2Fexample.com.

You may think this is a bad idea, but in either case, it’s a feature of our current implementation and I was hoping to fully port it to Servant.
  • Should I attempt writing a new combinator similar to `Capture` that works for unencoded URLs?
  • Should I use the `Raw` combinator and use a vanilla WAI app/handler?
  • Or is there another, simpler approach?
Relevant code: https://github.com/gasi/zoomhub/commit/11c987d2ee2336edd3c7eb4fd0daebec6753a15f

I appreciate your insight.

Cheers,
Daniel

--
Reply | Threaded
Open this post in threaded view
|

Re: How do I capture raw URL path segments? (Equivalent of `Capture` for unencoded strings, e.g. URLs)

Sönke Hahn
Hi Daniel!

On Wednesday, March 2, 2016 at 4:23:41 PM UTC+8, Daniel Gasienica wrote:
You may think this is a bad idea,

Kind of, yes. :)
 
but in either case, it’s a feature of our current implementation and I was hoping to fully port it to Servant.

Fair enough. 
  • Should I attempt writing a new combinator similar to `Capture` that works for unencoded URLs?
This could certainly be done. IIRC there once was talk about a combinator that gives you all the remaining path segments, e.g. as a list. If you look at the `HasServer` instance for `Capture` you should find some inspiration for writing this. (If you do, please, consider posting a gist of it here.)
  • Should I use the `Raw` combinator and use a vanilla WAI app/handler?
That could also be done. `Raw` doesn't behave that nicely when using other interpretations, e.g. `servant-client` or `servant-swagger`. Not sure if that's relevant for you. 
  • Or is there another, simpler approach?
There's another option for working around these limitations in servant: building a middleware around your servant app that extracts the information you need from the request and passes it to the approriate handler. Here's a sketch (warning, not type-checked):

app :: Application
app request respond = do
  let rawPath = rawPathInfo request
  (serve myApi (handlers rawPath)) request respond

handlers :: ByteString -> Server MyApi
handlers rawPath =
  ... :<|>
  urlShortener rawPath :<|>
  ...

Through laziness this won't even come with a performance penalty, I hope.

While I like the simplicity of the last approach, the most idiomatic way would be to write your own combinator.

HTH,
Sönke

--
Reply | Threaded
Open this post in threaded view
|

Re: How do I capture raw URL path segments? (Equivalent of `Capture` for unencoded strings, e.g. URLs)

Daniel Gasienica
Sönke!

You may think this is a bad idea,

Kind of, yes. :) 
 
but in either case, it’s a feature of our current implementation and I was hoping to fully port it to Servant.

Fair enough. 

I am glad you still chose to help me ;) For what it’s worth, I think this is a neat user-facing feature. We can tell people to prepend their URL with `http://zoomhub.net/` and hit Enter — voilà. Example: http://zoomhub.net/http://upload.wikimedia.org/wikipedia/commons/3/36/SeattleI5Skyline.jpg
  • Should I attempt writing a new combinator similar to `Capture` that works for unencoded URLs?
This could certainly be done. IIRC there once was talk about a combinator that gives you all the remaining path segments, e.g. as a list. If you look at the `HasServer` instance for `Capture` you should find some inspiration for writing this. (If you do, please, consider posting a gist of it here.)

Great! This is exactly what I did after attempting the middleware as POC. The Servant source and your pointers to it were super helpful. Generally, the type-level programming is beyond my current Haskell knowledge but I was able to pull it off. As promised, here’s a link to the source: https://github.com/gasi/zoomhub/blob/9ffc7f28f9da5e029e4f6a1cb9dc8be2ebc875de/src/ZoomHub/Servant/RawCapture.hs

As you can see, I didn’t know how to end the routing, so I set `pathInfo = [""]`, but I’d love to hear about a better way. I thought maybe `succeedWith` but that seems to be for taking over the response based on the examples. If you find `RawCapture` might be useful for other Servant user, I’d be happy to contribute its code to the main project. Let me know!
 
  • Should I use the `Raw` combinator and use a vanilla WAI app/handler?
That could also be done. `Raw` doesn't behave that nicely when using other interpretations, e.g. `servant-client` or `servant-swagger`. Not sure if that's relevant for you. 

Ah, I didn’t know that, but it makes sense. At some point, I was hoping to convince some of my Node.js loving friends to take a look at Haskell and I think `servant-client` and/or `servant-swagger` might be compelling demos, so I’ll try to stay away from `Raw` (currently still have it for some `serveDirectory` logic; speaking of which, what’s a good way to serve static files and stay compatible with `servant-client` et al.?)

  • Or is there another, simpler approach?
There's another option for working around these limitations in servant: building a middleware around your servant app that extracts the information you need from the request and passes it to the approriate handler. Here's a sketch (warning, not type-checked):

app :: Application
app request respond = do
  let rawPath = rawPathInfo request
  (serve myApi (handlers rawPath)) request respond

handlers :: ByteString -> Server MyApi
handlers rawPath =
  ... :<|>
  urlShortener rawPath :<|>
  ...

Through laziness this won't even come with a performance penalty, I hope.

Thanks, I tried that first and it worked as well: https://github.com/gasi/zoomhub/commit/c99dea19ae0baaa3e54c5aa7630ff1f0151aa416 

While I like the simplicity of the last approach, the most idiomatic way would be to write your own combinator.

Agreed and done! :) 

Thanks for all your help and let me know how I can help out the Servant team as a relative Haskell novice.

Cheers,
Daniel

--
Reply | Threaded
Open this post in threaded view
|

Re: How do I capture raw URL path segments? (Equivalent of `Capture` for unencoded strings, e.g. URLs)

Daniel Gasienica
Of course, the night didn’t end with just needing one combinator. I wrote another one: `RequiredQueryParam`:
https://github.com/gasi/zoomhub/pull/9/files#diff-a684fa5fca009d4be328147b36dc15c7R1

Use case: Distinguish between http://zoomhub.net/ (homepage) and http://zoomhub.net/?url=http://example.com (URL submission).

Maybe you find this URL design decision (remember: legacy!) to be less questionable.

On that note, our current implementation (Node.js) provides error messages for invalid query parameters:
http://zoomhub.net/?url=bloedsinn --> Please give us the full URL, including "http://" or "https://".

Code: https://github.com/gasi/zoomhub/blob/3cd4b4a5932cd708aa6a14ae59f1d57be145b585/node/lib/routes._coffee#L166-L168

<strike>Admittedly, this is done very imperatively in Node.js, but I was wondering if your team has considered supporting error reporting on invalid (query) params in Servant? Maybe using a `fromText` equivalent with `Either String (Maybe a)` implementation or similar instead of `Maybe a`.</strike> I just solved this using a fall-through: https://github.com/gasi/zoomhub/commit/9a635809fc656abf9ecd509a5fa5fc8dd9cad525

What do you think?

On this note, is there any interest in me contributing the `RequiredQueryParam` combinator?

Cheers,
Daniel

On Wednesday, March 2, 2016 at 10:21:51 PM UTC-8, Daniel Gasienica wrote:
Sönke!

You may think this is a bad idea,

Kind of, yes. :) 
 
but in either case, it’s a feature of our current implementation and I was hoping to fully port it to Servant.

Fair enough. 

I am glad you still chose to help me ;) For what it’s worth, I think this is a neat user-facing feature. We can tell people to prepend their URL with `<a href="http://zoomhub.net/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNEXkhJThK1WXQGi721wau_z2P4Lyg&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNEXkhJThK1WXQGi721wau_z2P4Lyg&#39;;return true;">http://zoomhub.net/` and hit Enter — voilà. Example: <a href="http://zoomhub.net/http://upload.wikimedia.org/wikipedia/commons/3/36/SeattleI5Skyline.jpg" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2Fhttp%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2F3%2F36%2FSeattleI5Skyline.jpg\46sa\75D\46sntz\0751\46usg\75AFQjCNGA8klGeYvuKMfg6ZYizVcBTztaww&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2Fhttp%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2F3%2F36%2FSeattleI5Skyline.jpg\46sa\75D\46sntz\0751\46usg\75AFQjCNGA8klGeYvuKMfg6ZYizVcBTztaww&#39;;return true;">http://zoomhub.net/http://upload.wikimedia.org/wikipedia/commons/3/36/SeattleI5Skyline.jpg
  • Should I attempt writing a new combinator similar to `Capture` that works for unencoded URLs?
This could certainly be done. IIRC there once was talk about a combinator that gives you all the remaining path segments, e.g. as a list. If you look at the `HasServer` instance for `Capture` you should find some inspiration for writing this. (If you do, please, consider posting a gist of it here.)

Great! This is exactly what I did after attempting the middleware as POC. The Servant source and your pointers to it were super helpful. Generally, the type-level programming is beyond my current Haskell knowledge but I was able to pull it off. As promised, here’s a link to the source: <a href="https://github.com/gasi/zoomhub/blob/9ffc7f28f9da5e029e4f6a1cb9dc8be2ebc875de/src/ZoomHub/Servant/RawCapture.hs" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fblob%2F9ffc7f28f9da5e029e4f6a1cb9dc8be2ebc875de%2Fsrc%2FZoomHub%2FServant%2FRawCapture.hs\46sa\75D\46sntz\0751\46usg\75AFQjCNEb4EqZNLskgEjgvPcsZmZRaOeflw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fblob%2F9ffc7f28f9da5e029e4f6a1cb9dc8be2ebc875de%2Fsrc%2FZoomHub%2FServant%2FRawCapture.hs\46sa\75D\46sntz\0751\46usg\75AFQjCNEb4EqZNLskgEjgvPcsZmZRaOeflw&#39;;return true;">https://github.com/gasi/zoomhub/blob/9ffc7f28f9da5e029e4f6a1cb9dc8be2ebc875de/src/ZoomHub/Servant/RawCapture.hs

As you can see, I didn’t know how to end the routing, so I set `pathInfo = [""]`, but I’d love to hear about a better way. I thought maybe `succeedWith` but that seems to be for taking over the response based on the examples. If you find `RawCapture` might be useful for other Servant user, I’d be happy to contribute its code to the main project. Let me know!
 
  • Should I use the `Raw` combinator and use a vanilla WAI app/handler?
That could also be done. `Raw` doesn't behave that nicely when using other interpretations, e.g. `servant-client` or `servant-swagger`. Not sure if that's relevant for you. 

Ah, I didn’t know that, but it makes sense. At some point, I was hoping to convince some of my Node.js loving friends to take a look at Haskell and I think `servant-client` and/or `servant-swagger` might be compelling demos, so I’ll try to stay away from `Raw` (currently still have it for some `serveDirectory` logic; speaking of which, what’s a good way to serve static files and stay compatible with `servant-client` et al.?)

  • Or is there another, simpler approach?
There's another option for working around these limitations in servant: building a middleware around your servant app that extracts the information you need from the request and passes it to the approriate handler. Here's a sketch (warning, not type-checked):

app :: Application
app request respond = do
  let rawPath = rawPathInfo request
  (serve myApi (handlers rawPath)) request respond

handlers :: ByteString -> Server MyApi
handlers rawPath =
  ... :<|>
  urlShortener rawPath :<|>
  ...

Through laziness this won't even come with a performance penalty, I hope.

Thanks, I tried that first and it worked as well: <a href="https://github.com/gasi/zoomhub/commit/c99dea19ae0baaa3e54c5aa7630ff1f0151aa416" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fcommit%2Fc99dea19ae0baaa3e54c5aa7630ff1f0151aa416\46sa\75D\46sntz\0751\46usg\75AFQjCNGemhAz_vJ2a6Hy1Sn-OMbQ_UjxxQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fcommit%2Fc99dea19ae0baaa3e54c5aa7630ff1f0151aa416\46sa\75D\46sntz\0751\46usg\75AFQjCNGemhAz_vJ2a6Hy1Sn-OMbQ_UjxxQ&#39;;return true;">https://github.com/gasi/zoomhub/commit/c99dea19ae0baaa3e54c5aa7630ff1f0151aa416 

While I like the simplicity of the last approach, the most idiomatic way would be to write your own combinator.

Agreed and done! :) 

Thanks for all your help and let me know how I can help out the Servant team as a relative Haskell novice.

Cheers,
Daniel

--
Reply | Threaded
Open this post in threaded view
|

Re: How do I capture raw URL path segments? (Equivalent of `Capture` for unencoded strings, e.g. URLs)

Nickolay Kudasov
Hi Daniel,

Making "required" versions of some combinators is something many people actually would like to see in servant.
However, the design space there is big enough that we haven't yet settled on how exactly those would look in servant.

However, your contributions are welcome to the servant-contrib set of packages: https://github.com/haskell-servant/servant-contrib
Unfortunately it is empty so far, because, well, servant maintainers don't have enough time to also put user suggestions there.
But if you want, you can put any of the combinators you think might be useful to others in servant-contrib.
You can put both the RequiredQueryParam and CaptureURL (or whatever you call it) combinators there.

Kind regards,
Nick

On Thu, 3 Mar 2016 at 11:07 Daniel Gasienica <[hidden email]> wrote:
Of course, the night didn’t end with just needing one combinator. I wrote another one: `RequiredQueryParam`:

Use case: Distinguish between http://zoomhub.net/ (homepage) and http://zoomhub.net/?url=http://example.com (URL submission).

Maybe you find this URL design decision (remember: legacy!) to be less questionable.

On that note, our current implementation (Node.js) provides error messages for invalid query parameters:
http://zoomhub.net/?url=bloedsinn --> Please give us the full URL, including "http://" or "https://".


<strike>Admittedly, this is done very imperatively in Node.js, but I was wondering if your team has considered supporting error reporting on invalid (query) params in Servant? Maybe using a `fromText` equivalent with `Either String (Maybe a)` implementation or similar instead of `Maybe a`.</strike> I just solved this using a fall-through: https://github.com/gasi/zoomhub/commit/9a635809fc656abf9ecd509a5fa5fc8dd9cad525

What do you think?

On this note, is there any interest in me contributing the `RequiredQueryParam` combinator?

Cheers,
Daniel


On Wednesday, March 2, 2016 at 10:21:51 PM UTC-8, Daniel Gasienica wrote:
Sönke!

You may think this is a bad idea,

Kind of, yes. :) 
 
but in either case, it’s a feature of our current implementation and I was hoping to fully port it to Servant.

Fair enough. 

I am glad you still chose to help me ;) For what it’s worth, I think this is a neat user-facing feature. We can tell people to prepend their URL with `http://zoomhub.net/` and hit Enter — voilà. Example: http://zoomhub.net/http://upload.wikimedia.org/wikipedia/commons/3/36/SeattleI5Skyline.jpg
  • Should I attempt writing a new combinator similar to `Capture` that works for unencoded URLs?
This could certainly be done. IIRC there once was talk about a combinator that gives you all the remaining path segments, e.g. as a list. If you look at the `HasServer` instance for `Capture` you should find some inspiration for writing this. (If you do, please, consider posting a gist of it here.)

Great! This is exactly what I did after attempting the middleware as POC. The Servant source and your pointers to it were super helpful. Generally, the type-level programming is beyond my current Haskell knowledge but I was able to pull it off. As promised, here’s a link to the source: https://github.com/gasi/zoomhub/blob/9ffc7f28f9da5e029e4f6a1cb9dc8be2ebc875de/src/ZoomHub/Servant/RawCapture.hs

As you can see, I didn’t know how to end the routing, so I set `pathInfo = [""]`, but I’d love to hear about a better way. I thought maybe `succeedWith` but that seems to be for taking over the response based on the examples. If you find `RawCapture` might be useful for other Servant user, I’d be happy to contribute its code to the main project. Let me know!
 
  • Should I use the `Raw` combinator and use a vanilla WAI app/handler?
That could also be done. `Raw` doesn't behave that nicely when using other interpretations, e.g. `servant-client` or `servant-swagger`. Not sure if that's relevant for you. 

Ah, I didn’t know that, but it makes sense. At some point, I was hoping to convince some of my Node.js loving friends to take a look at Haskell and I think `servant-client` and/or `servant-swagger` might be compelling demos, so I’ll try to stay away from `Raw` (currently still have it for some `serveDirectory` logic; speaking of which, what’s a good way to serve static files and stay compatible with `servant-client` et al.?)

  • Or is there another, simpler approach?
There's another option for working around these limitations in servant: building a middleware around your servant app that extracts the information you need from the request and passes it to the approriate handler. Here's a sketch (warning, not type-checked):

app :: Application
app request respond = do
  let rawPath = rawPathInfo request
  (serve myApi (handlers rawPath)) request respond

handlers :: ByteString -> Server MyApi
handlers rawPath =
  ... :<|>
  urlShortener rawPath :<|>
  ...

Through laziness this won't even come with a performance penalty, I hope.


While I like the simplicity of the last approach, the most idiomatic way would be to write your own combinator.

Agreed and done! :) 

Thanks for all your help and let me know how I can help out the Servant team as a relative Haskell novice.

Cheers,
Daniel

--

--
Reply | Threaded
Open this post in threaded view
|

Re: How do I capture raw URL path segments? (Equivalent of `Capture` for unencoded strings, e.g. URLs)

Daniel Gasienica
Nick,

Thanks for the pointer to the GitHub issue as well as the `servant-contrib` effort. I’ll try to submit my combinators once I have proven them to be working well :)

Cheers,
Daniel

On Thursday, March 3, 2016 at 12:55:11 AM UTC-8, Nickolay Kudasov wrote:
Hi Daniel,

Making "required" versions of some combinators is something many people actually would like to see in servant.
However, the design space there is big enough that we haven't yet settled on how exactly those would look in servant.
The discussion is available here: <a href="https://github.com/haskell-servant/servant/issues/241" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fhaskell-servant%2Fservant%2Fissues%2F241\46sa\75D\46sntz\0751\46usg\75AFQjCNEwdQGGxdIzRZn3XEVx2c2YFnx2wA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fhaskell-servant%2Fservant%2Fissues%2F241\46sa\75D\46sntz\0751\46usg\75AFQjCNEwdQGGxdIzRZn3XEVx2c2YFnx2wA&#39;;return true;">https://github.com/haskell-servant/servant/issues/241

However, your contributions are welcome to the servant-contrib set of packages: <a href="https://github.com/haskell-servant/servant-contrib" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fhaskell-servant%2Fservant-contrib\46sa\75D\46sntz\0751\46usg\75AFQjCNF33MhUDVnZNSOi-M3Uxbk0l0Ec4Q&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fhaskell-servant%2Fservant-contrib\46sa\75D\46sntz\0751\46usg\75AFQjCNF33MhUDVnZNSOi-M3Uxbk0l0Ec4Q&#39;;return true;">https://github.com/haskell-servant/servant-contrib
Unfortunately it is empty so far, because, well, servant maintainers don't have enough time to also put user suggestions there.
But if you want, you can put any of the combinators you think might be useful to others in servant-contrib.
You can put both the RequiredQueryParam and CaptureURL (or whatever you call it) combinators there.

Kind regards,
Nick

On Thu, 3 Mar 2016 at 11:07 Daniel Gasienica <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="65KKOGUDHgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">daniel.g...@...> wrote:
Of course, the night didn’t end with just needing one combinator. I wrote another one: `RequiredQueryParam`:
<a href="https://github.com/gasi/zoomhub/pull/9/files#diff-a684fa5fca009d4be328147b36dc15c7R1" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fpull%2F9%2Ffiles%23diff-a684fa5fca009d4be328147b36dc15c7R1\46sa\75D\46sntz\0751\46usg\75AFQjCNGKvzbs0n-Zdl4tIRzuKOYb9NgrDw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fpull%2F9%2Ffiles%23diff-a684fa5fca009d4be328147b36dc15c7R1\46sa\75D\46sntz\0751\46usg\75AFQjCNGKvzbs0n-Zdl4tIRzuKOYb9NgrDw&#39;;return true;">https://github.com/gasi/zoomhub/pull/9/files#diff-a684fa5fca009d4be328147b36dc15c7R1

Use case: Distinguish between <a href="http://zoomhub.net/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNEXkhJThK1WXQGi721wau_z2P4Lyg&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNEXkhJThK1WXQGi721wau_z2P4Lyg&#39;;return true;">http://zoomhub.net/ (homepage) and <a href="http://zoomhub.net/?url=http://example.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F%3Furl%3Dhttp%3A%2F%2Fexample.com\46sa\75D\46sntz\0751\46usg\75AFQjCNGqFLbaGGN37AqhQrZ2ETRaVgkndw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F%3Furl%3Dhttp%3A%2F%2Fexample.com\46sa\75D\46sntz\0751\46usg\75AFQjCNGqFLbaGGN37AqhQrZ2ETRaVgkndw&#39;;return true;">http://zoomhub.net/?url=http://example.com (URL submission).

Maybe you find this URL design decision (remember: legacy!) to be less questionable.

On that note, our current implementation (Node.js) provides error messages for invalid query parameters:
<a href="http://zoomhub.net/?url=bloedsinn" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F%3Furl%3Dbloedsinn\46sa\75D\46sntz\0751\46usg\75AFQjCNGAnkxM5AHE6QZ4retZy5UM5sNHBw&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F%3Furl%3Dbloedsinn\46sa\75D\46sntz\0751\46usg\75AFQjCNGAnkxM5AHE6QZ4retZy5UM5sNHBw&#39;;return true;">http://zoomhub.net/?url=bloedsinn --> Please give us the full URL, including "http://" or "https://".

Code: <a href="https://github.com/gasi/zoomhub/blob/3cd4b4a5932cd708aa6a14ae59f1d57be145b585/node/lib/routes._coffee#L166-L168" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fblob%2F3cd4b4a5932cd708aa6a14ae59f1d57be145b585%2Fnode%2Flib%2Froutes._coffee%23L166-L168\46sa\75D\46sntz\0751\46usg\75AFQjCNFUqPXu7X85sO907pDkfQeOYx0yxw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fblob%2F3cd4b4a5932cd708aa6a14ae59f1d57be145b585%2Fnode%2Flib%2Froutes._coffee%23L166-L168\46sa\75D\46sntz\0751\46usg\75AFQjCNFUqPXu7X85sO907pDkfQeOYx0yxw&#39;;return true;">https://github.com/gasi/zoomhub/blob/3cd4b4a5932cd708aa6a14ae59f1d57be145b585/node/lib/routes._coffee#L166-L168

<strike>Admittedly, this is done very imperatively in Node.js, but I was wondering if your team has considered supporting error reporting on invalid (query) params in Servant? Maybe using a `fromText` equivalent with `Either String (Maybe a)` implementation or similar instead of `Maybe a`.</strike> I just solved this using a fall-through: <a href="https://github.com/gasi/zoomhub/commit/9a635809fc656abf9ecd509a5fa5fc8dd9cad525" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fcommit%2F9a635809fc656abf9ecd509a5fa5fc8dd9cad525\46sa\75D\46sntz\0751\46usg\75AFQjCNHCsF1o3mEtVK3XIm4-kt0KConLaQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fcommit%2F9a635809fc656abf9ecd509a5fa5fc8dd9cad525\46sa\75D\46sntz\0751\46usg\75AFQjCNHCsF1o3mEtVK3XIm4-kt0KConLaQ&#39;;return true;">https://github.com/gasi/zoomhub/commit/9a635809fc656abf9ecd509a5fa5fc8dd9cad525

What do you think?

On this note, is there any interest in me contributing the `RequiredQueryParam` combinator?

Cheers,
Daniel


On Wednesday, March 2, 2016 at 10:21:51 PM UTC-8, Daniel Gasienica wrote:
Sönke!

You may think this is a bad idea,

Kind of, yes. :) 
 
but in either case, it’s a feature of our current implementation and I was hoping to fully port it to Servant.

Fair enough. 

I am glad you still chose to help me ;) For what it’s worth, I think this is a neat user-facing feature. We can tell people to prepend their URL with `<a href="http://zoomhub.net/" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNEXkhJThK1WXQGi721wau_z2P4Lyg&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNEXkhJThK1WXQGi721wau_z2P4Lyg&#39;;return true;">http://zoomhub.net/` and hit Enter — voilà. Example: <a href="http://zoomhub.net/http://upload.wikimedia.org/wikipedia/commons/3/36/SeattleI5Skyline.jpg" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2Fhttp%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2F3%2F36%2FSeattleI5Skyline.jpg\46sa\75D\46sntz\0751\46usg\75AFQjCNGA8klGeYvuKMfg6ZYizVcBTztaww&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2Fhttp%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2F3%2F36%2FSeattleI5Skyline.jpg\46sa\75D\46sntz\0751\46usg\75AFQjCNGA8klGeYvuKMfg6ZYizVcBTztaww&#39;;return true;">http://zoomhub.net/http://upload.wikimedia.org/wikipedia/commons/3/36/SeattleI5Skyline.jpg
  • Should I attempt writing a new combinator similar to `Capture` that works for unencoded URLs?
This could certainly be done. IIRC there once was talk about a combinator that gives you all the remaining path segments, e.g. as a list. If you look at the `HasServer` instance for `Capture` you should find some inspiration for writing this. (If you do, please, consider posting a gist of it here.)

Great! This is exactly what I did after attempting the middleware as POC. The Servant source and your pointers to it were super helpful. Generally, the type-level programming is beyond my current Haskell knowledge but I was able to pull it off. As promised, here’s a link to the source: <a href="https://github.com/gasi/zoomhub/blob/9ffc7f28f9da5e029e4f6a1cb9dc8be2ebc875de/src/ZoomHub/Servant/RawCapture.hs" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fblob%2F9ffc7f28f9da5e029e4f6a1cb9dc8be2ebc875de%2Fsrc%2FZoomHub%2FServant%2FRawCapture.hs\46sa\75D\46sntz\0751\46usg\75AFQjCNEb4EqZNLskgEjgvPcsZmZRaOeflw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fblob%2F9ffc7f28f9da5e029e4f6a1cb9dc8be2ebc875de%2Fsrc%2FZoomHub%2FServant%2FRawCapture.hs\46sa\75D\46sntz\0751\46usg\75AFQjCNEb4EqZNLskgEjgvPcsZmZRaOeflw&#39;;return true;">https://github.com/gasi/zoomhub/blob/9ffc7f28f9da5e029e4f6a1cb9dc8be2ebc875de/src/ZoomHub/Servant/RawCapture.hs

As you can see, I didn’t know how to end the routing, so I set `pathInfo = [""]`, but I’d love to hear about a better way. I thought maybe `succeedWith` but that seems to be for taking over the response based on the examples. If you find `RawCapture` might be useful for other Servant user, I’d be happy to contribute its code to the main project. Let me know!
 
  • Should I use the `Raw` combinator and use a vanilla WAI app/handler?
That could also be done. `Raw` doesn't behave that nicely when using other interpretations, e.g. `servant-client` or `servant-swagger`. Not sure if that's relevant for you. 

Ah, I didn’t know that, but it makes sense. At some point, I was hoping to convince some of my Node.js loving friends to take a look at Haskell and I think `servant-client` and/or `servant-swagger` might be compelling demos, so I’ll try to stay away from `Raw` (currently still have it for some `serveDirectory` logic; speaking of which, what’s a good way to serve static files and stay compatible with `servant-client` et al.?)

  • Or is there another, simpler approach?
There's another option for working around these limitations in servant: building a middleware around your servant app that extracts the information you need from the request and passes it to the approriate handler. Here's a sketch (warning, not type-checked):

app :: Application
app request respond = do
  let rawPath = rawPathInfo request
  (serve myApi (handlers rawPath)) request respond

handlers :: ByteString -> Server MyApi
handlers rawPath =
  ... :<|>
  urlShortener rawPath :<|>
  ...

Through laziness this won't even come with a performance penalty, I hope.

Thanks, I tried that first and it worked as well: <a href="https://github.com/gasi/zoomhub/commit/c99dea19ae0baaa3e54c5aa7630ff1f0151aa416" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fcommit%2Fc99dea19ae0baaa3e54c5aa7630ff1f0151aa416\46sa\75D\46sntz\0751\46usg\75AFQjCNGemhAz_vJ2a6Hy1Sn-OMbQ_UjxxQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fgasi%2Fzoomhub%2Fcommit%2Fc99dea19ae0baaa3e54c5aa7630ff1f0151aa416\46sa\75D\46sntz\0751\46usg\75AFQjCNGemhAz_vJ2a6Hy1Sn-OMbQ_UjxxQ&#39;;return true;">https://github.com/gasi/zoomhub/commit/c99dea19ae0baaa3e54c5aa7630ff1f0151aa416 

While I like the simplicity of the last approach, the most idiomatic way would be to write your own combinator.

Agreed and done! :) 

Thanks for all your help and let me know how I can help out the Servant team as a relative Haskell novice.

Cheers,
Daniel

--

--
Reply | Threaded
Open this post in threaded view
|

Re: How do I capture raw URL path segments? (Equivalent of `Capture` for unencoded strings, e.g. URLs)

Sönke Hahn
In reply to this post by Daniel Gasienica
Hi Daniel,

On Thursday, March 3, 2016 at 2:21:51 PM UTC+8, Daniel Gasienica wrote:
I am glad you still chose to help me ;) For what it’s worth, I think this is a neat user-facing feature. We can tell people to prepend their URL with `<a href="http://zoomhub.net/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNEXkhJThK1WXQGi721wau_z2P4Lyg&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNEXkhJThK1WXQGi721wau_z2P4Lyg&#39;;return true;">http://zoomhub.net/` and hit Enter — voilà. Example: <a href="http://zoomhub.net/http://upload.wikimedia.org/wikipedia/commons/3/36/SeattleI5Skyline.jpg" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2Fhttp%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2F3%2F36%2FSeattleI5Skyline.jpg\46sa\75D\46sntz\0751\46usg\75AFQjCNGA8klGeYvuKMfg6ZYizVcBTztaww&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\75http%3A%2F%2Fzoomhub.net%2Fhttp%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2F3%2F36%2FSeattleI5Skyline.jpg\46sa\75D\46sntz\0751\46usg\75AFQjCNGA8klGeYvuKMfg6ZYizVcBTztaww&#39;;return true;">http://zoomhub.net/http://upload.wikimedia.org/wikipedia/commons/3/36/SeattleI5Skyline.jpg

Yes, that's a nice feature.

Even if it weren't a nice feature, I think that servant should still try to play nicely with all kinds of weird APIs/endpoints. Mainly because they do exist in the wild and often you can't or don't want to change them.
 
As you can see, I didn’t know how to end the routing, so I set `pathInfo = [""]`, but I’d love to hear about a better way.

I think this exactly what you're supposed to do in `wai`.

Great, that servant worked out for you here. :) And thanks for sharing the links.

Cheers,
Sönke

--