PlainText, JSON, and Content-Type

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

PlainText, JSON, and Content-Type

Jonathan Jouty-2
Hey,

I've been toying around with servant lately and I'm having trouble diagnosing the behaviour with Content-Types, namely with PlainText.
I'll be referring to Main.hs here: https://gist.github.com/jonathanjouty/5d9cfc36a7af10f4eb32c98e299b759d

My toy API should accept PlainText, decode it as JSON of type Book, and return a JSON response.
However see "curl-test" in the Gist above, I can't get it to work and keep getting "415 Unsupported Media Type" :(

Am I missing something?

I've also tried with ReqBody '[JSON, PlainText] Book, in which case setting Content-Type to application/json works fine, but text/plain does not.
The end goal is to have a permissive API that accepts text/plain and application/json, and even not setting a Content-Type at all!

Another related question is whether something like this (also lines 31-36 in Main.hs) is a good idea?

instance A.FromJSON a => MimeUnrender PlainText a where
  mimeUnrender _ = A.eitherDecode

I understand why it wouldn't be a good idea for servant, but is there any reason I wouldn't define this in my application?

Thanks in advance.
- Jonathan.

--
Reply | Threaded
Open this post in threaded view
|

Re: PlainText, JSON, and Content-Type

Julian Arni-4
Hey Jonathan,



On Tue, Aug 30, 2016 at 10:14:07AM -0700, Jonathan Jouty wrote:

> Hey,
>
> I've been toying around with servant lately and I'm having trouble
> diagnosing the behaviour with Content-Types, namely with PlainText.
> I'll be referring to Main.hs
> here: https://gist.github.com/jonathanjouty/5d9cfc36a7af10f4eb32c98e299b759d
>
> My toy API should accept PlainText, decode it as JSON of type Book, and
> return a JSON response.
> However see "curl-test" in the Gist above, I can't get it to work and keep
> getting "415 Unsupported Media Type" :(
>
> Am I missing something?

  You have the syntax of the content-type param a little wrong. It should be:

    >  Content-Type:text/plain;charset=utf-8

Note the equals sign between charset and utf-8, and the dash between utf and 8!



>
> I've also tried with ReqBody '[JSON, PlainText] Book, in which case setting
> Content-Type to application/json works fine, but text/plain does not.
> The end goal is to have a permissive API that accepts text/plain and
> application/json, and even not setting a Content-Type at all!
>
> Another related question is whether something like this (also lines 31-36
> in Main.hs) is a good idea?
>
> instance A.FromJSON a => MimeUnrender PlainText a where
>   mimeUnrender _ = A.eitherDecode
>
> I understand why it wouldn't be a good idea for servant, but is there any
> reason I wouldn't define this in my application?
No reason, though why would you do that? You already have the same presentation
available with "application/json".

>
> Thanks in advance.
> - Jonathan.
>
> --
>
>
>
>
>

--
Julian K. Arni
Haskell Consultant, Turing Jump
https://turingjump.com

--






signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: PlainText, JSON, and Content-Type

Jonathan Jouty-2
> You have the syntax of the content-type param a little wrong

D'oh!
I was certain I had copy-pasted it from [1], don't know how that happened.
Thanks!

No reason, though why would you do that? You already have the same presentation
> available with "application/json".

I've added ServantExtraJSONTypes.hs to the Gist [2].
This illustrates what I'm trying to do.

From playing around with curl it seems like OctetStream accepts empty or no Content-Type, which is the behaviour I'm looking for.
Additionally text/plain works.
My hypothetical use-case is to have a "foolproof" endpoint for dealing with bad clients that don't set Content-Type properly.

Am I missing an obvious better way to achieve this?

On a related note I wish there was an easy way to make it also accept "Content-Type: text/plain" (i.e. without specifying a charset, and just assuming utf-8).

On 30 August 2016 at 19:50, Julian Arni <[hidden email]> wrote:
Hey Jonathan,



On Tue, Aug 30, 2016 at 10:14:07AM -0700, Jonathan Jouty wrote:
> Hey,
>
> I've been toying around with servant lately and I'm having trouble
> diagnosing the behaviour with Content-Types, namely with PlainText.
> I'll be referring to Main.hs
> here: https://gist.github.com/jonathanjouty/5d9cfc36a7af10f4eb32c98e299b759d
>
> My toy API should accept PlainText, decode it as JSON of type Book, and
> return a JSON response.
> However see "curl-test" in the Gist above, I can't get it to work and keep
> getting "415 Unsupported Media Type" :(
>
> Am I missing something?


  You have the syntax of the content-type param a little wrong. It should be:

    >  Content-Type:text/plain;charset=utf-8

Note the equals sign between charset and utf-8, and the dash between utf and 8!



>
> I've also tried with ReqBody '[JSON, PlainText] Book, in which case setting
> Content-Type to application/json works fine, but text/plain does not.
> The end goal is to have a permissive API that accepts text/plain and
> application/json, and even not setting a Content-Type at all!
>
> Another related question is whether something like this (also lines 31-36
> in Main.hs) is a good idea?
>
> instance A.FromJSON a => MimeUnrender PlainText a where
>   mimeUnrender _ = A.eitherDecode
>
> I understand why it wouldn't be a good idea for servant, but is there any
> reason I wouldn't define this in my application?

No reason, though why would you do that? You already have the same presentation
available with "application/json".

>
> Thanks in advance.
> - Jonathan.
>
> --
> > > > >

--
Julian K. Arni
Haskell Consultant, Turing Jump
https://turingjump.com



--
Jonathan Jouty

--
Reply | Threaded
Open this post in threaded view
|

Re: PlainText, JSON, and Content-Type

Nickolay Kudasov
In reply to this post by Julian Arni-4
Hi Jonathan,

Another related question is whether something like this (also lines 31-36 in Main.hs) is a good idea?
instance A.FromJSON a => MimeUnrender PlainText a where
  mimeUnrender _ = A.eitherDecode
I understand why it wouldn't be a good idea for servant, but is there any reason I wouldn't define this in my application?

Normally you don't want this and you want to use "application/json".​
However, if you are writing an API for an external service which responds with JSON content under "text/plain" flag... then you can just introduce your custom content type. That is exactly what I did in smsaero [1]. There I introduced the SmsAeroJson custom content type and just used that for endpoints.


​Kind regards,
Nick​


On Tue, 30 Aug 2016 at 20:50 Julian Arni <[hidden email]> wrote:
Hey Jonathan,



On Tue, Aug 30, 2016 at 10:14:07AM -0700, Jonathan Jouty wrote:
> Hey,
>
> I've been toying around with servant lately and I'm having trouble
> diagnosing the behaviour with Content-Types, namely with PlainText.
> I'll be referring to Main.hs
> here: https://gist.github.com/jonathanjouty/5d9cfc36a7af10f4eb32c98e299b759d
>
> My toy API should accept PlainText, decode it as JSON of type Book, and
> return a JSON response.
> However see "curl-test" in the Gist above, I can't get it to work and keep
> getting "415 Unsupported Media Type" :(
>
> Am I missing something?


  You have the syntax of the content-type param a little wrong. It should be:

    >  Content-Type:text/plain;charset=utf-8

Note the equals sign between charset and utf-8, and the dash between utf and 8!



>
> I've also tried with ReqBody '[JSON, PlainText] Book, in which case setting
> Content-Type to application/json works fine, but text/plain does not.
> The end goal is to have a permissive API that accepts text/plain and
> application/json, and even not setting a Content-Type at all!
>
> Another related question is whether something like this (also lines 31-36
> in Main.hs) is a good idea?
>
> instance A.FromJSON a => MimeUnrender PlainText a where
>   mimeUnrender _ = A.eitherDecode
>
> I understand why it wouldn't be a good idea for servant, but is there any
> reason I wouldn't define this in my application?

No reason, though why would you do that? You already have the same presentation
available with "application/json".

>
> Thanks in advance.
> - Jonathan.
>
> --
> > > > >

--
Julian K. Arni
Haskell Consultant, Turing Jump
https://turingjump.com

--

--
Reply | Threaded
Open this post in threaded view
|

Re: PlainText, JSON, and Content-Type

Julian Arni-4
In reply to this post by Jonathan Jouty-2
On Tue, Aug 30, 2016 at 10:49:29PM +0200, Jonathan Jouty wrote:

> > You have the syntax of the content-type param a little wrong
>
> D'oh!
> I was certain I had copy-pasted it from [1], don't know how that happened.
> Thanks!
>
> > No reason, though why would you do that? You already have the same
> presentation
> > available with "application/json".
>
> I've added ServantExtraJSONTypes.hs to the Gist [2].
> This illustrates what I'm trying to do.
>
> From playing around with curl it seems like OctetStream accepts empty or no
> Content-Type, which is the behaviour I'm looking for.
> Additionally text/plain works.
> My hypothetical use-case is to have a "foolproof" endpoint for dealing with
> bad clients that don't set Content-Type properly.
Dealing with the web is like raising a froward child.  Every time you give it an
inch, it will take it, settle there, grow accustomed to it and ask for more and
this time cry until it gets it, because last time it got i.  Years later,
you'll have raised a person so difficult to be around that awfulness of its
life, and of the lives of everyone around it, including you, will make you
wonder at how cheaply you sold peace - for a few characters - a metaphorical
chocolate bar!

If you *really* want to completely turn off content-type negotiation, it's
possible. You can define a new combinator that ignore the Accept header and
always returns JSON. You can look up the definition of the HasServer instance
for ReqBody [0] to see how to this.

You'll be going out of your way to break RFCs, mind you.

>
> Am I missing an obvious better way to achieve this?
>
> On a related note I wish there was an easy way to make it also accept
> "Content-Type: text/plain" (i.e. without specifying a charset, and just
> assuming utf-8).

You can define a new version of PlainText, and keep everything about it the
same except for the Accept definition. That will match both 'text/plain' and
'text/plain;charset=utf-8'.  A little verbose, admittedly, but you only have to
do this once.

[0] https://github.com/haskell-servant/servant/blob/master/servant-server/src/Servant/Server/Internal.hs#L430

>
> [1]
> https://hackage.haskell.org/package/servant-0.8/docs/Servant-API-ContentTypes.html#t:PlainText
> [2] https://gist.github.com/jonathanjouty/5d9cfc36a7af10f4eb32c98e299b759d
>
> On 30 August 2016 at 19:50, Julian Arni <[hidden email]> wrote:
>
> > Hey Jonathan,
> >
> >
> >
> > On Tue, Aug 30, 2016 at 10:14:07AM -0700, Jonathan Jouty wrote:
> > > Hey,
> > >
> > > I've been toying around with servant lately and I'm having trouble
> > > diagnosing the behaviour with Content-Types, namely with PlainText.
> > > I'll be referring to Main.hs
> > > here: https://gist.github.com/jonathanjouty/
> > 5d9cfc36a7af10f4eb32c98e299b759d
> > >
> > > My toy API should accept PlainText, decode it as JSON of type Book, and
> > > return a JSON response.
> > > However see "curl-test" in the Gist above, I can't get it to work and
> > keep
> > > getting "415 Unsupported Media Type" :(
> > >
> > > Am I missing something?
> >
> >
> >   You have the syntax of the content-type param a little wrong. It should
> > be:
> >
> >     >  Content-Type:text/plain;charset=utf-8
> >
> > Note the equals sign between charset and utf-8, and the dash between utf
> > and 8!
> >
> >
> >
> > >
> > > I've also tried with ReqBody '[JSON, PlainText] Book, in which case
> > setting
> > > Content-Type to application/json works fine, but text/plain does not.
> > > The end goal is to have a permissive API that accepts text/plain and
> > > application/json, and even not setting a Content-Type at all!
> > >
> > > Another related question is whether something like this (also lines 31-36
> > > in Main.hs) is a good idea?
> > >
> > > instance A.FromJSON a => MimeUnrender PlainText a where
> > >   mimeUnrender _ = A.eitherDecode
> > >
> > > I understand why it wouldn't be a good idea for servant, but is there any
> > > reason I wouldn't define this in my application?
> >
> > No reason, though why would you do that? You already have the same
> > presentation
> > available with "application/json".
> >
> > >
> > > Thanks in advance.
> > > - Jonathan.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > Groups "haskell-servant" group.
> > >
> > an email to [hidden email].
> > >
> > >
> > com.
> > >
> >
> >
> > --
> > Julian K. Arni
> > Haskell Consultant, Turing Jump
> > https://turingjump.com
> >
>
>
>
> --
> Jonathan Jouty
>
> --
>
>
>
>
>
--
Julian K. Arni
Haskell Consultant, Turing Jump
https://turingjump.com

--






signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: PlainText, JSON, and Content-Type

Jonathan Jouty-2
Thanks for the info Nick.

> [...] a metaphorical chocolate bar!

Beautifully executed advice!
I'll stick to it, knowing that if I need to, the chocolate bar is readily available :)
Thanks.

For those stumbling on this topic, here is a relevant Reddit thread:

Note that Julian persists in making the Content-Type behaviour sane, which I think is the way to go.

On 31 August 2016 at 01:05, Julian Arni <[hidden email]> wrote:
On Tue, Aug 30, 2016 at 10:49:29PM +0200, Jonathan Jouty wrote:
> > You have the syntax of the content-type param a little wrong
>
> D'oh!
> I was certain I had copy-pasted it from [1], don't know how that happened.
> Thanks!
>
> > No reason, though why would you do that? You already have the same
> presentation
> > available with "application/json".
>
> I've added ServantExtraJSONTypes.hs to the Gist [2].
> This illustrates what I'm trying to do.
>
> From playing around with curl it seems like OctetStream accepts empty or no
> Content-Type, which is the behaviour I'm looking for.
> Additionally text/plain works.
> My hypothetical use-case is to have a "foolproof" endpoint for dealing with
> bad clients that don't set Content-Type properly.

Dealing with the web is like raising a froward child.  Every time you give it an
inch, it will take it, settle there, grow accustomed to it and ask for more and
this time cry until it gets it, because last time it got i.  Years later,
you'll have raised a person so difficult to be around that awfulness of its
life, and of the lives of everyone around it, including you, will make you
wonder at how cheaply you sold peace - for a few characters - a metaphorical
chocolate bar!

If you *really* want to completely turn off content-type negotiation, it's
possible. You can define a new combinator that ignore the Accept header and
always returns JSON. You can look up the definition of the HasServer instance
for ReqBody [0] to see how to this.

You'll be going out of your way to break RFCs, mind you.

>
> Am I missing an obvious better way to achieve this?
>
> On a related note I wish there was an easy way to make it also accept
> "Content-Type: text/plain" (i.e. without specifying a charset, and just
> assuming utf-8).

You can define a new version of PlainText, and keep everything about it the
same except for the Accept definition. That will match both 'text/plain' and
'text/plain;charset=utf-8'.  A little verbose, admittedly, but you only have to
do this once.

[0] https://github.com/haskell-servant/servant/blob/master/servant-server/src/Servant/Server/Internal.hs#L430
>
> [1]
> https://hackage.haskell.org/package/servant-0.8/docs/Servant-API-ContentTypes.html#t:PlainText
> [2] https://gist.github.com/jonathanjouty/5d9cfc36a7af10f4eb32c98e299b759d
>
> On 30 August 2016 at 19:50, Julian Arni <[hidden email]> wrote:
>
> > Hey Jonathan,
> >
> >
> >
> > On Tue, Aug 30, 2016 at 10:14:07AM -0700, Jonathan Jouty wrote:
> > > Hey,
> > >
> > > I've been toying around with servant lately and I'm having trouble
> > > diagnosing the behaviour with Content-Types, namely with PlainText.
> > > I'll be referring to Main.hs
> > > here: https://gist.github.com/jonathanjouty/
> > 5d9cfc36a7af10f4eb32c98e299b759d
> > >
> > > My toy API should accept PlainText, decode it as JSON of type Book, and
> > > return a JSON response.
> > > However see "curl-test" in the Gist above, I can't get it to work and
> > keep
> > > getting "415 Unsupported Media Type" :(
> > >
> > > Am I missing something?
> >
> >
> >   You have the syntax of the content-type param a little wrong. It should
> > be:
> >
> >     >  Content-Type:text/plain;charset=utf-8
> >
> > Note the equals sign between charset and utf-8, and the dash between utf
> > and 8!
> >
> >
> >
> > >
> > > I've also tried with ReqBody '[JSON, PlainText] Book, in which case
> > setting
> > > Content-Type to application/json works fine, but text/plain does not.
> > > The end goal is to have a permissive API that accepts text/plain and
> > > application/json, and even not setting a Content-Type at all!
> > >
> > > Another related question is whether something like this (also lines 31-36
> > > in Main.hs) is a good idea?
> > >
> > > instance A.FromJSON a => MimeUnrender PlainText a where
> > >   mimeUnrender _ = A.eitherDecode
> > >
> > > I understand why it wouldn't be a good idea for servant, but is there any
> > > reason I wouldn't define this in my application?
> >
> > No reason, though why would you do that? You already have the same
> > presentation
> > available with "application/json".
> >
> > >
> > > Thanks in advance.
> > > - Jonathan.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > Groups "haskell-servant" group.
> > > > > an email to [hidden email].
> > > > > > > > msgid/haskell-servant/582463e3-069e-4af6-ac46-a69078013c7d%40googlegroups.
> > com.
> > > > >
> >
> > --
> > Julian K. Arni
> > Haskell Consultant, Turing Jump
> > https://turingjump.com
> >
>
>
>
> --
> Jonathan Jouty
>
> --
> > >
>
>
--
Julian K. Arni
Haskell Consultant, Turing Jump
https://turingjump.com



--
Jonathan Jouty

--