Re: upload file with ftp

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: upload file with ftp

Kees Bleijenberg

Hi Brandon,

 

The ftp-client I use (total commander) does not use passive mode.  I changed the host to another server. And now I do not get an exception, no error messages, but nothing is uploaded.

 

Unfortunately is ‘working in passive mode’ not so easy (at least not to me). After login I send the ‘pasv’ command and I get back a IP address and a port (I assume). Do I have to create a new connection with a handle to this new port for the stor command?  I don’t know how to do that in network.ftp.client. A hint or better a working example would be nice.

 

Kees

Van: Brandon Allbery [mailto:[hidden email]]
Verzonden: dinsdag 5 januari 2021 13:49
Aan: Kees Bleijenberg <[hidden email]>
CC: haskell-cafe <[hidden email]>
Onderwerp: Re: [Haskell-cafe] FW: upload file with ftp

 

At a guess, you're using a modern FTP client that defaults to passive mode for compatibility with firewalls, and you'll need to check the docs to see how to do passive mode with Network.Client.FTP.

 

On Tue, Jan 5, 2021 at 7:24 AM Kees Bleijenberg <[hidden email]> wrote:

I want to upload a file with ftp using library Network.FTP.Client.
The OS is Windows 64 and ghc version is 8.6.4.

This is the code:

import System.FilePath.Windows
import Network.FTP.Client
import qualified Data.ByteString as B
....
testFtp :: IO (Either String ())
testFtp = do
   let host = "copecco"  -- this is from my hosts file       
       username = ****
       pwd = ****
       ftpDir = "registratie"
       fileToSend = "globals.pas"  -- a test text file
   putStrLn $ "Connect to " ++ host
   withFTP host 21 $ \h ftpResponse -> do
         print ftpResponse
         if frStatus ftpResponse == Success
         then do
                  putStrLn "Connected"
                  loginResp <- login h username pwd
                  print loginResp
                  if frStatus loginResp == Success
                  then do
                           putStrLn $ "Change directory to " ++ ftpDir
                           cwdResp <- cwd h ftpDir
                           print cwdResp
                           if frStatus cwdResp == Success
                           then do 
                                    putStrLn "Read file from disk"
                                    let ftpFn =  takeFileName fileToSend
                                    fileContents <- B.readFile fileToSend
                                    putStrLn $ "File size: " ++ show
(B.length fileContents) ++ " bytes"
                                    stor h ftpFn fileContents TI
                                    return $ Right ()
                           else return $ Left $ "Ftp error cwd. Code: " ++
show (frCode ftpResponse)
                  else return $ Left $ "Ftp error login. Code: " ++ show
(frCode ftpResponse)
         else return $ Left $ "Connect to host " ++ host  ++ " failed. Code:
" ++ show (frCode ftpResponse)

This is the response:
Connect to copecco

220 (vsFTPd 3.0.2)

Connected

230 Login successful.

Change directory to registratie

250 Directory successfully changed.

Read file

File size: 27015 bytes

*** Exception: Network.Socket.connect: <socket: 444>: failed (Connection
timed out (WSAETIMEDOUT)) 

Everything works fine until the stor commmand.
When I upload the same file with another ftp client program everything works
(no permission problems).

What is wrong and how do I get the result codes for the stor command?

Kees


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.


 

--

brandon s allbery kf8nh


_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: upload file with ftp

Joachim Durchholz
Am 05.01.21 um 16:54 schrieb Kees Bleijenberg:
> Unfortunately is ‘working in passive mode’ not so easy (at least not to
> me). After login I send the ‘pasv’ command and I get back a IP address
> and a port (I assume). Do I have to create a new connection with a
> handle to this new port for the stor command?

Yes.

 >  I don’t know how to do that in network.ftp.client.

If network.ftp.client does not support that out of the box, you'll
either have to find another library or add PASV support to it.

My knowledge of the FTP protocol is pretty rusty, but I believe you the
model after PASV is that you continue using normal commands but the data
transfers happen over the other connection.
I'd watch out for how the protocol marks the beginning and end of a data
stream (it's not a single connection anymore so you lose the sequencing
guarantees of TCP packets; I'd assume they either send a special packet
on the data connection, or they transmit the file sizes via the normal
channel).

However, I can't avoid wondering why anybody would want to use the FTP
protocol at all. It's so unsafe, so bottleneck-endangered, the only
reason to do that would be interfacing with existing FTP servers, and
even these are being phased out in favor of HTTPS, with POST for uploads.
(I'm not questioning the validity of your use case, just wondering how
such a use case can come up in today's world.)

Regards,
Jo
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: upload file with ftp

Brandon Allbery
In normal mode the server opens a connection to the FTP client; in PASV mode the client instead opens a connection to the server, so it'll work through a firewall.

On Tue, Jan 5, 2021 at 2:09 PM Joachim Durchholz <[hidden email]> wrote:
Am 05.01.21 um 16:54 schrieb Kees Bleijenberg:
> Unfortunately is ‘working in passive mode’ not so easy (at least not to
> me). After login I send the ‘pasv’ command and I get back a IP address
> and a port (I assume). Do I have to create a new connection with a
> handle to this new port for the stor command?

Yes.

 >  I don’t know how to do that in network.ftp.client.

If network.ftp.client does not support that out of the box, you'll
either have to find another library or add PASV support to it.

My knowledge of the FTP protocol is pretty rusty, but I believe you the
model after PASV is that you continue using normal commands but the data
transfers happen over the other connection.
I'd watch out for how the protocol marks the beginning and end of a data
stream (it's not a single connection anymore so you lose the sequencing
guarantees of TCP packets; I'd assume they either send a special packet
on the data connection, or they transmit the file sizes via the normal
channel).

However, I can't avoid wondering why anybody would want to use the FTP
protocol at all. It's so unsafe, so bottleneck-endangered, the only
reason to do that would be interfacing with existing FTP servers, and
even these are being phased out in favor of HTTPS, with POST for uploads.
(I'm not questioning the validity of your use case, just wondering how
such a use case can come up in today's world.)

Regards,
Jo
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.


--
brandon s allbery kf8nh

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: upload file with ftp

Kees Bleijenberg
In reply to this post by Joachim Durchholz
Joachim, Brandon,

After investigating the source for the store command in  Network.FTP.Client (https://hackage.haskell.org/package/ftp-client-0.5.1.4/docs/Network-FTP-Client.html) I found that the stor command already sets the connection in passive mode. I tried my original code on Linux and everything works fine. So probably the problem is (as always) Windows.  

A few years ago I tried some Haskell ftp client libs. On Windows they all raised access violations as soon as I connected to a server. Now,  a few years later the Network library has improved (not sure) and I tried it again. Big improvements: Login, get a list of files, change directory works fine but stor does not.  So there is still no working FTP-client lib for Haskell (I think).

I know FTP is not secure but it is easy to use. Almost everybody uses it for uploading files to a website. Writing code for uploading a file with http(s)in Haskell is a lot more work than ftp and I wonder whether the https stuff will work on Windows. And  with http(s) you have to write an extra server side program (php) to save the uploaded file, return error messages...
I just want a simple way to upload a generated file to my private website.
Until now I upload a file by running a program within my Haskell program. That ftp-program sends a file with ftp to a server. The ftp-program is written in Delphi/Lazarus.

Thanks for your help.
Kees


-----Oorspronkelijk bericht-----
Van: Haskell-Cafe [mailto:[hidden email]] Namens Joachim Durchholz
Verzonden: dinsdag 5 januari 2021 20:09
Aan: [hidden email]
Onderwerp: Re: [Haskell-cafe] upload file with ftp

Am 05.01.21 um 16:54 schrieb Kees Bleijenberg:
> Unfortunately is ‘working in passive mode’ not so easy (at least not to
> me). After login I send the ‘pasv’ command and I get back a IP address
> and a port (I assume). Do I have to create a new connection with a
> handle to this new port for the stor command?

Yes.

 >  I don’t know how to do that in network.ftp.client.

If network.ftp.client does not support that out of the box, you'll
either have to find another library or add PASV support to it.

My knowledge of the FTP protocol is pretty rusty, but I believe you the
model after PASV is that you continue using normal commands but the data
transfers happen over the other connection.
I'd watch out for how the protocol marks the beginning and end of a data
stream (it's not a single connection anymore so you lose the sequencing
guarantees of TCP packets; I'd assume they either send a special packet
on the data connection, or they transmit the file sizes via the normal
channel).

However, I can't avoid wondering why anybody would want to use the FTP
protocol at all. It's so unsafe, so bottleneck-endangered, the only
reason to do that would be interfacing with existing FTP servers, and
even these are being phased out in favor of HTTPS, with POST for uploads.
(I'm not questioning the validity of your use case, just wondering how
such a use case can come up in today's world.)

Regards,
Jo
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
Reply | Threaded
Open this post in threaded view
|

Re: upload file with ftp

Joachim Durchholz
Am 06.01.21 um 10:42 schrieb Kees Bleijenberg:
> Joachim, Brandon,
>
> After investigating the source for the store command in  Network.FTP.Client (https://hackage.haskell.org/package/ftp-client-0.5.1.4/docs/Network-FTP-Client.html) I found that the stor command already sets the connection in passive mode. I tried my original code on Linux and everything works fine. So probably the problem is (as always) Windows.

Unlikely it's Windows' network stack - there is nothing special about an
outgoing FTP connection.

> A few years ago I tried some Haskell ftp client libs. On Windows they all raised access violations as soon as I connected to a server. Now,  a few years later the Network library has improved (not sure) and I tried it again. Big improvements: Login, get a list of files, change directory works fine but stor does not.  So there is still no working FTP-client lib for Haskell (I think).

Or the difference is that you have a different firewall setup.
Possibly caused by Windows itself, or maybe some antivirus product.

You should be able to test this hypothesis by using telnet or some other
program that allows you to open any port.

Access violations are typically the result of opening a reserved port
without proper permissions.

> I know FTP is not secure but it is easy to use.

It is not, actually. You have to worry about:
* passive vs. normal mode,
* text vs. binary mode (text mode is obsolete since more than a decade),
* directory listings being misrepresented because
   * encoding issues
   * some FTP server with its own idea of how to represent these
     (manageable with command-line clients, GUI clients may fall over)
* flipped bits (TCP does only so much here),
* aborted transfers (supported but not with client/server mismatch)
* security: your FTP password is visible to anybody in the same network
   segment (including malware they might have caught).

rsync, sftp and scp eliminate every single one of these worries.

 >
Almost everybody uses it for uploading files to a website.

Actually the vast majority of file uploads go through HTTP POST
nowadays. It's been over a decade since I have even seen something with
an FTP upload, though I guess very old services might still stick to it
just because upgrading is a pretty big step for all parties involved.

My knee-jerk reaction to an FTP upload site: Big Red Flag.
Admins who ignore basic security advice, like not offering a server site
that forces you to send passwords in the clear.
(sftp is NOT affected by this. Protocol-wise, it's a totally different
technology.)

> Writing code for uploading a file with http(s)in Haskell is a lot more work than ftp and I wonder whether the https stuff will work on Windows.

Between Linux and Windows, HTTPS does not have any significant differences.
Hmm... well, okay, it is different if you want to use the system's proxy
settings and certificate store. I don't know how that works, so I can't
say much about this.

However, I believe the bigger difference is between HTTP and HTTPS.
Setting up the certificates and doing the connection handshake is a
pretty complicated stuff.
I don't know good the Haskell https libraries are at this; at least for
Java, I can report that the default libraries do work but aren't exactly
good at controlling the connection parameters or accurately reporting
error situations. (That's why everybody uses Apache HttpClient, except
the people who wrote the legacy app I'm tasked with maintaining and
improving... well, if everything worked, there wouldn't interesting
work, so there :-) )

> And  with http(s) you have to write an extra server side program (php) to save the uploaded file, return error messages...
> I just want a simple way to upload a generated file to my private website.

Use the SSH-based protocols then - either sftp/ssh (server side is the
same, but without root access setup can be complicated) or rsync (almost
universally available, and setting up an rsync server should be simple).

Caveat: I don't know whether Haskell libraries for any of these exist.

BTW dealing with HTTP POST is easier than setting up your own server.
Been there, done that, on both sides.

> Until now I upload a file by running a program within my Haskell program. That ftp-program sends a file with ftp to a server. The ftp-program is written in Delphi/Lazarus.

In that case, you'll be better off just falling back to rsync.
Expect to spend an afternoon sifting through the options if you want to
transfer entire directories.
If your server doesn't know how to handle rsync and you can't isntall
it: Use a different provider.

FTP is generally the worst option, in oh so many ways.
Really. Scout's honor.

Regards,
Jo
_______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.