
Network Working Group                                          B. Alcock
Request for comments: nnnn                   University of Hertfordshire
Updates: RFC 959                                                May 2000
Category: Experimental


                   Recursive Transfer Mode for FTP


Status of this Memo

   This memo defines an Experimental Protocol for the Internet
   community. This memo does not specify an Internet standard of any
   kind. Discussion and suggestions for improvement are requested.
   Distribution of this memo is unlimited.

Abstract

   This is a proposal for an additional FTP transfer mode that will
   allow entire directory structures to be transferred using a
   persistent connection with the options of preserving permissions
   and compressing files as they are streamed.


1. Introduction

   Although file transfer on the Internet is now dominated by HTTP,
   many people still use FTP, if only to upload web pages to be
   downloaded via HTTP. This extension is intended to be a platform
   independent method of transferring directory trees in an efficient
   manner over a persistent connection.


2. Basic Additions

   The MODE command must now accept the parameter 'R' which will
   identify the Recursive mode. The client must set the TYPE to 'I'
   for the duration of the transfer.


















Alcock                                                          [Page 1]



RFC nnnn           Recursive Transfer Mode for FTP              May 2000

3. Data Stream

   3.1 Structure
   
   The structure of the data stream consists of a series of escaped
   descriptor codes and data.

   The escape character is 0xFF. The sequence 0xFF, 0xFF represents a
   literal 0xFF.

   All numeric values transferred must be in network byte order (big
   endian).

   Descriptor      Meaning
   0x01            File header follows
   0x02            Compressed file header follows
   0x03            Directory header follows
   0x04            Link header follows
   0x05            CRC-32 follows
   0xFC            End of compressed file
   0xFD            End of directory
   0xFE            End of stream
   0xFF            Literal 0xFF


   3.2 Headers

   3.2.1 Standard Header Fields

   Permissions:
   The permissions are UNIX style permissions. They may be ignored or
   overridden by the receiving end. If the sending end does not support
   permissions it must send zero and the receiving end should use its
   default permissions.

   Filename/Directory Name Length:
   This field contains the number of bytes to read for the filename.

   Filename/Directory Name:
   This field holds the name of the file or directory to be created. It
   should not be NULL terminated.

   3.2.2 File Header (0x01)
   
   Size    Description
   4       Permissions
   4       Filename Length
   x       Filename
   4       File Size
   
   A binary image of the file is sent immediately after the header. No


Alcock                                                          [Page 2]



RFC nnnn           Recursive Transfer Mode for FTP              May 2000

   end marker is required to be sent. The File Size field should be used
   by the receiving end to determine how many bytes to read. A CRC-32
   header may follow the data.
   
   3.2.3 Compressed File Header (0x02)

   Size    Description
   4       Permissions
   4       Filename Length
   x       Filename
   4       File Size

   A compressed binary image of the file is sent immediately after the
   header. The format of this compressed image is a the ZLIB compressed
   data format [2]. All occurences of 0xFF bytes must be escaped. The
   File Size field is that of the decompressed file. The data sent
   is terminated using the End of Compressed File escape sequence. A
   CRC-32 header may follow the End of Compressed File escape sequence.

   3.2.3 Directory Header

   Size    Description
   4       Permissions
   4       Directory Name Length
   x       Directory Name

   No data follows this header. Any further headers refer to entries
   within this directory. An End of Directory escape sequence is sent
   to resume transfers in the parent directory.

   3.2.4 Link Header

   Size    Description
   4       Link Name Length
   4       Link Target Length
   x       Link Name
   x       Link Target Name

   The link target must be transmitted as a relative link and NOT an
   absolute link. No data follows this header. The receiving end may
   choose to reject links either due to lack of operating system support
   for links or any other reason the implementor sees fit. Links to
   directories should not be recursed. No data follows this header.


   3.3 End Markers

   3.3.1 Compressed File End Marker (0xFC)

   The compressed file end marker is used to denote the end of the ZLIB
   compressed stream.


Alcock                                                          [Page 3]



RFC nnnn           Recursive Transfer Mode for FTP              May 2000

   3.3.2 Directory End Marker (0xFD)

   The directory end marker is sent to indicate that the directory tree
   entered using the Directory Header has been successfully transferred
   and further transfers should continue in the parent directory.

   3.3.3 Stream End Marker (0xFE)

   This is sent after all transfers have been completed. The number of
   Directory Headers received should equal the number of Directory End
   Markers received. The data connection should be closed once this
   escape sequence is received.


   3.4 Error Checking

   The CRC-32 escape sequence is optional and may be sent after a file
   or compressed file transfer. 4 bytes holding the CRC in network byte
   order will immediately follow the escape sequence. Its value will be
   based upon only the data transferred between the end of the data's
   corresponding header and the start of the CRC-32 escape sequence.


4. The Compressed Stream

   The stream of compressed data is in the ZLIB compressed data format
   [2]. The only deviations from this are that any 0xFF bytes are escaped
   using the escape character 0xFF and the end of the stream is denoted
   using the sequence 0xFF, 0xFC.


5. Example Conversation

   TYPE I
   200 TYPE is I
   MODE R
   200 MODE is R
   PORT 10,1,1,1,10,00
   200 PORT command successful
   RETR dir-name
   150 Data connection opened
   226 Transfer completed
   

6. References

   [1] Postel, J., and J. Reynolds, "FILE TRANSFER PROTOCOL (FTP)",
   RFC 959, October 1985.

   [2] P. Deutsch, and J.L. Gailly, "ZLIB Compressed Data Format 
   Specification version 3.3", RFC 1952, May 1996.


Alcock                                                          [Page 4]

