Differences between current version and revision by previous author of HowToCompressedTCP.
Other diffs: Previous Major Revision, Previous Revision, or view the Annotated Edit History
| Newer page: | version 3 | Last edited on Thursday, October 21, 2004 5:24:34 pm | by AristotlePagaltzis | |
| Older page: | version 2 | Last edited on Friday, June 7, 2002 1:06:25 am | by perry | Revert |
@@ -1,276 +1 @@
-
-
-
-Compressed TCP/IP-Sessions using SSH-like tools
-
-
-
-----
-
-!!!Compressed TCP/IP-Sessions using SSH-like tools
-
-!!Sebastian Schreiber <Schreib@SySS
.de>
-2.2.2000
-
-
-
-
-!!1. Introduction
-
-
-
-
-!!2. Compressing HTTP/FTP,...
-
-
-
-
-!!3. Compressing Email
-
-
-*3.1 Incoming Emails (POP3, IMAP4)
-
-*3.2 Outgoing Email (SMTP)
-
-
-
-
-
-!!4. Thoughts about performance.
-
-
-
-
-!!5. Greetings
-----
-
-!!1. Introduction
-
-
-In the past, we used to compress files in order to save disk space. Today,
-disk space is cheap - but bandwidth is limited. By compressing data streams,
-you achieve two goals:
-
-
-1) You save bandwidth/transfered volume (that is important if you have
-to pay for traffic or if your network is loaded.).
-
-
-2) Speeding up low-bandwidth connections (Modem, GSM, ISDN).
-
-
-This !HowTo explains how to save both bandwith and connection time by using
-tools like SSH1, SSH2, OpenSSH or LSH.
-----
-
-!!2. Compressing HTTP/FTP,...
-
-
-My office is connected with a 64KBit ISDN line to the internet, so the
-maximum transfer rate is about 7K/s. You can speed up the connection by compressing
-it: when I download files, Netscape shows up a transfer rate of up to 40K/s
-(Logfiles are compressable by factor 15). SSH is a tool that is mainly designed
-to build up secure connections over unsecured networks. Further more, SSH is
-able to compress connections and to do port forwarding (like rinetd or redir).
-So it is the appropriate tool to compress any simple TCP/IP connection. "Simple"
-means, that only one TCP-connection is opened. An FTP-connections or the connection
-between M$-Outlook and MS-Exchange are not simple as several connections
-are established. SSH uses the !LempleZiv (LZ77) compression algorithm - so you
-will achieve the same high compression rate as winzip/pkzip. In order to compress
-all HTTP-connections from my intranet to the internet, I just have to execute
-one command on my dial-in machine:
-
-
-ssh -l <login ID> <hostname> -C -L8080:<proxy_at_ISP>:80
--f sleep 10000
-
-
-<hostname> = host that is located at my ISP. SSH-access is required.
-
-
-<login ID> = my login-ID on <hostname>
-
-
-<proxy_at_ISP> =the web proxy of my ISP
-
-
-My browser is configured to use localhost:8080 as proxy. My laptop connects
-to the same socket. The connection is compressed and forwarded to the real
-proxy by SSH. The infrastructure looks like:
-
-
-
-
-64KBit ISDN
-My PC--------------------------------A PC (Unix/Linux/Win-NT) at my ISP
-SSH-Client compressed SSH-Server, Port 22
-Port 8080 |
-| |
-| |
-| |
-|10MBit Ethernet |100MBit
-|not compressed |not compressed
-| |
-| |
-My second PC ISP's WWW-proxy
-with Netscape,... Port 80
-(Laptop)
-
-----
-
-!!3. Compressing Email
-
-!!3.1 Incoming Emails (POP3, IMAP4)
-
-
-
-Most people fetch their email from the mailserver via POP3. POP3 is a protocol
-with many disadvantages:
-
-
-
-
-
-#POP3 transfers password in clear text. (There are SSL-implementations of
-POP/IMAP and a challenge/response authentication, defined in RFC-2095/2195).
-#
-
-#POP3 causes much protocol overhead: first the client requests a message
-than the server sends the message. After that the client requests the transferred
-article to be deleted. The server confirms the deletion. After that the server
-is ready for the next transaction. So 4 transactions are needed for each email.
-#
-
-#POP3 transfers the mails without compression although email is highly compressible
-(factor=3.5).
-#
-
-
-
-You could compress POP3 by forwarding localhost:110 through a compressed
-connection to your ISP's POP3-socket. After that you have to tell your mail
-client to connect to localhost:110 in order to download mail. That secures
-and speeds up the connection -- but the download time still suffers from the
-POP3-inherent protocol overhead.
-
-
-
-
-
-It makes sense to substitute POP3 by a more efficient protocol. The idea
-is to download the entire mailbox at once without generating protocol overhead.
-Furthermore it makes sense to compress the connections. The appropriate tool
-which offers both features is SCP. You can download your mail-file like this:
-
-
-
-
-
-scp -C -l loginId:/var/spool/mail/loginid /tmp/newmail
-
-
-
-
-
-But there is a problem: what happens if a new email arrives at the server
-during the download of your mailbox? The new mail would be lost. Therefore
-it makes more sense to use the following commands:
-
-
-ssh -l loginid mailserver -f mv /var/spool/mail/loginid /tmp/loginid_fetchme
-
-
-scp -C -l loginid:/tmp/my_new_mail /tmp/loginid_fetchme
-
-
-A move (mv) is a elementary operation, so you won't get into truble if
-you receive new mail during the execution of the comands. But if the mail server
-directories /tmp/ and /var/spool/mail are not on the same disc you might get
-problems. A solution is to create a lockfile on the server before you execute
-the mv: touch /var/spool/mail/loginid.lock. You should remove it, after that.
-A better solution is to move the file loginid in the same directory:
-
-
-ssh -l loginid mailserver -f mv /var/spool/mail/loginid /var/spool/mail/loginid_fetchme
-
-
-After that you can use formail instead of procmail in order to filter /tmp/newmail
-into the right folder(s): formail -s procmail < /tmp/newmail
-
-!!3.2 Outgoing Email (SMTP)
-
-
-
-You send email over compresses and encrypted SSH-connections, in order
-to:
-
-
-
-
-
-*Save network traffic
-*
-
-*Secure the connection (This does not make sense, if the mail is transported
-over untrusted networks, later.)
-*
-
-*Authenticate the sender.
-Many mail servers deny mail relaying in order
-to prevent abuse. If you send an email over an SSH-connection, the remote mail
-server (i.e. sendmail or MS-exchange) thinks to be connected, locally.
-*
-
-
-
-If you have SSH-access on the mail server, you need the following command:
-
-
-ssh -C -l loginid mailserver -L2525:mailserver:25
-
-
-If you don't have SSH-access on the mail server but to a server that is
-allowed to use your mail server as relay, the command is:
-
-
-ssh -C -l loginid other_server -L2525:mailserver:25
-
-
-After that you can configure your mail client (or mail server: see "smarthost")
-to send out mails to localhost port 2525.
-----
-
-!!4. Thoughts about performance.
-
-
-Of course compression/encryption takes CPU time. It turned out that an
-old Pentium-133 is able to encrypt and compress about 1GB/hour -- that's quite
-a lot. If you compile SSH with the option "--with-none" you can
-tell SSH to use no encryption. That saves a little performance. Here is a comprise
-between several download methods (during the test, a noncompressed 6MB-file
-was transfered from a 133MHz-Pentium-1 to a 233MHz Pentium2 laptop over a 10MBit
-ethernet without other load).
-
-
-
-
-+-------------------+--------+----------+-----------+----------------------+
-| | FTP |encrypted |compressed |compressed & encrypted|
-+-------------------+--------+----------+-----------+----------------------+
-+-------------------+--------+----------+-----------+----------------------+
-| Elapsed Time | |7.6s | 26s | 9s | 23s |
-+-------------------+--------+----------+-----------+----------------------+
-| Throughput | 790K/s | 232K/s | 320K/s | 264K/s |
-+-------------------+--------+----------+-----------+----------------------+
-|Compression Factor | 1 | 1 | 3.8 | 3.8 |
-+-------------------+--------+----------+-----------+----------------------+
-
-----
-
-!!5. Greetings
-
-
-Thanks to Harald Koumlnig <koenig@tat.physik.uni-tuebingen.de>, who
-used rcp in order to download complete mailboxes. The latest version of this
-howto is available on http://www.syss.de/howto.
-----
+Describe [HowToCompressedTCP] here
.
