Linux: Kernel: Network

[ Bloke.com || Linux || JavaScript || Java || Volleyball || Link Me ]
Free: [ Guestbook || MessageBot || Plugins || Counter || AusPrices || Advertise ]

Home - Linux - Kernel - Network

Don't try this yourself, you might break something. These recommendations were from Pete Vogel, who runs a bigger site than you do (can you do 85Mb/s on just one server? .. and he has many!!). Don't change these params unless you understand them. I am working on adding explainations. Amongst other things (like breaking your network connectivity), you might actually make your machine slower if you just blindly use them. In fact, I only set a couple of them. And always best to set one at a time to see if you notice the improvement you are looking for.

echo 65534 > /proc/sys/fs/file-max
echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
echo 0 > /proc/sys/net/ipv4/tcp_window_scaling
echo 0 > /proc/sys/net/ipv4/tcp_timestamps
echo 0 > /proc/sys/net/ipv4/tcp_sack
echo 0 > /proc/sys/net/ipv4/tcp_ecn
echo 2 > /proc/sys/net/ipv4/tcp_orphan_retries
echo 1 > /proc/sys/net/ipv4/tcp_syncookies

/proc/sys/fs/file-max: This is basically the number of file descriptors available in the kernel. Which also effects the number of fd's a process can have open. For large sites you will definately need to upgrade this, and for some os'es you will need to use ulimit to increase the number of fd's available for that process.
For red-hat (and other linux's) you can use:
```
echo 32767 > /proc/sys/fs/file-max
```
which I put into /etc/rc.d/rc.local (at/near the end). You can check your current usage using:
```
[cameron@jack cameron]$ cat /proc/sys/fs/file-nr
5941	2844	32767
[cameron@jack cameron]$ 
```
and what this means is that since the kernel has been running the max number of fd's in use was 5941, the current number is 2844, and the max available is 32767.
/proc/sys/net/ipv4/tcp_tw_recycle:
```
BOOLEAN
	Enable fast recycling TIME-WAIT sockets. Default value is 1.
	It should not be changed without advice/request of technical
	experts.
```
The definition of TIME_WAIT in "man netstat" is
```
       TIME_WAIT
              The socket is waiting after close to handle packets
              still in the network.
```
On Red Hat, the default is 0, so I checked on one of the loaded machines to see 2500 sockets in the TIME_WAIT state
```
[cameron@jack cameron]$ netstat -n | grep TIME_WAIT | wc
   2446   14676  198126
[cameron@jack cameron]$ netstat -n | wc
   2847   17104  230009
```
As you can see, for me, this takes up a lot of the network resources. By turning this on, I saw a slight drop in the number of connections (about 200), and a definate drop in the number of TIME_WAIT connections. I figured that this was good. I see that machine handling slightly more traffic, but my measuring mechanism is "poor". This is the next day after I made the change (at a busier time of the day).
```
[cameron@jack httpd-2.0.35]$ netstat -n | grep TIME_WAIT | wc ; netstat -n | wc
   2291   13746  185571
   2905   17452  234696
[cameron@jack httpd-2.0.35]$ 
```
So by turning it on, you should see a reduction in the number of FD's used in the kernel, but you might get some extra junk traffic??
/proc/sys/net/ipv4/tcp_window_scaling:
```
BOOLEN
	Enable window scaling as defined in RFC1323
```
These are recommendations to make TCP/IP work over very high speeds. I left the default at 1.

/proc/sys/net/ipv4/tcp_timestamps: Todo

BOOLEAN
	Enable timestamps as defined in RFC1323

an except:

      On the other hand, a Timestamps option may appear in any data or
      ACK segment, adding 12 bytes to the 20-byte TCP header.  We
      believe that the bandwidth saved by reducing unnecessary
      retransmissions will more than pay for the extra header bandwidth.

/proc/sys/net/ipv4/tcp_sack:

BOOLEAN
	Enable select acknowledgments (SACKS).

/proc/sys/net/ipv4/tcp_ecn:

BOOLEAN
	Enable Explicit Congestion Notification in TCP.

/proc/sys/net/ipv4/tcp_orphan_retries:

INTEGER
	How may times to retry before killing TCP connection, closed
	by our side. Default value 7 corresponds to ~50sec-16min
	depending on RTO. If you machine is loaded WEB server,
	you should think about lowering this value, such sockets
	may consume significant resources. Cf. tcp_max_orphans.

/proc/sys/net/ipv4/tcp_syncookies:

BOOLEAN
	Only valid when the kernel was compiled with CONFIG_SYNCOOKIES
	Send out syncookies when the syn backlog queue of a socket 
	overflows. This is to prevent against the common 'syn flood attack'
	Default: FALSE

	Note, that syncookies is fallback facility.
	It MUST NOT be used to help highly loaded servers to stand
	against legal connection rate. If you see synflood warnings
	in your logs, but investigation	shows that they occur
	because of overload with legal connections, you should tune
	another parameters until this warning disappear.
	See: tcp_max_syn_backlog, tcp_synack_retries, tcp_abort_on_overflow.

	syncookies seriously violate TCP protocol, do not allow
	to use TCP extensions, can result in serious degradation
	of some services (f.e. SMTP relaying), visible not by you,
	but your clients and relays, contacting you. While you see
	synflood warnings in logs not being really flooded, your server
	is seriously misconfigured.

MTU on DSL lines
It is said that DSL wraps packets in it's own packet. So for some DSL connections decreasing your MTU (Maximum Transfer Unit), from 1500 to 1490 is advantagous. You can do this using:

 ifconfig eth0 mtu 1490

Which I then added to /etc/rc.d/rc.local (for red hat 7.2)

Links:

http://hr.uoregon.edu/davidrl/Documentation/networking/ip-sysctl.txt - local copy

Last Change: Sunday, 06-Dec-2009 20:07:29 EST

Disclaimer

The information provided within these pages is provided AS IS, and without any warranty. Following these directions may (but not limited to) crash your computer, delete all the information on your hard disk, open up security holes or cause your house to burn down. I made these pages to provide some information about the setup that I have done, but I did not proofread it for correctness, and in most cases did not test it. There are commands in these pages that would definately delete or corrupt all the data on your computer (especially the dualboot section). In fact it happened to me.... So you are on your own!

Cameron Gregory