Haproxy, ip_conntrack and problems with it

Written by Koko. Posted in Linux, Web/application servers

Haproxy is a loadbalancer, relatively easy to configure and administer . To not wonder like me why 4 apaches fail on  1000 users , I advise you to pay attention to the loadbalancer configuration and in particular ip_conntrack.

 

What is ip_conntrac?

Kernel module that handles the tracking of the connection.

Since the module has the ability to eat a huge amount of RAM for its activity is limited by /proc/sys/net/ipv4/ip_conntrack_max – text file with the default value 65536 .

Problem …

At peak times , the balancer cant handle all requests  because ip_conntrack table is full and starts to drop packages. In /var/log/ messages/ we have entries like:

ip_conntrack: CT 10097988 : table full, dropping packet.

This leads inevitably to the collapse of the service you provide .

 

It is advisable to raise the value of the above mentioned file as 1000000 . In my oppinion , 1000 000 takes about 300M RAM. To give you an idea of ??the actual  load you can run the following command:

cat /proc/sys/net/ipv4/ip_conntrack_count

This will show the current module values.


Apache load balancing by source IP

Written by Koko. Posted in Web/application servers

As you may already know Apache doesn’t do loadbalancing based on source ip. This gets a very big deal, when you use the apache as a proxy for some application servers, which need sticky session, otherwise something gets wrong.

So a colleague of mine came across with a nice solution based on what ends your IP address.

 

This nice hack uses mod_rewrite and mod_proxy and basically transfers all clients which IPs are ending on 1, 2, 3, 4 and 0 to one of the proxy balancers and if it couldn’t match (mod_rewrite doesn’t work, or IP ending on 5,6,7,8,9) it sends to the other one.

Here’s the code that should be added to your .htaccess or conf file in conf.d/ directory:

 

RewriteLog /var/log/httpd/rewrite.log # Turns on the log
RewriteLogLevel 5 # Log level
<Proxy balancer://cluster1> # Place your IPs here, these will be the application servers.
BalancerMember http://192.168.1.1:8080 route=dns-name-of-the-machine disablereuse=On retry=0
</Proxy># Place your IPs here, these will be the application servers.
<Proxy balancer://cluster2>
BalancerMember http://192.168.1.2:8080 route=dns-name-of-the-machine disablereuse=On retry=0
</Proxy>
RewriteEngine On # Switching on mod_rewrite, if its not on
RewriteCond %{REMOTE_ADDR} 1$|2$|3$|4$|0$ # First condition matching 1,2,3,4,0 ending IPs
RewriteRule ^/path(.*)$ balancer://cluster1/path$1 [P] # change PATH here, according to your path
ProxyPass /path balancer://cluster2/path # Second condition, if the first fails or ip is ending on 5,6,7,8,9
ProxyPassReverse /path balancer://cluster2/path

 

NOTE: Please be careful when you are pasting this in already used configs. If the rewrite rule isn’t working, take a good look if you have another rule that is preventing this one.


Fixing an inaccessible flash drive

Written by Koko. Posted in Network & Storage, Other, Windows

Hello!

 

I’ve recently had a call from a friend of mine which had a very important copy of a document on a flash drive that was corrupted. Basically, when the flash drive was plugged into any USB drive it was saying that it needs to be formatted before using. I knew that the file there was a very important one, so decided not to “kill” it immediately, but first read some user feedback on that problem.

This usually happens when you unplug your flash drive incorrectly. I’m always unplugging my flash drives the stupid way, but I don’t keep something very important on them. I use cloud storage for important files :)

After an hour of reading and trying different stuff I decided to format it - THIS IS THE ONLY WAY OF SAVING SOMETHING FROM THE FLASH DRIVE. 

Right click on it and select Format… You should see the following menu

format

It’s essential to select the quick format option, as it won’t erase the data completely. Also, it’s good to format the flash drive with the same file system as it was before. Usually those flash drives are formatted with FAT32 file system.

Go ahead and click Start. After the format download Recuva - it’s a piece of software that its recovering files.

Start it. On the first screen select Next. You will see a screen like this

recuva1

 

Select Other (show all files) and click Next.

recuva2

 

Select your flash drive’s letter. In my case this is H:. Click Next.

recuva3

 

Make sure you’ve selected Enable deep scan on the last screen. Click start. Wait for the scan to finish.

recuva4

 

Here are your results (it’s possible that you see nothing here. You are screwed in such case :( )

After you’ve selected the files you want to restore, click on the Recover button and select location where you want them restored.

 

Of course you can chose another software for restoring files. I’ve chosen Recuva, because its effective and free.

 


http://sec.kokocorner.com – New kokocorner project

Written by Koko. Posted in Uncategorized

Hello!

 

It’s been a while since my last post so here are some news from me. I’m starting to write articles on regular basis, so we’ll have some nice content out here.

I’ve also launched a second blog focused mainly on mobile/computer/networking security, which I’m recently interested in. I’m currently in a process of posting some content there so please be patient. The official launch is soon!

The address of the new security-oriented blog is http://sec.kokocorner.com

 

P.S.

The blog is still bulgarian only, I’m currently working on the English version too.