Skip to content

Lots Of Proxies from 1 Server using 3Proxy

My goal was to get 8k IP’s setup on a single server I’d never done this before but someone mentioned you could so I figured it was possible.

As mentioned in the title the proxy server we’re going to use is a Russian server called 3proxy. The beauty of 3proxy is it’s ultra lightweight, fast and you can run multiple IP’s in a single daemon because it has internal threads. Check out http://3proxy.ru/ which is the home of 3proxy.

We started off by getting all the IP’s pointed to the beefy dedicated server and then started playing with proxy software. We first started off with tinyproxy without much success. I had never used this before but my hosting company suggested and mentioned it would work for this setup. Unfortunately when you try to load 8k IP’s it starts a new daemon process in the OS for each IP and will eventually crash.

Next we moved to my old standby I’ve been using for a few years now, 3proxy which we tried the same thing as we did with tinyproxy which was start a daemon for each individual IP. We tried this first as this is how I have all my other servers configured and thought 3proxy might have a smaller footprint. Guess what? This crashed just the same. The next config was throwing 8k IP’s in a single daemon. CRASH! After that we gave 4 daemon’s, 2k per a shot. This actually ran but excrutiatingly slow. The load times of pages was 10x’s what it was running a single c-block and single daemon which was our base test case. What was interesting was the server was very responsive with a super low load using this setup, but for some reason the proxies were very unresponsive. I don’t know much about the internal coding of 3proxy but I imagine it has something to do with how it’s designed internally. Lastly I decided I’d try running an individual daemon for each c-block all at the same time. Now, for each daemon we need to run it on a different listening port. This was the magic bullet, seems if you don’t put to many IP’s in a single daemon they run smooth. As of right now I’m running about a 8 server load which high but things are still running very smoothly and the load times are almost as fast as not using a proxy at all.

If you want to run a lot of proxies on single server I ‘d suggest 3proxy which is a nice free opensource piece of software. Then run no more then 255 IP’s per daemon. The number of daemon’s per server you’re going to be able to run is solely dependent on how good the server is.

P.S. That fellow who told me I could run them all on the the same server later said no you couldn’t after the first couple of attempts. Also mentioned he’d never done it before.

If you’re interested in running a setup like this here’s a couple things to help.

1. This first script is going to make your config files for your daemons that 3proxy needs to run. These config’s are set as open proxies right and you should put some IP whitelisting or login/password protection on them. Read more on the 3proxy website about how to do this.

<?
// this is the port we'll start incrementing from
$port = 22406;
// first class C
$beginningclassc = 122;
//last class C
$endingclassc = 224;
//this is our class C id's like this XXX.XXX.122
for($c = $beginningclassc;$c <= $endingclassc;$c++)
{
// these are our settings for our config file
$filecontents = "#!/usr/local/bin/3proxy
nserver 127.0.0.1
nscache 65536
timeouts 1 5 30 60 180 1800 15 60
log /usr/home/3proxy.log
daemon
#auth iponly
auth none
dnspr
flush
auth none
";
// adding each of the IP's to the config file
for($i=2;$i<255;$i++)
{
$filecontents .= "external 111.222.$c.$i\n";
$filecontents .= "internal 111.222.$c.$i\n";
//
$filecontents .= "proxy -a -n -p$port\n";
}
// increments the port for the next loop
$port++;
// saves the file to a dir of your choosing with a filename
// that has the C segment in it for identification
file_put_contents("c:/files/proxy-$c.cfg",$filecontents);
}
?>


2. This second script is a bash script that runs on your server that will start each of the daemons. Instead of running
>3proxy proxy-112.cfg
all you have to do is run this script on your server and it’ll start them all for you.

#!/bin/bash
COUNTER=122
while [ $COUNTER -lt 224 ]; do
3proxy /root/3proxy/proxy-$COUNTER.cfg
let COUNTER=COUNTER+1
done

If you don’t know how to use this make a file on your server called “proxy-start”. Then chmod it 777 by the command chmod 777 proxy-start. Then just run it using ./proxy-start. That should get all your blocks up and running. If you have an issue you can just do a killall -9 3proxy which will kill all your 3proxy processes and you can start again.

Hope this helps someone πŸ™‚

5 Comments

  1. Am I the only one left wondering the most important piece of information left missing from this article–namely how to get 8k IP addresses?

  2. cpaking cpaking

    just shell out some big money and ask your DC nicely πŸ™‚

  3. Jeremy Jeremy

    That’s what I’m wondering. I have problems getting a hundred, and find it next to impossible to get any portion of them on different C-classes.

  4. Well for small time guys it’s easy to get IP’s on different c-blocks. Just get low cost reseller hosting one a bunch of different hosts. As far as large scale IP address acquisition you really have to know people. If you just go to a host without having a long history or a referral most of the time you’re going to get shut down.

    If you’re looking for private paid proxies for scraping or things like that drop me a line we can probably work something out. Mail to [email protected][thisdomain].com πŸ™‚

  5. nogenius nogenius

    For larger scale proxy projects such as this, is it out of the question to use SQUID?

Comments are closed.