Here's a straightforward Go http (tcp) connection test script
func main() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "Hello, client")
}))
defer ts.Close()
var wg sync.WaitGroup
for i := 0; i < 2000; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
resp, err := http.Get(ts.URL)
if err != nil {
panic(err)
}
greeting, err := ioutil.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
panic(err)
}
fmt.Printf("%s", i, greeting)
}(i)
}
wg.Wait()
}
And If I run this in Ubuntu I get:
panic: Get http://127.0.0.1:33202: dial tcp 127.0.0.1:33202: too many open files
Other posts say to make sure Close
the connection, which I am doing it all here.
And others say to increase the limit of maximum connection with ulimit
or try sudo sysctl -w fs.inotify.max_user_watches=100000
but still does not work.
How do I run millions of tcp connection goroutines in a single server? It crashes only with 2,000 connections.
Thanks,
I think you need to change your max file descriptors. I have run into the same problem on one of my development VMs before and needed to change the file descriptors max, not anything with inotify settings.
FWIW, your program runs fine on my VM.
·> ulimit -n
120000
But after I run
·> ulimit -n 500
·> ulimit -n
500
I get:
panic: Get http://127.0.0.1:51227: dial tcp 127.0.0.1:51227: socket: too many open files
** Don't fall into the trap that Praveen did **
Note ulimit
!= ulimit -n
.
➜ cmd git:(wip-poop) ✗ ulimit -a
-t: cpu time (seconds) unlimited
-f: file size (blocks) unlimited
-d: data seg size (kbytes) unlimited
-s: stack size (kbytes) 8192
-c: core file size (blocks) 0
-v: address space (kbytes) unlimited
-l: locked-in-memory size (kbytes) unlimited
-u: processes 1418
-n: file descriptors 4864