I want to make multiple HTTP requests in only one connection with using concurrency, but my code makes about four or five. But when I don't use concurrency, it will only have one connection.

package main

import (
    "fmt"
    "net/http"
    "net/url"
    "sync"
    "time"
)

func main() {
    proxyURL, err := url.Parse("http://localhost:9099")//In the traffic sniffer tool I see this program makes 5 requests
    if err != nil {
        fmt.Println(err)
        return
    }
    
    var wg sync.WaitGroup
    client := &http.Client{
        Transport: &http.Transport{
            MaxIdleConns:        10,
            IdleConnTimeout:     90 * time.Second,
            DisableKeepAlives:   false,
            Proxy: http.ProxyURL(proxyURL),
        },}

    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func(index int) {
            defer wg.Done()
            resp, err := client.Get(";)
            if err != nil {
                fmt.Println(err)
                return
            }
            err = resp.Body.Close()
            if err != nil {
                fmt.Println(err)
                return 
            }
            fmt.Println(index)
        }(i)
    }

    wg.Wait()
    fmt.Println("done")
}

Edit: Thanks to you, but I found another alternative solution: just establish a connecition before running goroutine, it's faster than setting "MaxConnsPerHost: 1" when there are lot of requests

I want to make multiple HTTP requests in only one connection with using concurrency, but my code makes about four or five. But when I don't use concurrency, it will only have one connection.

package main

import (
    "fmt"
    "net/http"
    "net/url"
    "sync"
    "time"
)

func main() {
    proxyURL, err := url.Parse("http://localhost:9099")//In the traffic sniffer tool I see this program makes 5 requests
    if err != nil {
        fmt.Println(err)
        return
    }
    
    var wg sync.WaitGroup
    client := &http.Client{
        Transport: &http.Transport{
            MaxIdleConns:        10,
            IdleConnTimeout:     90 * time.Second,
            DisableKeepAlives:   false,
            Proxy: http.ProxyURL(proxyURL),
        },}

    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func(index int) {
            defer wg.Done()
            resp, err := client.Get("https://www.httpbin./get")
            if err != nil {
                fmt.Println(err)
                return
            }
            err = resp.Body.Close()
            if err != nil {
                fmt.Println(err)
                return 
            }
            fmt.Println(index)
        }(i)
    }

    wg.Wait()
    fmt.Println("done")
}

Edit: Thanks to you, but I found another alternative solution: just establish a connecition before running goroutine, it's faster than setting "MaxConnsPerHost: 1" when there are lot of requests

• go
• http

Share Improve this question Follow edited Jan 23 at 3:34 HenTaku asked Jan 17 at 17:54 HenTakuHenTaku 1155 bronze badges 2

• go fmt.Println(index) ? – Николай Лубышев Commented Jan 19 at 7:52
• If you want the requests to be concurrent, then you will have multiple connections. You can’t send multiple requests at the same time over the same connection. You are also not reading the response, and if the body isn’t drained the connection can’t be reused – Mr_Pink Commented Jan 20 at 19:12

Add a comment  | 

2 Answers 2

Sorted by: Reset to default 0

You can consider using MaxConnsPerHost

// MaxConnsPerHost optionally limits the total number of
// connections per host, including connections in the dialing,
// active, and idle states. On limit violation, dials will block.
//
// Zero means no limit.
MaxConnsPerHost int

usage

Transport: &http.Transport{
   MaxConnsPerHost: 1,
   ..rest of config...
}

A note: if the remote server doesn't support keep-alive, you might not reuse the same connection but open a new one every time

`Body 'is still open until the function is finished:

defer resp.Body.Close()

instead of try:

resp.Body.Close()