薛惟仁 筆記本

歡迎來到知識寶庫

0%

[Golang] Unit Testing of HTTP Client using Unix Domain Socket

發表於 分類於 閱讀次數：
HTTP 搭配 TCP/IP 來運作是常見的案例，但實際上 HTTP 本身只假定下層協定會提供可靠的傳輸，所以只要任何滿足可靠傳輸的協定都能成為 HTTP 的下層。在行程間通訊 (IPC, Inter-Process Communication) 應用情境中，較常使用 Unix Domain Socket (UDS) 來取代 TCP/IP，例如：docker 指令透過 Docker Server 建立的 UDS 發送 HTTP Request 給 Docker Server。因此，對開發者來說，知道如何對使用 UDS 作為下層傳輸協定的 HTTP Client 進行測試是重要的。

前言

HTTP 搭配 TCP/IP 來運作是常見的案例，但實際上 HTTP 本身只假定下層協定會提供可靠的傳輸，所以只要任何滿足可靠傳輸的協定都能成為 HTTP 的下層。在行程間通訊 (IPC, Inter-Process Communication) 應用情境中，較常使用 Unix Domain Socket (UDS) 來取代 TCP/IP，例如：docker 指令透過 Docker Server 建立的 UDS 發送 HTTP Request 給 Docker Server。因此，對開發者來說，知道如何對使用 UDS 作為下層傳輸協定的 HTTP Client 進行測試是重要的。

環境

  1. go version go1.19.3

測試手段的選擇

在 [Golang] HTTP Client Unit Test by Mock HTTP Server 一文中已詳細說明不同測試手段的優缺點，這裡同樣會使用 Unit Test by Mock HTTP Server 的手法來撰寫測試案例！

情境說明

API Server (/var/mysock.sock) 提供兩個端點 (Endpoints) 讓我們可以取得或操作使用者相關資訊。

  • GET /api/v1/users 取得使用者資訊

    • 請求成功：回傳 200 OK 狀態碼與所有使用者的名稱

      1
      2
      3
      4
      5
      [
          "Jack",
          "Marry",
          "Sandy"
      ]

    • 請求失敗：回傳 4xx 或 5xx 狀態碼並帶著錯誤訊息

      1
      2
      3
      {
          "msg": "something wrong!"
      }

  • POST /api/v1/user 創建使用者

    • 請求格式：Content-Type 需為 application/json，且 playload 格式如下

      1
      2
      3
      {
          "name": "Jack"
      }

    • 請求成功：回傳 201 Created 狀態碼與該使用者相關資訊

      1
      2
      3
      4
      {
          "id": "ABC-111",
          "name": "Jack"
      }

    • 請求失敗：回傳 4xx 或 5xx 狀態碼並帶著錯誤訊息

      1
      2
      3
      {
          "msg": "something wrong!"
      }

這是虛構出來的情境，我們著重在於 HTTP Client 的測試，至於 HTTP Server 是否存在並不會影響到我們的測試結果，所以這裡不會關心 Server 的實作！

HTTP Client 撰寫與測試

Source Code 已上傳於 GitHub，歡迎自行 Clone/Fork 來運行！以下摘錄重要的部分，每一行代碼的作用皆詳細註解在程式碼中，請耐心閱讀、吸收並化為自身內力。

取得使用者資訊

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
type errorResponse struct {
    Msg string `json:"msg"`
}

func GetUsers(sock string) ([]string, error) {
    // Create an UDS-based http client.
    client := http.Client{
        Transport: &http.Transport{
            DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
                // The default transport protocol for
                // HTTP clients is TCP, which we can
                // modify to UDS by creating a new
                // Unix Domain Socket connection.
                return net.Dial("unix", sock)
            },
        },
    }

    // Send the http request to the server.
    // For UDS-based HTTP, the domain in the URL
    // is not important and is ignored here with
    // an underscore (_).
    resp, err := client.Get("http://_/api/v1/users")
    if err != nil {
        return nil, err
    }

    // Reading and parsing the response body.
    body, err := io.ReadAll(resp.Body)
    if err != nil {
        return nil, err
    }

    if resp.StatusCode == http.StatusOK {
        // If the request is successful,
        // return the user information.
        var data []string
        err = json.Unmarshal(body, &data)
        if err != nil {
            return nil, err
        }
        return data, err
    } else {
        // If it fails, return the "msg" in the
        // response body.
        var data errorResponse
        err = json.Unmarshal(body, &data)
        if err != nil {
            return nil, err
        }
        return nil, errors.New(data.Msg)
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
// NewUnixDomainSocketServer starts and returns a new Server based
// on unix domain socket. The caller should call Close when finished,
// to shut it down and delete the socket file.
func NewUnixDomainSocketServer(handler http.Handler) *httptest.Server {
    // Use a non-existent socket file to create a UDS connection.
    sockPath := "dummy.sock"
    l, err := net.Listen("unix", sockPath)
    if err != nil {
        panic(fmt.Sprintf("httptest: failed to listen on unix domain socket %v: %v", sockPath, err))
    }

    // Create a UDS-based mock http server.
    ts := &httptest.Server{
        Listener: l,
        Config:   &http.Server{Handler: handler},
    }

    // Run the server.
    ts.Start()

    return ts
}

func TestGetUsers(t *testing.T) {
    t.Run("happy path, we can get users info", func(t *testing.T) {
        // Create a router that routes http requests to specific handlers.
        router := http.NewServeMux()

        // We expect to have the mock http server process /api/v1/users
        // while faking its response as we expect it to look.
        router.HandleFunc("/api/v1/users", func(w http.ResponseWriter, r *http.Request) {
            /*
                This is where we can test the format of our request!
                Let's say assert that the request was made with the correct header.
                For example, check if the request comes with an authorization header
                and a value of "bearer xxx".
                assert.Equal(t, "bearer xxx", r.Header.Get("Authorization"))
            */

            // We expect the http method is GET.
            assert.Equal(t, http.MethodGet, r.Method)

            /*
                This is where we mock the response of the the request. That is,
                what we expect the API Server to send back.
            */

            // return 200 OK and users info.
            w.WriteHeader(http.StatusOK)
            w.Write([]byte(`[
                "Jack",
                "Marry",
                "Sandy"
            ]`))
        })

        // Create an UDS-based http server and register the router with a
        // predefined mock handler.
        fakeServer := NewUnixDomainSocketServer(router)

        // We should always close the http server at the end of the test
        // to release related resources and delete the socket file.
        defer fakeServer.Close()

        // The format of the URL from the UDS-based mock http server is
        // 'http://dummy.sock', we only need the part after '//', i.e.
        // 'dummy.sock'.
        sock := strings.Split(fakeServer.URL, "//")[1]

        // Calling a function to be tested.
        users, err := GetUsers(sock)

        // Test the results of the function as we expect.
        assert.NoError(t, err)
        assert.Len(t, users, 3)
        assert.Equal(t, []string{"Jack", "Marry", "Sandy"}, users)
    })

    t.Run("unhappy path, API server has some problems", func(t *testing.T) {
        // Create a router that routes http requests to specific handlers.
        router := http.NewServeMux()

        // We expect to have the mock http server process /api/v1/users
        // while faking its response as we expect it to look.
        router.HandleFunc("/api/v1/users", func(w http.ResponseWriter, r *http.Request) {
            /*
                This is where we can test the format of our request!
                Let's say assert that the request was made with the correct header.
                For example, check if the request comes with an authorization header
                and a value of "bearer xxx".
                assert.Equal(t, "bearer xxx", r.Header.Get("Authorization"))
            */

            // We expect the http method is GET.
            assert.Equal(t, http.MethodGet, r.Method)

            /*
                This is where we mock the response of the the request. That is,
                what we expect the API server to send back.
            */

            // return 500 Internal Server Error.
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte(`{
                "msg": "get error"
            }`))
        })

        // Create an UDS-based http server and register the router with a
        // predefined mock handler.
        fakeServer := NewUnixDomainSocketServer(router)

        // We should always close the http server at the end of the test
        // to release related resources and delete the socket file.
        defer fakeServer.Close()

        // The format of the URL from the UDS-based mock http server is
        // 'http://dummy.sock', we only need the part after '//', i.e.
        // 'dummy.sock'.
        sock := strings.Split(fakeServer.URL, "//")[1]

        // Calling a function to be tested.
        _, err := GetUsers(sock)

        // Test the results of the function as we expect.
        assert.Error(t, err)
        assert.EqualError(t, err, "get error")
    })
}

創建使用者

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
type CreateUserRequest struct {
    Name string `json:"name"`
}

type CreateUserResponse struct {
    ID   string `json:"id"`
    Name string `json:"name"`
}

func CreateUser(sock, userName string) (*CreateUserResponse, error) {
    // Create an UDS-based http client.
    client := http.Client{
        Transport: &http.Transport{
            DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
                // The default transport protocol for
                // HTTP clients is TCP, which we can
                // modify to UDS by creating a new
                // Unix Domain Socket connection.
                return net.Dial("unix", sock)
            },
        },
    }

    // Create a payload that should be POSTed to the server.
    payload := CreateUserRequest{
        Name: userName,
    }

    // Encode the payload into json format.
    var buf bytes.Buffer
    err := json.NewEncoder(&buf).Encode(payload)
    if err != nil {
        return nil, err
    }

    // Create a new http POST request with the payload
    // and modify the Content-Type header.
    // For UDS-based HTTP, the domain in the URL
    // is not important and is ignored here with
    // an underscore (_).
    req, err := http.NewRequest(http.MethodPost, "http://_/api/v1/user", &buf)
    if err != nil {
        return nil, err
    }
    req.Header.Add("Content-Type", "application/json")

    // Send the http request to the server.
    resp, err := client.Do(req)
    if err != nil {
        return nil, err
    }

    // Reading and parsing the response body.
    body, err := io.ReadAll(resp.Body)
    if err != nil {
        return nil, err
    }

    if resp.StatusCode == http.StatusCreated {
        // If the request is successful,
        // return the user information.
        var data CreateUserResponse
        err = json.Unmarshal(body, &data)
        if err != nil {
            return nil, err
        }
        return &data, nil
    } else {
        // If it fails, return the "msg" in the
        // response body.
        var data errorResponse
        err = json.Unmarshal(body, &data)
        if err != nil {
            return nil, err
        }
        return nil, errors.New(data.Msg)
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
// NewUnixDomainSocketServer starts and returns a new Server based
// on unix domain socket. The caller should call Close when finished,
// to shut it down and delete the socket file.
func NewUnixDomainSocketServer(handler http.Handler) *httptest.Server {
    // Use a non-existent socket file to create a UDS connection.
    sockPath := "dummy.sock"
    l, err := net.Listen("unix", sockPath)
    if err != nil {
        panic(fmt.Sprintf("httptest: failed to listen on unix domain socket %v: %v", sockPath, err))
    }

    // Create a UDS-based mock http server.
    ts := &httptest.Server{
        Listener: l,
        Config:   &http.Server{Handler: handler},
    }

    // Run the server.
    ts.Start()

    return ts
}

func TestCreateUser(t *testing.T) {
    t.Run("happy path, API server has some problems", func(t *testing.T) {
        // Create a router that routes http requests to specific handlers.
        router := http.NewServeMux()

        // We expect to have the mock http server process /api/v1/user
        // while faking its response as we expect it to look.
        router.HandleFunc("/api/v1/user", func(w http.ResponseWriter, r *http.Request) {
            /*
                This is where we can test the format of our request!
                Let's say assert that the request was made with the correct header.
                For example, check if the request comes with an authorization header
                and a value of "bearer xxx".
                assert.Equal(t, "bearer xxx", r.Header.Get("Authorization"))
            */

            // We expect the http method is POST.
            assert.Equal(t, http.MethodPost, r.Method)

            // Check if the Content-Type header is application/json.
            assert.Equal(t, "application/json", r.Header.Get("Content-Type"))

            // Check the payload format of the request.
            body, err := io.ReadAll(r.Body)
            assert.NoError(t, err)
            assert.JSONEq(t, `{"name": "Jack"}`, string(body))

            /*
                This is where we mock the response of the the request. That is,
                what we expect the API Server to send back.
            */

            // return 201 Created and user info.
            w.WriteHeader(http.StatusCreated)
            w.Write([]byte(`{
                "id": "id_foo",
                "name": "name_foo"
            }`))
        })

        // Create an UDS-based http server and register the router with a
        // predefined mock handler.
        fakeServer := NewUnixDomainSocketServer(router)

        // We should always close the http server at the end of the test
        // to release related resources and delete the socket file.
        defer fakeServer.Close()

        // The format of the URL from the UDS-based mock http server is
        // 'http://dummy.sock', we only need the part after '//', i.e.
        // 'dummy.sock'.
        sock := strings.Split(fakeServer.URL, "//")[1]

        // Calling a function to be tested.
        user, err := CreateUser(sock, "Jack")

        // Test the results of the function as we expect.
        assert.NoError(t, err)
        assert.Equal(t, "id_foo", user.ID)
        assert.Equal(t, "name_foo", user.Name)
    })
    t.Run("unhappy path, some error occur", func(t *testing.T) {
        // Create a router that routes http requests to specific handlers.
        router := http.NewServeMux()

        // We expect to have the mock http server process /api/v1/user
        // while faking its response as we expect it to look.
        router.HandleFunc("/api/v1/user", func(w http.ResponseWriter, r *http.Request) {
            /*
                This is where we can test the format of our request!
                Let's say assert that the request was made with the correct header.
                For example, check if the request comes with an authorization header
                and a value of "bearer xxx".
                assert.Equal(t, "bearer xxx", r.Header.Get("Authorization"))
            */

            // We expect the http method is POST.
            assert.Equal(t, http.MethodPost, r.Method)

            // Check if the Content-Type header is application/json.
            assert.Equal(t, "application/json", r.Header.Get("Content-Type"))

            // Check the payload format of the request.
            body, err := io.ReadAll(r.Body)
            assert.NoError(t, err)
            assert.JSONEq(t, `{"name": "Jack"}`, string(body))

            /*
                This is where we mock the response of the the request. That is,
                what we expect the API Server to send back.
            */

            // return 500 Internal Server Error.
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte(`{
                "msg": "get error"
            }`))
        })

        // Create an UDS-based http server and register the router with a
        // predefined mock handler.
        fakeServer := NewUnixDomainSocketServer(router)

        // We should always close the http server at the end of the test
        // to release related resources and delete the socket file.
        defer fakeServer.Close()

        // The format of the URL from the UDS-based mock http server is
        // 'http://dummy.sock', we only need the part after '//', i.e.
        // 'dummy.sock'.
        sock := strings.Split(fakeServer.URL, "//")[1]

        // Calling a function to be tested.
        _, err := CreateUser(sock, "Jack")

        // Test the results of the function as we expect.
        assert.Error(t, err)
        assert.EqualError(t, err, "get error")
    })
}

不難看出 NewUnixDomainSocketServer 是我們撰寫測試的關鍵，因為 httptest 官方套件並沒有提供 UDS-based HTTP Server，所以我們透過 net.Listen 建立一個 UDS Connection，再將該 Connection 提供給 httptest.Server 改變底層的傳輸協定，最後啟動 Server 作為測試用的 UDS-based Mock HTTP Server！

結語

Golang 官方與社群的力量雖然強大，但並不是每個功能都會有的，所以我們除了學會使用既有的 Library，同樣需要理解這些代碼背後的原理，才能站在巨人肩膀上來根據需求衍生出新的功能。倘若心有餘力，我們可以將新功能回饋給社群，形成良好的循環！

參考文獻

  1. Hypertext Transfer Protocol

  2. Inter-process communication

  3. Unix domain socket

  4. What Are Unix Sockets and How Do They Work?

  5. Docker Socket File for IPC

  6. 4.2.4 Connecting to the MySQL Server Using Command Options

很高興能在這裡幫助到您，歡迎登入 Liker 為我鼓掌 5 次，或者成為我的讚賞公民，鼓勵我繼續創造優質文章。
相關文章
以最優質的內容回應您的鼓勵