薛惟仁 筆記本

歡迎來到知識寶庫

0%

[Golang] HTTP Client Unit Test by Mock HTTP Server

發表於 分類於 閱讀次數：
HTTP 是廣受使用的傳輸協定之一，其為客戶端 (Client) 與伺服器 (Server) 通訊訂定一個 Request/Response 標準，可以輕易地使用瀏覽器、爬蟲工具等遵守 HTTP 協定的 Client 來對 Server 發出請求以取得資料。在程式開發中，透過 HTTP 呼叫第三方 Server 的 API 是常見的一個行為，例如：LINE Notify、Firebase 等服務，背後皆是使用 HTTP 與其 Server 交換資料。為了確保邏輯正確性且提高代碼品質，正確撰寫 HTTP Client 的測試就變得格外重要。

前言

HTTP 是廣受使用的傳輸協定之一，其為客戶端 (Client) 與伺服器 (Server) 通訊訂定一個 Request/Response 標準，可以輕易地使用瀏覽器、爬蟲工具等遵守 HTTP 協定的 Client 來對 Server 發出請求以取得資料。在程式開發中，透過 HTTP 呼叫第三方 Server 的 API 是常見的一個行為，例如：LINE Notify、Firebase 等服務，背後皆是使用 HTTP 與其 Server 交換資料。為了確保邏輯正確性且提高代碼品質，正確撰寫 HTTP Client 的測試就變得格外重要。

環境

  1. go version go1.19.3

測試手段的選擇

正確選擇測試手段是非常重要的，好的測試方法會 帶我們飛 帶來好維護的測試代碼。

Integration Test

每次測試皆對 Server 發出真實請求以取得真實回應，想法直接但有幾個值得思考的問題

  1. 若真的對 Server 發出請求，要怎麼確保每次 Server 的回覆都固定不變，讓我們的測試可以順利通過？
  2. 若測試時 Server 出現內部問題 (例如：關機維護)，此時測試不會通過，但不代表我們的代碼有錯。
  3. 若是使用以呼叫次數計費的 API，代表每次運行測試都有一定的金錢成本在。

這些問題的根本原因都是一樣的，是因為 Integration Test 依賴於真實 Server，若我們可以讓測試不依賴真實 Server，則這些問題就不會存在，Mock HTTP Server 為這些問題的一種解決方案。

Unit Test by Mock HTTP Server

某種程度上可說這種方法是 Integration Test，但測試本身不受外部依賴的運作行為干擾，因為現在依賴的是 Mock HTTP Server，使用完全自己假造的 HTTP Server 取代真實 Server 來進行測試，這樣我們可以確保

  1. 每次請求都會以我們事先撰寫內容作為回應，這個內容是永恆不變的。
  2. Mock HTTP Server 不會有內部問題，因此不影響測試結果。
  3. API Server 是我們假造的，所以也免除了費用問題。

綜合上述，Unit Test by Mock HTTP Server 將會是我們採用的測試手段！

情境說明

API Server (http://example.com) 提供兩個端點 (Endpoints) 讓我們可以取得或操作使用者相關資訊。

  • GET /api/v1/users 取得使用者資訊

    • 請求成功：回傳 200 OK 狀態碼與所有使用者的名稱

      1
      2
      3
      4
      5
      [
          "Jack",
          "Marry",
          "Sandy"
      ]

    • 請求失敗：回傳 4xx 或 5xx 狀態碼並帶著錯誤訊息

      1
      2
      3
      {
          "msg": "something wrong!"
      }

  • POST /api/v1/user 創建使用者

    • 請求格式：Content-Type 需為 application/json，且 playload 格式如下

      1
      2
      3
      {
          "name": "Jack"
      }

    • 請求成功：回傳 201 Created 狀態碼與該使用者相關資訊

      1
      2
      3
      4
      {
          "id": "ABC-111",
          "name": "Jack"
      }

    • 請求失敗：回傳 4xx 或 5xx 狀態碼並帶著錯誤訊息

      1
      2
      3
      {
          "msg": "something wrong!"
      }

這是虛構出來的情境，example.com 是為製作教學文件提供方便而存在的網站，我們可以真的去呼叫 http://example.com/api/v1/users，但他的回應不會如我們預期。

HTTP Client 撰寫與測試

Source Code 已上傳於 GitHub，歡迎自行 Clone/Fork 來運行！以下摘錄重要的部分，每一行代碼的作用皆詳細註解在程式碼中，請耐心閱讀、吸收並化為自身內力。

取得使用者資訊

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
type errorResponse struct {
    Msg string `json:"msg"`
}

func GetUsers(baseUrl string) ([]string, error) {
    url := baseUrl + "/api/v1/users"

    // Send the http request to the server.
    resp, err := http.Get(url)
    if err != nil {
        return nil, err
    }

    // Reading and parsing the response body.
    body, err := io.ReadAll(resp.Body)
    if err != nil {
        return nil, err
    }

    if resp.StatusCode == http.StatusOK {
        // If the request is successful,
        // return the user information.
        var data []string
        err = json.Unmarshal(body, &data)
        if err != nil {
            return nil, err
        }
        return data, err
    } else {
        // If it fails, return the "msg" in the
        // response body.
        var data errorResponse
        err = json.Unmarshal(body, &data)
        if err != nil {
            return nil, err
        }
        return nil, errors.New(data.Msg)
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
func TestGetUsers(t *testing.T) {
    t.Run("happy path, we can get users info", func(t *testing.T) {
        // Create a router that routes http requests to specific handlers.
        router := http.NewServeMux()

        // We expect to have the mock http server process /api/v1/users
        // while faking its response as we expect it to look.
        router.HandleFunc("/api/v1/users", func(w http.ResponseWriter, r *http.Request) {
            /*
                This is where we can test the format of our request!
                Let's say assert that the request was made with the correct header.

                For example, check if the request comes with an authorization header
                and a value of "bearer xxx".

                assert.Equal(t, "bearer xxx", r.Header.Get("Authorization"))
            */

            // We expect the http method is GET.
            assert.Equal(t, http.MethodGet, r.Method)

            /*
                This is where we mock the response of the the request. That is,
                what we expect the API Server to send back.
            */

            // return 200 OK and users info.
            w.WriteHeader(http.StatusOK)
            w.Write([]byte(`[
                "Jack",
                "Marry",
                "Sandy"
            ]`))
        })

        // Create an http server and register the router with a
        // predefined mock handler.
        fakeServer := httptest.NewServer(router)

        // We should always close the http server at the end of the test
        // to release related resources.
        defer fakeServer.Close()

        // The URL from the mock http server is in the format of http://127.0.0.1
        // and the port is random.
        baseUrl := fakeServer.URL

        // Calling a function to be tested.
        users, err := GetUsers(baseUrl)

        // Test the results of the function as we expect.
        assert.NoError(t, err)
        assert.Len(t, users, 3)
        assert.Equal(t, []string{"Jack", "Marry", "Sandy"}, users)
    })

    t.Run("unhappy path, API server has some problems", func(t *testing.T) {
        // Create a router that routes http requests to specific handlers.
        router := http.NewServeMux()

        // We expect to have the mock http server process /api/v1/users
        // while faking its response as we expect it to look.
        router.HandleFunc("/api/v1/users", func(w http.ResponseWriter, r *http.Request) {
            /*
                This is where we can test the format of our request!
                Let's say assert that the request was made with the correct header.

                For example, check if the request comes with an authorization header
                and a value of "bearer xxx".

                assert.Equal(t, "bearer xxx", r.Header.Get("Authorization"))
            */

            // We expect the http method is GET.
            assert.Equal(t, http.MethodGet, r.Method)

            /*
                This is where we mock the response of the the request. That is,
                what we expect the API server to send back.
            */

            // return 500 Internal Server Error.
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte(`{
                "msg": "get error"
            }`))
        })

        // Create an http server and register the router with a
        // predefined mock handler.
        fakeServer := httptest.NewServer(router)

        // We should always close the http server at the end of the test
        // to release related resources.
        defer fakeServer.Close()

        // The URL from the mock http server is in the format of http://127.0.0.1
        // and the port is random.
        baseUrl := fakeServer.URL

        // Calling a function to be tested.
        _, err := GetUsers(baseUrl)

        // Test the results of the function as we expect.
        assert.Error(t, err)
        assert.EqualError(t, err, "get error")
    })
}

創建使用者

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
type CreateUserRequest struct {
    Name string `json:"name"`
}

type CreateUserResponse struct {
    ID   string `json:"id"`
    Name string `json:"name"`
}

func CreateUser(baseUrl, userName string) (*CreateUserResponse, error) {
    url := baseUrl + "/api/v1/user"

    // Create a payload that should be POSTed to the server.
    payload := CreateUserRequest{
        Name: userName,
    }

    // Encode the payload into json format.
    var buf bytes.Buffer
    err := json.NewEncoder(&buf).Encode(payload)
    if err != nil {
        return nil, err
    }

    // Create a new http POST request with the payload
    // and modify the Content-Type header.
    req, err := http.NewRequest(http.MethodPost, url, &buf)
    if err != nil {
        return nil, err
    }
    req.Header.Add("Content-Type", "application/json")

    // Send the http request to the server.
    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return nil, err
    }

    // Reading and parsing the response body.
    body, err := io.ReadAll(resp.Body)
    if err != nil {
        return nil, err
    }

    if resp.StatusCode == http.StatusCreated {
        // If the request is successful,
        // return the user information.
        var data CreateUserResponse
        err = json.Unmarshal(body, &data)
        if err != nil {
            return nil, err
        }
        return &data, nil
    } else {
        // If it fails, return the "msg" in the
        // response body.
        var data errorResponse
        err = json.Unmarshal(body, &data)
        if err != nil {
            return nil, err
        }
        return nil, errors.New(data.Msg)
    }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
func TestCreateUser(t *testing.T) {
    t.Run("happy path, API server has some problems", func(t *testing.T) {
        // Create a router that routes http requests to specific handlers.
        router := http.NewServeMux()

        // We expect to have the mock http server process /api/v1/user
        // while faking its response as we expect it to look.
        router.HandleFunc("/api/v1/user", func(w http.ResponseWriter, r *http.Request) {
            /*
                This is where we can test the format of our request!
                Let's say assert that the request was made with the correct header.

                For example, check if the request comes with an authorization header
                and a value of "bearer xxx".

                assert.Equal(t, "bearer xxx", r.Header.Get("Authorization"))
            */

            // We expect the http method is POST.
            assert.Equal(t, http.MethodPost, r.Method)

            // Check if the Content-Type header is application/json.
            assert.Equal(t, "application/json", r.Header.Get("Content-Type"))

            // Check the payload format of the request.
            body, err := io.ReadAll(r.Body)
            assert.NoError(t, err)
            assert.JSONEq(t, `{"name": "Jack"}`, string(body))

            /*
                This is where we mock the response of the the request. That is,
                what we expect the API Server to send back.
            */

            // return 201 Created and user info.
            w.WriteHeader(http.StatusCreated)
            w.Write([]byte(`{
                "id": "id_foo",
                "name": "name_foo"
            }`))
        })

        // Create an http server and register the router with a
        // predefined mock handler.
        fakeServer := httptest.NewServer(router)

        // We should always close the http server at the end of the test
        // to release related resources.
        defer fakeServer.Close()

        // The URL from the mock http server is in the format of http://127.0.0.1
        // and the port is random.
        baseUrl := fakeServer.URL

        // Calling a function to be tested.
        user, err := CreateUser(baseUrl, "Jack")

        // Test the results of the function as we expect.
        assert.NoError(t, err)
        assert.Equal(t, "id_foo", user.ID)
        assert.Equal(t, "name_foo", user.Name)
    })
    t.Run("unhappy path, some error occur", func(t *testing.T) {
        // Create a router that routes http requests to specific handlers.
        router := http.NewServeMux()

        // We expect to have the mock http server process /api/v1/user
        // while faking its response as we expect it to look.
        router.HandleFunc("/api/v1/user", func(w http.ResponseWriter, r *http.Request) {
            /*
                This is where we can test the format of our request!
                Let's say assert that the request was made with the correct header.

                For example, check if the request comes with an authorization header
                and a value of "bearer xxx".

                assert.Equal(t, "bearer xxx", r.Header.Get("Authorization"))
            */

            // We expect the http method is POST.
            assert.Equal(t, http.MethodPost, r.Method)

            // Check if the Content-Type header is application/json.
            assert.Equal(t, "application/json", r.Header.Get("Content-Type"))

            // Check the payload format of the request.
            body, err := io.ReadAll(r.Body)
            assert.NoError(t, err)
            assert.JSONEq(t, `{"name": "Jack"}`, string(body))

            /*
                This is where we mock the response of the the request. That is,
                what we expect the API Server to send back.
            */

            // return 500 Internal Server Error.
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte(`{
                "msg": "get error"
            }`))
        })

        // Create an http server and register the router with a
        // predefined mock handler.
        fakeServer := httptest.NewServer(router)

        // We should always close the http server at the end of the test
        // to release related resources.
        defer fakeServer.Close()

        // The URL from the mock http server is in the format of http://127.0.0.1
        // and the port is random.
        baseUrl := fakeServer.URL

        // Calling a function to be tested.
        _, err := CreateUser(baseUrl, "Jack")

        // Test the results of the function as we expect.
        assert.Error(t, err)
        assert.EqualError(t, err, "get error")
    })
}

這篇文章記載如何有效的測試 HTTP Client，但這個方法只限用於透過 TCP 傳輸的 HTTP Client，若想知道如何測試 Unix Domain Socket based HTTP Client，可以參考 [Golang] Unit Testing of HTTP Client using Unix Domain Socket！

很高興能在這裡幫助到您，歡迎登入 Liker 為我鼓掌 5 次，或者成為我的讚賞公民，鼓勵我繼續創造優質文章。
相關文章
以最優質的內容回應您的鼓勵