こんにちは。KOUKIです。
とあるWeb系企業でGoエンジニアをしています。
今回は、Golang、MongoDB、Redisの開発環境をDocker上で構築し、キャッシュ有無によるA/Bテスト(パフォーマンステスト)のやり方を記事にしました。
<目次>
事前準備
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
# フォルダ/ファイル mkdir go-app cd go-app touch main.go touch .air.toml touch Dockerfile touch docker-compose.yml mkdir handler touch handler.go mkdir models touch modelsrecipe.go touch recipes.json mkdir redis/redis.conf touch apache-benchmark.p # モジュール brew install gnuplot go mod init go-app go get go.mongodb.org/mongo-driver go get github.com/go-redis/redis go get github.com/gin-gonic/gin |
アプリケーション
GoのWebフレームワークの一つであるginを使って、簡単なRest APIを実装します。
Dockerfile
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
# Dockerfile-golang FROM golang:1.16 WORKDIR /app COPY go.* . # airの設定ファイルをコピー COPY *air.toml . RUN go mod download COPY . . RUN curl -sSfL https://raw.githubusercontent.com/cosmtrek/air/master/install.sh | sh -s -- -b $(go env GOPATH)/bin CMD ["air"] |
docker-compose.yml
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
version: "3.1" services: backend: build: context: . dockerfile: ./Dockerfile ports: - 8080:8080 volumes: - .:/app environment: - MONGO_URI=mongodb://admin:password@mongo:27017/test?authSource=admin - MONGO_DATABASE=demo depends_on: - mongo mongo: image: mongo:4.4.3 environment: MONGO_INITDB_ROOT_USERNAME: admin MONGO_INITDB_ROOT_PASSWORD: password ports: - 27017:27017 volumes: - ./db:/data/db - ./configdb:/data/configdb # mongodb://admin:password@localhost:27017/test redis: image: redis:latest ports: - 6379:6379 volumes: - ./redis/conf:/usr/local/etc/redis redis-gui: image: redislabs/redisinsight:latest ports: - 8001:8001 links: - redis |
main.go
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
package main import ( "context" "fmt" "log" "os" "github.com/gin-gonic/gin" "github.com/go-redis/redis" "github.com/hoge/gin-sample/recipes-api/handlers" "go.mongodb.org/mongo-driver/mongo" "go.mongodb.org/mongo-driver/mongo/options" "go.mongodb.org/mongo-driver/mongo/readpref" ) var recipesHandler *handlers.RecipesHandler func init() { ctx := context.Background() log.Println(os.Getenv("MONGO_URI")) client, err := mongo.Connect(ctx, options.Client().ApplyURI(os.Getenv("MONGO_URI"))) if err = client.Ping(context.TODO(), readpref.Primary()); err != nil { log.Fatal(err) } log.Println("Connection to MongoDB") collection := client.Database(os.Getenv("MONGO_DATABASE")).Collection("recipes") redisClient := redis.NewClient(&redis.Options{ Addr: "redis:6379", Password: "", DB: 0, }) status := redisClient.Ping() fmt.Println("Connection To Redis: ", status) recipesHandler = handlers.NewRecipeHandler(ctx, collection, redisClient) } func main() { router := gin.Default() router.GET("/recipes", recipesHandler.ListRecipesHandler) router.Run() } |
handler.go
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
package handlers import ( "context" "encoding/json" "fmt" "log" "net/http" "time" "github.com/gin-gonic/gin" "github.com/go-redis/redis" "github.com/hoge/gin-sample/recipes-api/models" "go.mongodb.org/mongo-driver/bson" "go.mongodb.org/mongo-driver/bson/primitive" "go.mongodb.org/mongo-driver/mongo" ) type RecipesHandler struct { collection *mongo.Collection ctx context.Context redisClient *redis.Client } func NewRecipeHandler(ctx context.Context, collection *mongo.Collection, redisClient *redis.Client) *RecipesHandler { return &RecipesHandler{ collection: collection, ctx: ctx, redisClient: redisClient, } } func (handler *RecipesHandler) ListRecipesHandler(c *gin.Context) { val, err := handler.redisClient.Get("recipes").Result() if err == redis.Nil { log.Printf("Request to MongoDB") cur, err := handler.collection.Find(handler.ctx, bson.M{}) if err != nil { c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()}) return } defer cur.Close(handler.ctx) recipes := make([]models.Recipe, 0) for cur.Next(handler.ctx) { var recipe models.Recipe cur.Decode(&recipe) recipes = append(recipes, recipe) } data, _ := json.Marshal(recipes) handler.redisClient.Set("recipes", string(data), 0) c.JSON(http.StatusOK, recipes) } else if err != nil { c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()}) } else { log.Printf("Request to Redis") recipes := make([]models.Recipe, 0) json.Unmarshal([]byte(val), &recipes) c.JSON(http.StatusOK, recipes) } } |
ListRecipesHandlerメソッドでは、Redisにデータがあればそれを使い(キャッシュ)、存在しなければMongoDBからデータを取得する処理を実装しています。
recipe.go
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
package models import ( "time" "go.mongodb.org/mongo-driver/bson/primitive" ) // swagger:parameters recipes newRecipe type Recipe struct { //swagger:ignore ID primitive.ObjectID `json:"id" bson:"_id"` Name string `json:"name" bson:"name"` Tags []string `json:"tags" bson:"tags"` Ingredients []string `json:"ingredients bson:"ingredients""` Instructions []string `json:"instructions" bson:"instructions"` PublishedAt time.Time `json:"publishedAt" bson:"publishedAt"` } |
ホットリロード
Go製のairを使うとソースコードの変更を検知してビルドを自動で行ってくれるので、導入しておきましょう。
.air.tomlに設定情報を記述します。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
# .air.toml # Config file for [Air](https://github.com/cosmtrek/air) in TOML format # Working directory # . or absolute path, please note that the directories following must be under root. root = "." tmp_dir = "tmp" [build] # Just plain old shell command. You could use `make` as well. cmd = "go build -o ./tmp/main ." # Binary file yields from `cmd`. bin = "tmp/main" # Customize binary. full_bin = "APP_ENV=dev APP_USER=air ./tmp/main" # Watch these filename extensions. include_ext = ["go", "tpl", "tmpl", "html"] # Ignore these filename extensions or directories. exclude_dir = ["assets", "tmp", "vendor"] # Watch these directories if you specified. include_dir = [] # Exclude files. exclude_file = [] # This log file places in your tmp_dir. log = "air.log" # It's not necessary to trigger build each time file changes if it's too frequent. delay = 1000 # ms # Stop running old binary when build errors occur. stop_on_error = true # Send Interrupt signal before killing process (windows does not support this feature) send_interrupt = false # Delay after sending Interrupt signal kill_delay = 500 # ms [log] # Show log time time = false [color] # Customize each part's color. If no color found, use the raw app log. main = "magenta" watcher = "cyan" build = "yellow" runner = "green" [misc] # Delete tmp directory on exit clean_on_exit = true |
redis.conf
redisの初期設定を変更します。
|
1 2 |
maxmemory-policy allkeys-lru maxmemory 512mb |
apache-benchmmark.p
パフォーマンス計測で使用する設定ファイルです。
|
1 2 3 4 5 6 7 8 |
set terminal png set output "benchmark.png" set title "Cache benchmark" set size 1,0.7 set grid y set xlabel "request" set ylabel "response time (ms)" plot "with-cache.data" using 9 smooth sbezier with lines title "with cache", "without-cache.data" using 9 smooth sbezier with lines title "without cache" |
recipes.json
データの一部を記載します。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
[ { "id": "c0283p3d0cvuglq85log", "name": "Oregano Marinated Chicken", "tags": [ "main", "chicken" ], "ingredients": [ "4 (6 to 7-ounce) boneless skinless chicken breasts\r", "10 grinds black pepper\r", "1/2 tsp salt\r", "2 tablespoon extra-virgin olive oil\r", "1 teaspoon dried oregano\r", "1 lemon, juiced" ], "instructions": [ "To marinate the chicken: In a non-reactive dish, combine the lemon juice, olive oil, oregano, salt, and pepper and mix together", " Add the chicken breasts to the dish and rub both sides in the mixture", " Cover the dish with plastic wrap and let marinate in the refrigerator for at least 30 minutes and up to 4 hours", "\r\n\r\nTo cook the chicken: Heat a nonstick skillet or grill pan over high heat", " Add the chicken breasts and cook, turning once, until well browned, about 4 to 5 minutes on each side or until cooked through", " Let the chicken rest on a cutting board for a few minutes before slicing it into thin strips" ], "publishedAt": "2021-01-17T19:28:52.803062+01:00" }, { ... } } |
起動確認
準備が整ったので、下記のコマンドで起動を確認します。
|
1 2 3 |
docker-compose up backend_1 | 2021/08/29 10:41:06 Connection to MongoDB backend_1 | Connection To Redis: ping: PONG |
「Connection to MongoDB」、「Connection To Redis: ping: PONG」がコンソール上に表示されたら接続成功です。
計測
abコマンド
計測には、Apacheのabコマンドを利用します。2000リクエストを送信してみましょう。
|
1 2 3 4 5 6 7 8 9 10 11 12 |
ab -n 2000 -c 100 -g without-cache.data http://localhost:8080/recipes Time taken for tests: 148.191 seconds Time per request: 7409.556 [ms] (mean) Time per request: 74.096 [ms] (mean, across all concurrent requests) # キャッシュあり ab -n 2000 -c 100 -g with-cache.data http://localhost:8080/recipes Time taken for tests: 134.045 seconds Time per request: 6702.274 [ms] (mean) Time per request: 67.023 [ms] (mean, across all concurrent requests) |
「Time taken for tests」は、2,000リクエストが完了するまでの時間を表しています。
一方、「Time per request」は、1リクエストが完了するまでのミリセコンドを表しています。
キャッシュ有無で比較するとパフォーマンスがだいぶ違うことがわかりますね。
grunplot
grunplotを使うとパフォーマンス結果を可視化することが可能です。
|
1 2 3 |
gnuplot apache-benchmark. Unable to revert mtime: /Library/Fonts |
「Unable to revert mtime: /Library/Fonts」警告が出てきますが、「benchmark.png」ファイルがローカルに生成されます。
便利ですね^^
まとめ
abコマンドで手軽にAPIリクエストテストができるし、それを可視化することも可能なのでとても便利です。
これからガンガンテストをして、パフォーマンスを意識した実装ができるエンジニアになりたいと思います^^
それでは、また!
にほんブログ村
コメントを残す
コメントを投稿するにはログインしてください。