web-dev-qa-db-ja.com

Google Cloud Tasks APIのレート制限を制御できません

Google Cloudタスクのレート制限を1秒あたりの処理タスク数1に制限しようとしています。

私は自分のキューを作成しました：

gcloud tasks queues create my-queue \
          --max-dispatches-per-second=1 \
          --max-concurrent-dispatches=1 \
          --max-attempts=2 \
          --min-backoff=60s

それを説明すると私に与えられます：

name: projects/my-project/locations/us-central1/queues/my-queue
rateLimits:
  maxBurstSize: 10
  maxConcurrentDispatches: 1
  maxDispatchesPerSecond: 1.0
retryConfig:
  maxAttempts: 2
  maxBackoff: 3600s
  maxDoublings: 16
  minBackoff: 60s
state: RUNNING

一連のタスクを作成した後、1秒の時間内にタスクの多くが望ましくないほど処理されていることをログで確認できます。

2019-07-27 02:37:48 default[20190727t043306]  Received task with payload: {'id': 51}
2019-07-27 02:37:48 default[20190727t043306]  "POST /my_handler HTTP/1.1" 200
2019-07-27 02:37:49 default[20190727t043306]  Received task with payload: {'id': 52}
2019-07-27 02:37:49 default[20190727t043306]  "POST /my_handler HTTP/1.1" 200
2019-07-27 02:37:49 default[20190727t043306]  Received task with payload: {'id': 53}
2019-07-27 02:37:49 default[20190727t043306]  "POST /my_handler HTTP/1.1" 200
2019-07-27 02:37:49 default[20190727t043306]  Received task with payload: {'id': 54}
2019-07-27 02:37:49 default[20190727t043306]  "POST /my_handler HTTP/1.1" 200
2019-07-27 02:37:49 default[20190727t043306]  Received task with payload: {'id': 55}
2019-07-27 02:37:49 default[20190727t043306]  "POST /my_handler HTTP/1.1" 200
2019-07-27 02:37:49 default[20190727t043306]  Received task with payload: {'id': 56}
2019-07-27 02:37:49 default[20190727t043306]  "POST /my_handler HTTP/1.1" 200
2019-07-27 02:37:49 default[20190727t043306]  Received task with payload: {'id': 57}
2019-07-27 02:37:49 default[20190727t043306]  "POST /my_handler HTTP/1.1" 200
2019-07-27 02:37:49 default[20190727t043306]  Received task with payload: {'id': 58}

この1秒の時間間隔で1つだけのタスクを実行するように適切に強制するにはどうすればよいですか？

更新30/06：

基本的なセットアップで同じ問題をもう一度試しました。

セットアップとプロセスの詳細：

  1. ソースコード https://github.com/GoogleCloudPlatform/python-docs-samples/tree/master/appengine/flexible/tasks 、変更なし
  2. App.flexible.yamlではなくapp.yamlをデプロイする
  3. タスクを数回トリガーします：python create_app_engine_queue_task.py --project = $ PROJECT_ID --queue = $ QUEUE_ID --location = $ LOCATION_ID --payload = hello
  4. ログを確認：gcloud app logs read

今回は処理を開始するのにしばらく時間がかかりましたが、その後はほぼ同時にすべて処理されたようです。

完全なログ：

2019-07-30 00:22:37 default[20190730t021951]  [2019-07-30 00:22:37 +0000] [9] [INFO] Starting gunicorn 19.9.0
2019-07-30 00:22:37 default[20190730t021951]  [2019-07-30 00:22:37 +0000] [9] [INFO] Listening at: http://0.0.0.0:8081 (9)
2019-07-30 00:22:37 default[20190730t021951]  [2019-07-30 00:22:37 +0000] [9] [INFO] Using worker: threads
2019-07-30 00:22:37 default[20190730t021951]  [2019-07-30 00:22:37 +0000] [23] [INFO] Booting worker with pid: 23
2019-07-30 00:22:37 default[20190730t021951]  [2019-07-30 00:22:37 +0000] [26] [INFO] Booting worker with pid: 26
2019-07-30 00:27:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:27:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:27:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:27:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:27:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:27:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:41 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:41 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:42 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:42 default[20190730t021951]  Received task with payload: hello
2019-07-30 00:37:43 default[20190730t021951]  "POST /example_task_handler HTTP/1.1" 200
2019-07-30 00:37:43 default[20190730t021951]  Received task with payload: hello
google-cloud-tasks
12
Samuel Rizzo

短い答え：最大バーストサイズの構成が公開されていないため、現時点ではCloud Tasks APIでは不可能です。

回避策：App Engineのqueue.yamlを使用してタスクを管理します https://cloud.google.com/tasks/docs/queue-yaml

この機能にも興味がある場合は、投票して移植することができます https://issuetracker.google.com/issues/138813037

0
Samuel Rizzo

Queue.yamlを使用して作成されたキューと、Cloud Tasksによって作成されたキューは、どのように実行しても同じキューです。ただし、queue.yamlとCloud Tasksのキュー管理メソッドを併用すると問題が発生する可能性があります。詳細については https://cloud.google.com/tasks/docs/queue-yaml をご覧ください。

0
Ellen Evans