Tech TLDR;

Load Testing Google Cloud PubSub

February 26, 2021 by admin

I wrote the following tool to do load test on Google Cloud PubSub.

It’s a pretty simple but effective tool to generate a lot of data. This assumes user is using gcloud auth application-default login for credentials.

In this particular examples:

  • Every third request is adding a different attribute.
  • Python dict as a payload that get’s converted to JSON.
  • It runs 5 threads in parallel to write data fast

This was tested with 20K requests, and worked as expected.

from google.cloud import pubsub_v1 from uuid import uuid4 import json from multiprocessing.dummy import Pool TOPIC_PATH = 'YOUR TOPIC HERE' publisher = pubsub_v1.PublisherClient() print("Generating data") events = [] for i in range(20): data = { 'run': i, 'requestId': str(uuid4()) } events.append(data) def publishEvent(event): data = json.dumps(event).encode("utf-8") future = publisher.publish(TOPIC_PATH, data) print(future.result(), data) print("Starting to send") pool = Pool(5) # Number of concurrent threads asyncresponse = pool.map(publishEvent, events) pool.close() pool.join() print("Done")
Code language: PHP (php)

Hope it helps.

Filed Under: Software Engineering

Copyright © 2023 · eleven40 Pro Theme on Genesis Framework · WordPress · Log in