Setting up a Recurring Google Cloud Function With Terraform

I recently needed to run a simple Python script every 10 mins to alert me if a website had been updated, rather than going the traditional route of spinning up a VM with a cronjob running on it I wanted to investigate using a serverless approach. In this guide I'll show you have to setup a Google Cloud Function and Cloud Scheduler using Terraform.

Cloud Functions are serverless functions (written in Python, Node.js, or Go) that can be triggered by HTTP requests and Cloud Scheduler is a managed cron scheduler using the familiar crontab syntax. By combining both of these tools we can have a script that runs every 10 minutes without having to run any servers.

We'll use Terraform to manage the application infrastructure so it is reproducible and versioned. Infrastructure as code FTW!

Prerequisites

I'm going to assume that you have a Google Cloud account setup and are familiar with using Terraform, and have it installed locally. You'll also need to create a service account for Terraform to create the Google Cloud resources, if you don't already have one setup (Getting started with Terraform on Google Cloud Platform).

Project Setup

Create a project folder locally with the following layout:

•
├── generated/
├── src/
│   ├── main.py
│   └── requirements.txt
└── terraform/
    └── main.tf

The main.py file will contain your Python application, I'll leave that up to you. Let's focus on what needs to be added to the main.tf Terraform file.

Uploading your Code

First off we need to create a Cloud Storage bucket and upload a zipped copy of your code to it. The Cloud Function will unzip this code and execute it.

resource "google_storage_bucket" "bucket" {
  name = "cloud-function-tutorial-bucket" # This bucket name must be unique
}

data "archive_file" "src" {
  type        = "zip"
  source_dir  = "${path.root}/../src" # Directory where your Python source code is
  output_path = "${path.root}/../generated/src.zip"
}

resource "google_storage_bucket_object" "archive" {
  name   = "${data.archive_file.src.output_md5}.zip"
  bucket = google_storage_bucket.bucket.name
  source = "${path.root}/../generated/src.zip"
}

Notice on line 12 we use the MD5 hash of the zipped file as the filename, this is a workaround for a limitation in the Terraform Cloud Function implementation where it won't pick up changes in the source code unless the file name has also change.

Create the Cloud Function

Once we have the config for the code we can create the Cloud Function resource and reference the cdde in it.

resource "google_cloudfunctions_function" "function" {
  name        = "scheduled-cloud-function-tutorial"
  description = "An example Cloud Function that is triggered by a Cloud Schedule."
  runtime     = "python37"

  environment_variables = {
    FOO = "bar",
  }

  available_memory_mb   = 128
  source_archive_bucket = google_storage_bucket.bucket.name
  source_archive_object = google_storage_bucket_object.archive.name
  trigger_http          = true
  entry_point           = "http_handler" # This is the name of the function that will be executed in your Python code
}

The Cloud Function is setup so it can be triggered from a HTTP endpoint, this is what the scheduler will use to execute the function.

By default the Cloud Function can't be invoked by anyone, let's create a service account and give it the cloudfunctions.invoker role:

resource "google_service_account" "service_account" {
  account_id   = "cloud-function-invoker"
  display_name = "Cloud Function Tutorial Invoker Service Account"
}

resource "google_cloudfunctions_function_iam_member" "invoker" {
  project        = google_cloudfunctions_function.function.project
  region         = google_cloudfunctions_function.function.region
  cloud_function = google_cloudfunctions_function.function.name

  role   = "roles/cloudfunctions.invoker"
  member = "serviceAccount:${google_service_account.service_account.email}"
}

Setup the Scheduler

The final piece is to setup the scheduler job. The scheduler is going to use a http_target which will reference the cloud functions https_trigger_url and make a GET request to it every 10 mins.

resource "google_cloud_scheduler_job" "job" {
  name             = "cloud-function-tutorial-scheduler"
  description      = "Trigger the ${google_cloudfunctions_function.function.name} Cloud Function every 10 mins."
  schedule         = "*/10 * * * *" # Every 10 mins
  time_zone        = "Europe/Dublin"
  attempt_deadline = "320s"

  http_target {
    http_method = "GET"
    uri         = google_cloudfunctions_function.function.https_trigger_url

    oidc_token {
      service_account_email = google_service_account.service_account.email
    }
  }
}

We've used the OIDC token for authentication so that the Cloud Scheduler invokes the cloud function using the authorized service account we created earlier.

Running Terraform

From the terraform folder we can now initialise and apply the terraform changes:

cd teraform/
terraform init
terrafrom apply

Depending on how your Google Cloud project is setup you might have errors initially asking you to enable certain Google Cloud APIs. Follow the on screen instructions to resolve these issues and try terraform apply again.

Conclusion

And that's it. You now have a Cloud Function that will get called every 10 mins. From the Google Cloud console you can click into the cloud function and see the execution logs to make sure it's all running as planned.

There is a companion Github repo which contains a complete working example for reference.