How can I keep Google Cloud Functions warm?

Question

I know this may miss the point of using Cloud Functions in the first place, but in my specific case, I'm using Cloud Functions because it's the only way I can bridge Next.js with Firebase Hosting. I don't need to make it cost efficient, etc.

With that said, the cold boot times for Cloud Functions are simply unbearable and not production-ready, averaging around 10 to 15 seconds for my boilerplate.

I've watched this video by Google (https://www.youtube.com/watch?v=IOXrwFqR6kY) that talks about how to reduce cold boot time. In a nutshell: 1) Trim dependencies, 2) Trial & error for dependencies' versions for cache on Google's network, 3) Lazy loading.

But 1) there are only so many dependencies I can trim. 2) How would I know which version is more cached? 3) There are only so many dependencies I can lazy load.

Another way is to avoid the cold boot all together. What's a good way or hack that I can essentially keep my (one and only) cloud function warm?

Answer 1

With all "serverless" compute providers, there is always going to be some form of cold start cost that you can't eliminate. Even if you are able to keep a single instance alive by pinging it, the system may spin up any number of other instances to handle current load. Those new instances will have a cold start cost. Then, when load decreases, the unnecessary instances will be shut down.

There are ways to minimize your cold start costs, as you have discovered, but the costs can't be eliminated.

As of Sept 2021, you can now specify a minimum number of instances to keep active. This can help reduce (but not eliminate) cold starts. Read the Google Cloud blog and the documentation. For Firebase, read its documentation. Note that setting min instances incurs extra billing - keeping computing resources active is not a free service.

If you absolutely demand hot servers to handle requests 24/7, then you need to manage your own servers that run 24/7 (and pay the cost of those servers running 24/7). As you can see, the benefit of serverless is that you don't manage or scale your own servers, and you only pay for what you use, but you have unpredictable cold start costs associated with your project. That's the tradeoff.

Answer 2

You can now specify MIN_INSTANCE_LIMIT to keep instances running at all times.

Cloud Functions Doc: https://cloud.google.com/functions/docs/configuring/min-instances

Cloud Functions example from the docs:

gcloud beta functions deploy myFunction --min-instances 5

It's also available in Firebase Functions by specifying minInstances:

Firebase Functions Docs: https://firebase.google.com/docs/functions/manage-functions#min-max-instances

Frank announcing it on Twitter: https://twitter.com/puf/status/1433431768963633152

Firebase Function example from the docs:

exports.getAutocompleteResponse = functions
    .runWith({
      // Keep 5 instances warm for this latency-critical function
      minInstances: 5,
    })
    .https.onCall((data, context) => {
      // Autocomplete a user's search term
    });

Answer 3

You can trigger it via cron job as explained here: https://cloud.google.com/scheduler/docs/creating

Answer 4

Using Google Scheduler is a wise solution but the actual implementation is not so straightforward. Please check my article for details. Examples of functions:

myHttpFunction: functions.https.onRequest((request, response) => {
  // Check if available warmup parameter.                                   
  // Use  request.query.warmup parameter if warmup request is GET.                                   
  // Use request.body.warmup parameter if warmup request is POST.                                   
  if (request.query.warmup || request.body.warmup) {
    return response.status(200).type('application/json').send({status: "success", message: "OK"});
  }
});
myOnCallFunction: functions.https.onCall((data, context) => {
  // Check if available warmup parameter.
  if (data.warmup) {
    return {"success": true};
  }
});

Examples of gcloud cli comands:

gcloud --project="my-awesome-project" scheduler jobs create http  warmupMyOnCallFuntion --time-zone "America/Los_Angeles" --schedule="*/5 5-23 * * *" --uri="https://us-central1-my-awesome-project.cloudfunctions.net/myOnCallFuntion" --description="my warmup job" --headers="Content-Type=application/json" --http-method="POST" --message-body="{\"data\":{\"warmup\":\"true\"}}"

gcloud --project="my-awesome-project" scheduler jobs create http  warmupMyHttpFuntion --time-zone "America/Los_Angeles" --schedule="*/5 5-23 * * *" --uri="https://us-central1-my-awesome-project.cloudfunctions.net/myHttpFuntion?warmup=true" --description="my warmup job" --headers="Content-Type=application/json" --http-method="GET"

Answer 5

Google has just announced the ability to set min-instances for your Cloud Function deploys. This allows you to set the lower bound for scaling your functions down and minimises cold starts (they don't promise to eliminate them).

There is a small cost for keeping warm instances around (Idle time) - though at the time of writing, that seems undocumented on the Cloud Functions pricing page. They say:

If you set a minimum number of function instances, you are also billed for the time these instances are not active. This is called idle time and is priced at a different rate.

Answer 6

Cloud functions are generally best-suited to perform just one (small) task. More often than not I come across people who want to do everything inside one cloud function. To be honest, this is also how I started developing cloud functions.

With this in mind, you should keep your cloud function code clean and small to perform just one task. Normally this would be a background task, a file or record that needs to be written somewhere, or a check that has to be performed. In this scenario, it doesn't really matter if there is a cold start penalty.

But nowadays, people, including myself, rely on cloud functions as a backend for API Gateway or Cloud Endpoints. In this scenario, the user goes to a website and the website sends a backend request to the cloud function to get some additional information. Now the cloud function acts as an API and a user is waiting for it.

Typical cold cloud function:

Typical warm cloud function:

There are several ways to cope with a cold-start problem:

• Reduce dependencies and amount of code. As I said before, cloud functions are best-suited for performing single tasks. This will reduce the overall package size that has to be loaded to a server between receiving a request and executing the code, thus speeding things up significantly.
• Another more hacky way is to schedule a cloud scheduler to periodically send a warmup request to your cloud function. GCP has a generous free tier, which allows for 3 schedulers and 2 million cloud function invocations (depending on resource usage). So, depending on the number of cloud functions, you could easily schedule an http-request every few seconds. For the sake of clarity, I placed a snippet below this post that deploys a cloud function and a scheduler that sends warmup requests.

If you think you have tweaked the cold-start problem, you could also take measures to speed up the actual runtime:

• I switched from Python to Golang, which gave me a double-digit performance increase in terms of actual runtime. Golang speed is comparable to Java or C++.
• Declare variables, especially GCP clients like storage, pub/sub etc., on a global level (source). This way, future invocations of your cloud function will reuse that objects.
• If you are performing multiple independent actions within a cloud function, you could make them asynchronous.
• And again, clean code and fewer dependencies also improve the runtime.

Snippet:

# Deploy function
gcloud functions deploy warm-function \
  --runtime=go113 \
  --entry-point=Function \
  --trigger-http \
  --project=${PROJECT_ID} \
  --region=europe-west1 \
  --timeout=5s \
  --memory=128MB

# Set IAM bindings
gcloud functions add-iam-policy-binding warm-function \
  --region=europe-west1 \
  --member=serviceAccount:${PROJECT_ID}@appspot.gserviceaccount.com \
  --role=roles/cloudfunctions.invoker

# Create scheduler
gcloud scheduler jobs create http warmup-job \
  --schedule='*/5 * * * *' \
  --uri='https://europe-west1-${PROJECT_ID}.cloudfunctions.net/warm-function' \
  --project=${PROJECT_ID} \
  --http-method=OPTIONS \
  --oidc-service-account-email=${PROJECT_ID}@appspot.gserviceaccount.com \
  --oidc-token-audience=https://europe-west1-${PROJECT_ID}.cloudfunctions.net/warm-function

Answer 7

In order to keep the cold start to minimum there is not a single solution, it is a mixture of multiple techniques. The question is more how to make our lambdas so fast we don't care so much about cold starts - I am talking about a startup time in the range of 100-500 ms.

How to make your lambda faster ?

1. Keep your package size as small a possible (remove all big libraries where a fraction of it is used) - keep package size to max 20 MB. On every cold start this package is fetched and decompressed.
2. Try initialise on start of your application only the pieces you want. Nodejs - https://gist.github.com/Rich-Harris/41e8ccc755ea232a5e7b88dee118bcf5
3. If you use a JVM technology for your services, try to migrate them to Graalvm where the boot-up overhead is reduced to minimum.
   • micronaut + graalvm
   • quarkus + graalvm
   • helidon + graalvm
4. Use cloud infrastructure configs to reduce the cold starts.

In 2020 cold start are not such pain as few years ago. I would say more about AWS, but I am sure all above works well for any cloud provider.

In end of 2019 AWS intorduced Lambda concurrency provisioning -https://aws.amazon.com/about-aws/whats-new/2019/12/aws-lambda-announces-provisioned-concurrency/, you don't have to care so much about warming anymore.