Pricing of Google App Engine Flexible env, a $500 lesson

Question

I followed the Nodejs on App Engine Flexible env tutorial @: https://cloud.google.com/nodejs/getting-started/hello-world

Having successfully deployed and tested the tutorial, I changed the code to experiment a little and successfully deployed it... and then left it running since this was a testing environment (not public).

A month later, I receive a bill from Google for over $370!

In the transaction details I see the following:

Oct 1 – 31, 2017 App Engine Flex Instance RAM: 5948.774 Gibibyte-hours ([MYPROJECT]) $42.24

Oct 1 – 31, 2017 App Engine Flex Instance Core Hours: 5948.774 Hours ([MYPROJECT]) $312.91

How did this testing environment with almost 0 requests require about 6,000 hours of resources? In the worst, I would have assume 720 hrs running fulltime for a month @ $0.05 per hour would cost me ~$40. https://cloud.google.com/appengine/pricing

Can someone help shed light on this? I have not been able to find out why so many resources were needed?

Thanks for the help!

For more data, this is the traffic over the last month (basically 0):

And instance data

UPDATE: Note that I did bring one modification to the package.json: I added nodemon as a dependency and added it as part of my "nmp start" script. Though I doubt this explains the 6000 hours of resources:

  "scripts": {
    "deploy": "gcloud app deploy",
    "start": "nodemon app.js",
    "dev": "nodemon app js",
    "lint": "samples lint",
    "pretest": "npm run lint",
    "system-test": "samples test app",
    "test": "npm run system-test",
    "e2e-test": "samples test deploy"
  },

App.yaml (default-no change from tutorial)

runtime: nodejs
env: flex

Answer 1

After multiple back and forth with Google, and hours of reading blogs and looking at reports, I've finally (somewhat) found an explanation for what happened. I will post it here with my suggestions so that other people do not also fall victim to this problem.

Note, this may seem obvious to some, but as a new GAE user, all of this was brand new to me.

In short, when deploying to GAE and using the following command "$ gcloud app deploy", it creates a new version and sets it as the default, but also and more importantly, it does NOT remove the previous version that was deployed.

More info about versions and instances can be found here: https://cloud.google.com/appengine/docs/standard/python/an-overview-of-app-engine

So in my case, without knowing it, I had created multiple versions of my simple node app. These versions are still running in case one needs to switch following an error. But these versions also require instances, and the default, unless stated in the app.yaml, is 2 instances.

Google says:

App Engine by default scales the number of instances running up and down to match the load, thus providing consistent performance for your app at all times while minimizing idle instances and thus reducing cost.

However, from my experience, this was not the case. As I said earlier, I pushed my node app with nodemon which it seems was causing errors.

In the end, following the tutorial and not shutting down the project, I had 4 versions, each with 2 instances running full-time for 1.5 months serving 0 requests and generating lots of error messages and it cost me $500.

RECOMMENDATIONS IF YOU STILL WANT TO USE GAE FLEX ENV:

1. First and foremost, setup a billing budget & alerts so that you do not get surprised by an expensive invoice that is automatically charged to your CC: https://cloud.google.com/billing/docs/how-to/budgets

2. In a testing env, you most likely do not need multiple versions, so while deploying use the following command:
   $ gcloud app deploy --version v1

3. Update your app.yaml to force only 1 instance with minimal resources:

runtime: nodejs
env: flex

# This sample incurs costs to run on the App Engine flexible environment.
# The settings below are to reduce costs during testing and are not appropriate
# for production use. For more information, see:
# https://cloud.google.com/appengine/docs/flexible/nodejs/configuring-your-app-with-app-yaml
manual_scaling:
  instances: 1
resources:
  cpu: 1
  memory_gb: 0.5
  disk_size_gb: 10

4. Set daily spending limit

See this blog post for more info: https://medium.com/google-cloud/three-simple-steps-to-save-costs-when-prototyping-with-app-engine-flexible-environment-104fc6736495

I wish some of these steps had been included in the tutorial in order to protect those who are trying to learn and experiment, but it was not.

Google App Engine Flex env can be tricky if one does not know all these details. A friend pointed me to Heroku, that has both set pricing and Free/Hobby offers. I was able to quickly push a new node app there, and it worked like charm! https://www.heroku.com/pricing

It "only" cost me $500 to learn this lesson, but I do hope this helps others looking at Google App Engine Flex Env.