Cannot invoke Google Cloud Function from GCP Scheduler

Question

I've been trying to invoke a GCP function (--runtime nodejs8 --trigger-http) from GCP scheduler, both located within the same project. I can only make it work, if I grant unauthenticated access by adding the allUsers member to the functions permissions, with the Cloud Functions-Invoker role applied to it. However, when I only use the service account of the scheduler as the Cloud Functions-Invoker, I get a PERMISSION DENIED Error.

I created a hello world example, to show in detail, how my setup looks like.

1. I set up a service account:

gcloud iam service-accounts create scheduler --display-name="Task Schedule Runner"

2. Setting the role:

svc_policy.json:
{
    "bindings": [
      {
        "members": [
          "serviceAccount:[email protected]"
        ],
        "role": "roles/cloudscheduler.serviceAgent"
      }    
    ]
  }

gcloud iam service-accounts set-iam-policy [email protected] svc_policy.json  -q

3. Deploying the Cloud Function:

gcloud functions deploy helloworld --runtime nodejs8 --trigger-http --entry-point=helloWorld

4. Adding the service account as a member to the function:

gcloud functions add-iam-policy-binding helloworld --member serviceAccount:[email protected] --role roles/cloudfunctions.invoker

5. Creating the scheduler job:

gcloud beta scheduler jobs create http test-job --schedule "5 * * * *" --http-method=GET --uri=https://us-central1-mwsdata-1544225920485.cloudfunctions.net/helloworld --oidc-service-account-email=[e-mail] --oidc-token-audience=https://us-central1-mwsdata-1544225920485.cloudfunctions.net/helloworld

Log: PERMISSION DENIED

{
 httpRequest: {
 }
 insertId: "1ny5xuxf69w0ck"  
 jsonPayload: {
  @type: "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished"   
  jobName: "projects/mwsdata-1544225920485/locations/europe-west1/jobs/test-job"   
  status: "PERMISSION_DENIED"   
  targetType: "HTTP"   
  url: "https://us-central1-mwsdata-1544225920485.cloudfunctions.net/helloworld"   
 }
 logName: "projects/mwsdata-1544225920485/logs/cloudscheduler.googleapis.com%2Fexecutions"  
 receiveTimestamp: "2020-02-04T22:05:05.248707989Z"  
 resource: {
  labels: {
   job_id: "test-job"    
   location: "europe-west1"    
   project_id: "mwsdata-1544225920485"    
  }
  type: "cloud_scheduler_job"   
 }
 severity: "ERROR"  
 timestamp: "2020-02-04T22:05:05.248707989Z"  
}

Update

Here are the corresponding settings.

Scheduler Service Account

gcloud iam service-accounts get-iam-policy [email protected]

bindings:
- members:
  - serviceAccount:[email protected]
  role: roles/cloudscheduler.serviceAgent
etag: BwWdxuiGNv4=
version: 1

IAM Policy of the function:

gcloud functions get-iam-policy helloworld    
bindings:
- members:
  - serviceAccount:[email protected]
  role: roles/cloudfunctions.invoker
etag: BwWdxyDGOAY=
version: 1

Function Description

gcloud functions describe helloworld
availableMemoryMb: 256
entryPoint: helloWorld
httpsTrigger:
  url: https://us-central1-mwsdata-1544225920485.cloudfunctions.net/helloworld
ingressSettings: ALLOW_ALL
labels:
  deployment-tool: cli-gcloud
name: projects/mwsdata-1544225920485/locations/us-central1/functions/helloworld
runtime: nodejs8
serviceAccountEmail: [email protected]
sourceUploadUrl: https://storage.googleapis.com/gcf-upload-us-central1-671641e6-3f1b-41a1-9ac1-558224a1638a/b4a0e407-69b9-4f3d-a00d-7543ac33e013.zip?GoogleAccessId=[e-mail]&Expires=1580854835&Signature=S605ODVtOpnU4LIoRT2MnU4OQN3PqhpR0u2CjgcpRcZZUXstQ5kC%2F1rT6Lv2SusvUpBrCcU34Og2hK1QZ3dOPluzhq9cXEvg5MX1MMDyC5Y%2F7KGTibnV4ztFwrVMlZNTj5N%2FzTQn8a65T%2FwPBNUJWK0KrIUue3GemOQZ4l4fCf9v4a9h6MMjetLPCTLQ1BkyFUHrVnO312YDjSC3Ck7Le8OiXb7a%2BwXjTDtbawR20NZWfgCCVvL6iM9mDZSaVAYDzZ6l07eXHXPZfrEGgkn7vXN2ovMF%2BNGvwHvTx7pmur1yQaLM4vRRprjsnErU%2F3p4JO3tlbbFEf%2B69Wd9dyIKVA%3D%3D
status: ACTIVE
timeout: 60s
updateTime: '2020-02-04T21:51:15Z'
versionId: '1'

Scheduler Job Description

gcloud scheduler jobs describe test-job
attemptDeadline: 180s
httpTarget:
  headers:
    User-Agent: Google-Cloud-Scheduler
  httpMethod: GET
  oidcToken:
    audience: https://us-central1-mwsdata-1544225920485.cloudfunctions.net/helloworld
    serviceAccountEmail: [email protected]
  uri: https://us-central1-mwsdata-1544225920485.cloudfunctions.net/helloworld
lastAttemptTime: '2020-02-05T09:05:00.054111Z'
name: projects/mwsdata-1544225920485/locations/europe-west1/jobs/test-job
retryConfig:
  maxBackoffDuration: 3600s
  maxDoublings: 16
  maxRetryDuration: 0s
  minBackoffDuration: 5s
schedule: 5 * * * *
scheduleTime: '2020-02-05T10:05:00.085854Z'
state: ENABLED
status:
  code: 7
timeZone: Etc/UTC
userUpdateTime: '2020-02-04T22:02:31Z'

Answer 1

Here are the steps I followed to make Cloud Scheduler trigger an HTTP triggered Cloud Function that doesn't allow unauthenticated invocations:

1. Create a service account, which will have the following form [SA-NAME]@[PROJECT-ID].iam.gserviceaccount.com.
2. Adde the service account [SA-NAME]@[PROJECT-ID].iam.gserviceaccount.com as a project member and added the following roles to the service account: Cloud Functions Invoker and Cloud Scheduler Admin.
3. Deploy an HTTP triggered Cloud Function that doesn't allow public (unauthenticated) access (if you are using the UI, simply uncheck the Allow unauthenticated Invocations checkbox) and that used the recently created service account [SA-NAME]@[PROJECT-ID].iam.gserviceaccount.com on the Service account field (click more and look for the Service account field, by default it should be set to the App Engine default service account) and take notice of the Cloud Function's URL.
4. Create a Cloud Scheduler job with authentication by issuing the following command from the Cloud Shell: gcloud scheduler jobs create http [JOB-NAME] --schedule="* * * * *" --uri=[CLOUD-FUNCTIONS-URL] --oidc-service-account-email=[SA-NAME]@[PROJECT-ID].iam.gserviceaccount.com

In your specific case you are leaving the default App Engine service account for your Cloud Functions. Change it to the service account you created as specified on the previous steps.