AWS Lambda times out connecting to RedShift

Question

My Redshift cluster is in a private VPC. I've written the following AWS Lamba in Node.js which should connect to Redshift (dressed down for this question):

'use strict';
console.log('Loading function');

const pg = require('pg');

exports.handler = (event, context, callback) => {
var client = new pg.Client({
    user: 'myuser',
    database: 'mydatabase',
    password: 'mypassword',
    port: 5439,
    host: 'myhost.eu-west-1.redshift.amazonaws.com'
});


    // connect to our database
    console.log('Connecting...');
    client.connect(function (err) {
        if (err) throw err;

        console.log('CONNECTED!!!');

    });

};

I keep getting Task timed out after 60.00 seconds unfortunately. I see in the logs "Connecting...", but never "CONNECTED!!!".

Steps I've taken so far to get this to work:

• As per Connect Lambda to Redshift in Different Availability Zones I have the Redshift cluster and the Lamba function in the same VPC
• Also Redshift cluster and the Lamba function are on the same subnet
• The Redshift cluster and the Lamba function share the same security group
• Added an inbound rule at the security group of the Redshift cluster as per the suggestion here (https://github.com/awslabs/aws-lambda-redshift-loader/issues/86)
• The IAM role associated with the Lamba Function has the following policies: AmazonDMSRedshiftS3Role, AmazonRedshiftFullAccess, AWSLambdaBasicExecutionRole, AWSLambdaVPCAccessExecutionRole, AWSLambdaENIManagementAccess scrambled together from this source: http://docs.aws.amazon.com/lambda/latest/dg/vpc.html (I realize I have some overlap here, but figured that it shouldn't matter)
• Added Elastic IP to the Inbound rules of the Security Group as per an answer from a question listed prior (even if I don't even have a NAT gateway configured in the subnet)
• I don't have Enhanced VPC Routing enabled because I figured that I don't need it.
• Even tried it by adding the Inbound rule 0.0.0.0/0 ALL types, ALL protocols, ALL ports in the Security Group (following this question: Accessing Redshift from Lambda - Avoiding the 0.0.0.0/0 Security Group). But same issue!

So, does anyone have any suggestions as to what I should check?

*I should add that I am not a network expert, so perhaps I've made a mistake somewhere.

Answer 1

The timeout is probably because your lambda in VPC cannot access Internet in order to connect to your cluster(you seem to be using the public hostname to connect). Your connection options depend on your cluster configuration. Since both your lambda function and cluster are in the same VPC, you should use the private IP of your cluster to connect to it. In your case, I think simply using the private IP should solve your problem.

Depending on whether your cluster is publicly accessible, there are some points to keep in mind.

• If your cluster is configured to NOT be publicly accessible, you can use the private IP to connect to the cluster from your lambda running in a VPC and it should work.

• If you have a publicly accessible cluster in a VPC, and you want to connect to it by using the private IP address from within the VPC, make sure the following VPC parameters to true/yes:

  • DNS resolution
  • DNS hostnames

The steps to verify/change these settings are given here.

If you do not set these parameters to true, connections from within VPC will resolve to the EIP instead of the private IP and your lambda won't be able to connect without having Internet access(which will need a NAT gateway or a NAT instance).

Also, an important note from the documentation here.

If you have an existing publicly accessible cluster in a VPC, connections from within the VPC will continue to use the EIP to connect to the cluster even with those parameters set until you resize the cluster. Any new clusters will follow the new behavior of using the private IP address when connecting to the publicly accessible cluster from within the same VPC.