AWS CloudFormation stack fails with error Received 0 SUCCESS signal(s) out of 1

Nicholas picture Nicholas · Mar 5, 2017 · Viewed 15.8k times · Source

My AWS CloudFormation template fails with the error:

Received 0 SUCCESS signal(s) out of 1. Unable to satisfy 100% MinSuccessfulInstancesPercent requirement

AWS Coudformation stack events

I'm thinking my WaitConditionHandles are not set correctly (or maybe the EC2 instance is not sending one), but not sure how to fix this.

Everything (ASG, EC2 instances) does appear to be created correctly in AWS.

I'm using the following CloudFormation template:

AWSTemplateFormatVersion: "2010-09-09"
Description: "Auto Scaling Group"
Outputs:
  AsgArn: 
    Value: !Ref "AutoScalingGroup"
  AsgMinSize:
    Description: "The minimum size of the Auto Scaling Group"
    Value: !FindInMap [ "HighAvailability", "MinSize", !Ref "HighAvailabilityFlag" ]
Parameters:
  Ami:
    Description: "Base AMI"
    Type: "AWS::EC2::Image::Id"
  EnvironmentName:
    Description: "The environment name"
    Type: "String"
  HighAvailabilityFlag:
    Description: "Flag used to set the minimum and maximum size of the Auto Scaling Group"
    Default: false
    Type: "String"
    AllowedValues: [ "true", "false" ]
  KeyPairName:
    Description: "Name of EC2 key pair for logging in to the instances"
    Type: "String"
  SecurityGroupIds:
    Description: "The IDs of security groups that are permitted access to EC2 instances"
    Type: "String"
  Subnets:
    Description: "Subnets to associate with the ASG"
    Type: "List<AWS::EC2::Subnet::Id>"
  VersionToDeploy:
    Description: "Version to deploy"
    Type: "String"
  VpcId:
    Description: "The ID of the VPC"
    Type: "AWS::EC2::VPC::Id"
Mappings:
  HighAvailability:
    MinSize:
      "false": 1
      "true": 2
    MaxSize:
      "false": 1
      "true": 4
Resources:
  InstanceProfile:
    Properties:
      Path: "/"
      Roles:
        - !Ref "InstanceRole"
    Type: "AWS::IAM::InstanceProfile"
  InstanceRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action:
              - sts:AssumeRole
            Effect: "Allow"
            Principal:
              Service:
                - ec2.amazonaws.com
        Version: "2012-10-17"
      Path: "/"
    Type: "AWS::IAM::Role"
  Policy:
    Properties:
      PolicyDocument:
        Statement:
          - Action:
              - cloudformation:DescribeStacks
              - ec2:Describe*
            Effect: "Allow"
            Resource: "*"
        Version: "2012-10-17"
      PolicyName: "Service"
      Roles:
        - !Ref "InstanceRole"
    Type: "AWS::IAM::Policy"    
  AutoScalingGroup:
    Properties:
      HealthCheckGracePeriod: 300
      MetricsCollection:
        - Granularity: "1Minute"
      HealthCheckType: "ELB"
      LaunchConfigurationName: !Ref "LaunchConfiguration"
      MaxSize: !FindInMap [ "HighAvailability", "MaxSize", !Ref "HighAvailabilityFlag" ]
      MinSize: !FindInMap [ "HighAvailability", "MinSize", !Ref "HighAvailabilityFlag" ]
      VPCZoneIdentifier: !Ref "Subnets"
    CreationPolicy:
      ResourceSignal:
        Count: !FindInMap [ "HighAvailability", "MinSize", !Ref "HighAvailabilityFlag" ]
        Timeout: "PT5M"
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MinInstancesInService: !FindInMap [ "HighAvailability", "MinSize", !Ref "HighAvailabilityFlag" ]
        PauseTime: "PT5M"
        WaitOnResourceSignals: true
    Type: "AWS::AutoScaling::AutoScalingGroup"
  LaunchConfiguration:
    Properties:
      AssociatePublicIpAddress: true
      IamInstanceProfile: !Ref "InstanceProfile"
      ImageId: !Ref "Ami"
      InstanceType: "t2.micro"
      KeyName: !Ref "KeyPairName"
      SecurityGroups: !Split [ ",", !Join [ ",", [ !Ref "SecurityGroupIds" ] ] ]
      UserData:
        Fn::Base64:
          cfn-init.exe -v -s "AWS::StackName" --region "AWS::Region" 
          cfn-signal.exe -e 0 !Ref "WindowsServerWaitHandle"
    Type: "AWS::AutoScaling::LaunchConfiguration"
  WindowsServerWaitHandle:
    Type: "AWS::CloudFormation::WaitConditionHandle"
  WindowsServerWaitCondition:
    DependsOn: "AutoScalingGroup"
    Properties:
      Handle: !Ref "WindowsServerWaitHandle"
      Timeout: "1800"
      Count: 0
    Type: "AWS::CloudFormation::WaitCondition"

Once the EC2 instance is created I see a few logfiles being generated:

UserdataExecution.log

2017/03/05 05:54:47Z: Userdata execution begins
2017/03/05 05:54:47Z: Zero or more than one <persist> tag was not provided
2017/03/05 05:54:47Z: Unregistering the persist scheduled task
2017/03/05 05:54:50Z: Zero or more than one <runAsLocalSystem> tag was not provided
2017/03/05 05:54:50Z: Zero or more than one <script> tag was not provided
2017/03/05 05:54:50Z: Zero or more than one <powershell> tag was not provided
2017/03/05 05:54:50Z: Zero or more than one <powershellArguments> tag was not provided
2017/03/05 05:54:50Z: Userdata execution done

WindowsIsReadyToConsole.log

2017/03/03 04:46:27Z: Sending "Windows is Ready" message to console is scheduled successfully
2017/03/05 05:54:27Z: Sending windows is ready message started
2017/03/05 05:54:28Z: Opening COM port handle to write to the console
2017/03/05 05:54:30Z: Serial Port in use. Waiting for Serial Port...
2017/03/05 05:54:48Z: Message: Windows is Ready to use
2017/03/05 05:54:48Z: Sending windows is ready message done

Answer

Nicholas picture Nicholas · Mar 7, 2017

TLDR

This is a generic error which occurs when the EC2 cannot send a success signal to the ASG. There are many possible reasons why this might occur, but most likely whatever healthcheck it is you use is not working as intended.

Using the below userData should hard-code the healthcheck which is a great way to start testing your application and Cloud Formation template.

My issues

I removed all references to AWS::CloudFormation::WaitConditionHandle and AWS::CloudFormation::WaitCondition

There were issues with my UserData script:

  • The script needed <script> tags to be executed
  • The commands didn't have the correct parameters
  • The variables were not properly injected (for example ${AWS::StackName})

The result is:

UserData:
  "Fn::Base64":
    !Sub |
      <script>
        cfn-init.exe -v --stack ${AWS::StackName} --resource AutoScalingGroup --region ${AWS::Region}
        cfn-signal.exe -e 0 --stack ${AWS::StackName} --resource AutoScalingGroup --region ${AWS::Region}
      </script>