How and why I built IVR enabled AWS cloud environment

About me

I am a cloud developer and a researcher, a Pythonista. I have built an IVR system to manage my cloud resources on the AWS cloud platform.

The problem I wanted to solve

All of you those who have picked this article might already know what is AWS or what I meant by cloud resources. So let's say I have 10 EC2 machines which are nothing but virtual machines residing in the cloud. It can be accessed from anywhere if you have an Internet connection. In my case, I won't be using 2 of those machines on weekends. In most of the Fridays, I might be traveling and I may forget to shut down those machines. I get billed extra for that even if I am not using those machines as I may forget to stop those machines.
As I might be traveling my internet connectivity will be very poor. So even if I try to stop the machines while I travel it will take a lot of time loading and loading...
This is just one single problem I was facing. I may need to get the status of machines which are running on weekends. If I am traveling or away from the system how I will check that. So I thought of using IVR technology for building a solution for this.

What is IVR enabled AWS cloud environment?

Interactive Voice Response (IVR) is an automated telephony system that interacts with callers, gathers information and routes calls to the appropriate recipients. I used this technology for making my machines down or getting the status of machines by making a call.

Tech stack

For building this system I have used Twilio (for a phone number and voice response), AWS API Gateway, AWS Lambda function.

The process of building IVR enabled AWS cloud environment

You have to get an account on Twilio before we start. You might be already having an AWS account so we are good to go.
We will write different lambda functions for starting stopping and restarting the machines. Then we will map this functions to different URLs using API gateway. And Twilio make requests to those URL's to make an action. This is how this work in brief.
Now we will build a small poc on this. Let's say we need to make machines to be started, stopped, restarted and terminated using IVR.
Let's start with lambda functions, we need 5 lambda functions here.

1. To give main options to the user

def lambda_handler(event, context):
    return '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
           '<Response><Say>Hi, PLease select an option</Say>'\
           '<Gather numDigits="1" action="https://r2ytn1yq54.execute-api.us-east-2.amazonaws.com/dev/mainactions">'\
           '<Say>Press 1 to create a machine</Say><Say>Press 2 to perform day operations on virtual machine</Say>'\
           '<Say>Press 3 to check the health status of your cloud resources</Say></Gather></Response>'

2. For giving sub options and creating machine

import boto3

REGION = 'us-east-2' # region to launch instance.
AMI = 'ami-6a003c0f'
    # matching region/setup amazon linux ami, as per:
    # https://aws.amazon.com/amazon-linux-ami/
INSTANCE_TYPE = 't2.micro' # instance type to launch.

EC2 = boto3.client('ec2', region_name=REGION)

from random import randint

def createEc2():
    tg = randint(1000,9999)
    instance = EC2.run_instances(
        ImageId=AMI,
        InstanceType=INSTANCE_TYPE,
        MinCount=1, # required by boto, even though it's kinda obvious.
        MaxCount=1,
        InstanceInitiatedShutdownBehavior='terminate', # make shutdown in script terminate ec2
        TagSpecifications=[
        {
            'ResourceType': 'instance',
            'Tags': [
                {
                    'Key': 'Name',
                    'Value': str(tg)
                },
            ]
        },
    ]
    )
    res= '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
           '<Response><Say>Your server has been created and the id is '+str(tg)+'</Say></Response>'
    return res

def lambda_handler(event, context):
    print event
    option = str(event['data']['Digits'])
    if option == '1':
        res = createEc2();
        return res
    elif option == '2':
        res = '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
           '<Response><Say>Please select an operation</Say><Gather numDigits="1" action="https://r2ytn1yq54.execute-api.us-east-2.amazonaws.com/dev/operation"><Say>Press 1 to start the machine</Say><Say>Press 2 to stop the machine</Say><Say>Press 3 to terminate the machine</Say></Gather></Response>'
        return res
    elif option == '3':
        res= '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
           '<Response><Gather numDigits="1" action="https://r2ytn1yq54.execute-api.us-east-2.amazonaws.com/dev/s3-count"><Say>Press 1 for virtual machine status</Gather></Say>'\
           '<Gather numDigits="1" action="https://r2ytn1yq54.execute-api.us-east-2.amazonaws.com/dev/s3-count">'\
           '<Say>Press 2 for getting the count of S3 buckets</Say></Gather></Response>'
        return res

3. To start the machine

import boto3
ec2 = boto3.client('ec2', region_name='us-east-2')

def lambda_handler(event, context):
    tg = str(event['data']['Digits'])
    filters = [{'Name':'tag:Name', 'Values':[tg]}
          ]
    res = ec2.describe_instances(Filters=filters)
    id = res['Reservations'][0]['Instances'][0]['InstanceId']
    try:
        st = ec2.start_instances(InstanceIds=[id])
        return '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
           '<Response><Say>Succesfully started your server</Say></Response>'
    except:
        return '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
           '<Response><Say>Failed to start</Say></Response>'

4. To stop the machine

import boto3
ec2 = boto3.client('ec2', region_name='us-east-2')

def lambda_handler(event, context):
    tg = str(event['data']['Digits'])
    filters = [{'Name':'tag:Name', 'Values':[tg]}
          ]
    res = ec2.describe_instances(Filters=filters)
    id = res['Reservations'][0]['Instances'][0]['InstanceId']
    try:
        st = ec2.stop_instances(InstanceIds=[id])
        return '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
           '<Response><Say>Succesfully stopped your server</Say></Response>'
    except:
        return '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
           '<Response><Say>Failed to stop</Say></Response>'

5. To terminate the machine

import boto3
ec2 = boto3.client('ec2', region_name='us-east-2')

def lambda_handler(event, context):
    tg = str(event['data']['Digits'])
    filters = [{'Name':'tag:Name', 'Values':[tg]}
          ]
    res = ec2.describe_instances(Filters=filters)
    id = res['Reservations'][0]['Instances'][0]['InstanceId']
    try:
        st = ec2.terminate_instances(InstanceIds=[id])
        return '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
           '<Response><Say>Succesfully terminated your server</Say></Response>'
    except:
        return '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
           '<Response><Say>Failed to terminate</Say></Response>'

Now we need to create api endpoints for these 5 functions.

1. To invoke lambda function which gives main options
2. To invoke lambda function which gives sub options
3. To invoke the lambda function which start the machine
4. To invoke the lambda function which stop the machine
5. To invoke the lambda function which terminate the machine
   Create an API in API gateway service. Follow below screenshots for creating API endpoints, resources and methods in it.
   
   Create API as shown in the screenshot
   
   Create resource by going to actions
   
   Create resource named mainoptions
   
   create a post method by going to actions
   
   Configure integration request and add below code as body mapping template

   {
    "data": {
        #foreach( $token in $input.path('$').split('&') )
            #set( $keyVal = $token.split('=') )
            #set( $keyValSize = $keyVal.size() )
            #if( $keyValSize >= 1 )
                #set( $key = $util.urlDecode($keyVal[0]) )
                #if( $keyValSize >= 2 )
                    #set( $val = $util.urlDecode($keyVal[1]) )
                #else
                    #set( $val = '' )
                #end
                "$key": "$val"#if($foreach.hasNext),#end
            #end
        #end
    }
 }

Configure integration response and add below template for integration response

#set($inputRoot = $input.path('$')) 
$inputRoot 

Configure method response as shown in the screenshot

At the end your API dashboard will look like this

Finally deploy the API

After deployment get the api url.

Now we need to configure twilio.
After logging in to twilio, go to programmable voice dashboard and then go to numbers. If you don't have a number please create one. It's free for trial.
Click on manage number and then click on the number from the console.
See the screenshots below
Step 1: Click on manage numbers

Step 2: Click on the number in red color

Step 3: Give the url of main menu API in a call comes in field

Now everything is set. Try giving a call to the number. Even you can debug most of the errors in twilio console itself.

Challenges I faced

The only problem is that we need to implement security on top of this solution. Anyone who has this number will be able to do all the operations. I secured it with an OTP authentication. So I used a paid SMS subscription for that. And yes, it added a cost. Another alternative solution is integrating authentication with Google MFA.

Key learnings

I learned how to create an IVR system on top of cloud with this stack. And it was really interesting and fun learning for me.

Tips and advice

Google MFA authentication will work better than using any SMS service. I was getting timed out when there was some delay in getting OTP.

Final thoughts and next steps

This solution might be very useful when you have a large number of servers on any cloud platform. And even it can be used for other services as well, like regenerating access key, the secret key for an IAM user or getting the count of the buckets etc.
So enjoy!! Happy coding...