• You don't want to hardcode AWS credentials in pods.

• You want to follow AWS best practices using IAM roles for Kubernetes service accounts (IRSA).

Solution: Use IRSA (IAM Roles for Service Accounts):

With IRSA, each pod uses a Kubernetes service account that is linked to an IAM role, which defines what that pod can access in AWS.

This blog walks through enabling IRSA for your existing pods.

Assumptions

• You have an EKS cluster already running 

• Your pods are already deployed 

• You're using AWS CLI, kubectl, and eksctl

• You have access to create IAM roles and policies

Step-by-Step Guide

Step 1: Verify OIDC Provider for Your EKS Cluster

eksctl utils associate-iam-oidc-provider \
  --region <aws-region> \
  --cluster <cluster-name> \
  --approve

 Step 2: Create an IAM Policy with Required Permissions

For example, for S3 read access:

• Create s3-read-policy.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["<service-action>"],
      "Resource": ["<arn>"]
    }
  ]
}

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": ["arn:aws:s3:::rambunct-app"]
    },
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject"],
      "Resource": ["arn:aws:s3:::rambunct-app/*"]
    }
  ]
}

• Create a policy

aws iam create-policy \
  --policy-name <your-policy-name> \
  --policy-document file://s3-read-policy.json

Step 3: Create a Kubernetes Service Account Linked to IAM Role

This account will be used by your pod.

eksctl create iamserviceaccount \
  --name <serviceaccount-name> \
  --namespace <namespace> \
  --cluster <cluster-name> \
  --region <region> \
  --attach-policy-arn arn:aws:iam::<account-id>:policy/<policy-name> \
  --approve \
  --override-existing-serviceaccounts

Step 4: Patch the Pod (or Deployment) to Use the New Service Account

If you’re using plain pods (not Deployments):

kubectl delete pod <your-pod>

Then update your YAML ;

spec:
  serviceAccountName: <your-serviceaccount>

Apply it:

kubectl apply -f <your-pod>

 Step 5: Verify from Pod Logs

kubectl logs <your-pod>

Conclusion

By using IAM Roles for Service Accounts (IRSA), you can give your existing pods secure access to AWS services without compromising credentials. This is a production-grade method to connect workloads running in EKS with services like S3, Secrets Manager, and DynamoDB.

If your app needs AWS access, don’t use long-lived keys use IRSA. It's more secure, scalable, and AWS-native.

The solution to this common problem is multi-tenancy. Multi-tenancy is the practice of having multiple tenants (in this case, teams or applications) share the same EKS cluster while being logically isolated from each other. This is a highly efficient and cost-effective way to manage your Kubernetes infrastructure.

This blog post will walk you through a simple yet powerful way to achieve multi-tenancy on AWS EKS using two fundamental concepts: Kubernetes Namespaces and AWS IAM for EKS. By the end, you will have a practical setup that provides clear separation and security for different teams.

The Problem: A Single, Shared Cluster

Imagine a scenario where both the Backend Team and Frontend Team deploy their applications to the default namespace.

• Chaos: Deployments might accidentally have the same names, leading to conflicts.

• Resource Hogs: The Frontend Team's application could suddenly get a lot of traffic and use up all the cluster's CPU, making the Backend Team's services slow or unresponsive.

• Security Risks: An engineer from the Backend Team might have permissions to accidentally delete a critical service belonging to the Frontend Team.

• No Ownership: When something goes wrong, it's difficult to quickly figure out which team is responsible.

The Solution: Namespaces + IAM

Our solution is to create a dedicated namespace for each team. A namespace is a virtual partition inside a Kubernetes cluster. It gives you a way to divide cluster resources and provide a scope for names. But just creating a namespace isn't enough; we need to enforce who can access what. This is where AWS IAM comes in.

AWS EKS has a powerful feature that lets you map AWS IAM roles directly to Kubernetes RBAC (Role-Based Access Control) permissions. This allows you to say, "The IAM role for the Backend Team's developers can only access the backend-team namespace, and nothing else."

Here's the plan:

• Create an EKS Cluster: We'll start with a standard EKS cluster.

• Create Namespaces: We'll create two namespaces, backend-team and frontend-team.

• Create IAM Roles: We'll set up two IAM roles, BackendTeamDeveloper and FrontendTeamDeveloper.

• Map IAM to RBAC: We'll use EKS's built-in access management to grant each IAM role specific permissions for their respective namespaces.

• Deploy and Demo: We'll deploy a simple application for each team and demonstrate how they are isolated.

Practical Demo: Step-by-Step

This demo assumes you have the AWS CLI and kubectl configured.

Step 1: Create an EKS Cluster

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: multi-tenancy-demo
  region: us-east-1
  version: "1.28"

managedNodeGroups:
- name: standard-nodes
  instanceType: t3.medium
  desiredCapacity: 2

Now, create the cluster:

eksctl create cluster -f cluster.yaml

Step 2: Create Namespaces

Once your cluster is active, let's create a dedicated namespace for each team.

kubectl create namespace backend-team
kubectl create namespace frontend-team

You can verify the namespaces are created with:

kubectl get namespaces

Step 3: Create IAM Roles

In the AWS Console, navigate to the IAM service and create two IAM roles:

• BackendTeamDeveloper

• FrontendTeamDeveloper

These roles are what your team members will assume to interact with the cluster. For this demo, you can leave the roles without any specific permissions initially. The EKS access management will handle the permissions for the cluster itself.

Step 4: Map IAM Roles to Namespaces and Kubernetes Role RoleBinding

This is the most critical step for security. We'll use EKS's Access Management feature to map our IAM roles to specific permissions within the cluster.

You need to use the AWS CLI to create an Access Entry and Access Policy for each team. Replace 123456789 with your AWS account ID.

# For the Backend Team 

eksctl create accessentry --cluster multi-tenancy-demo \ --principal-arn arn:aws:iam::123456789:role/BackendTeamDeveloper \ --type STANDARD

# For the Frontend Team 

eksctl create accessentry --cluster multi-tenancy-demo \ --principal-arn arn:aws:iam::123456789:role/FrontendTeamDeveloper \ --type STANDARD

Now, we'll create the RBAC resources.

# This Role defines permissions for the backend team within their namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: backend-team-developer-role
  namespace: backend-team
rules:
- apiGroups: ["", "apps", "extensions"] 
  resources: ["pods", "deployments", "services", "configmaps"] 
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list"]
---
# This RoleBinding links the IAM principal to the Role
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: backend-team-developer-rolebinding
  namespace: backend-team
subjects:
- kind: User
  name: arn:aws:iam::123456789:role/BackendTeamDeveloper # This must match the principal ARN
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: backend-team-developer-role
  apiGroup: rbac.authorization.k8s.io

# This Role defines permissions for the frontend team within their namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: frontend-team-developer-role
  namespace: frontend-team
rules:
- apiGroups: ["", "apps", "extensions"]
  resources: ["pods", "deployments", "services", "configmaps"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
# This RoleBinding links the IAM principal to the Role
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: frontend-team-developer-rolebinding
  namespace: frontend-team
subjects:
- kind: User
 name: arn:aws:iam::123456789:role/FrontendTeamDeveloper # This must match the principal ARN
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: frontend-team-developer-role
  apiGroup: rbac.authorization.k8s.io

Apply these files to your cluster:

kubectl apply -f backend-team-rbac.yaml
kubectl apply -f frontend-team-rbac.yaml

Step 5: Demoing the Isolation

Let's see how this works in practice.First, assume the BackendTeamDeveloper IAM role using the AWS CLI.

aws sts assume-role --role-arn arn:aws:iam::123456789012:role/BackendTeamDeveloper --role-session-name BackendTeamSession

Use the returned credentials to configure your AWS CLI session and verify identity

aws sts get-caller-identity

Now, try to deploy an Nginx server into the backend-team namespace.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend-app
  namespace: backend-team
spec:
  replicas: 1
  selector:
    matchLabels:
      app: backend-app
  template:
    metadata:
      labels:
        app: backend-app
    spec:
      containers:
      - name: nginx
        image: nginx:latest

Deploy the application:

kubectl apply -f backend-app.yaml

This should work perfectly. The pod will be created in the backend-team namespace.

Now, with the same IAM role (BackendTeamDeveloper), try to list pods in the frontend-team namespace:

Error from server (Forbidden): pods is forbidden: User "arn:aws:sts::123456789:assumed-role/BackendTeamDeveloper/..." cannot list resource "pods" in API group "" in the namespace "frontend-team"

Again, a Forbidden error confirms our security model is working perfectly.

Conclusion

This hands-on guide confirms that secure multi-tenancy on AWS EKS is built on a two-step process: using eksctl to map IAM roles to Kubernetes identities, then using Kubernetes RBAC to enforce granular permissions. The Forbidden errors we encountered were the ultimate proof of a successful setup, confirming that our security model is fully functional and providing a robust foundation for shared EKS clusters.

⚠️ Cold starts become slower historically increasing from ~100 ms to nearly 1 second.

This slowdown has long been considered one of the trade-offs of using Lambda with VPCs. But how bad is it today? Does AWS still suffer from large cold start delays inside a VPC?

To find out, we ran a real-world experiment comparing cold starts of the same Lambda function inside a VPC and outside a VPC, measured execution times, and explored why these differences happen and how to fix them.

Q )Why Lambda Cold Starts Can Be Slower in a VPC ?

A “cold start” occurs when AWS spins up a new execution environment for your Lambda function. This usually happens when the function hasn’t been invoked for a while or when scaling up to handle more requests.

When a Lambda function runs outside a VPC, AWS handles networking internally, so the container can start almost immediately.

When it’s placed inside a VPC, however, Lambda must first:

• Create and attach an Elastic Network Interface (ENI) to your VPC subnet

• Assign private IP addresses

• Apply security groups

This ENI setup adds latency to the cold start. Historically, this could add 600–1200 ms to initialization.

AWS has since improved this process with Hyperplane ENIs, which reuse network interfaces more efficiently but the only way to know the real impact is to measure it yourself.

Step 1 : Writing a Sample Lambda Function

Let’s create a simple Python Lambda function to measure cold start time and container reuse.

import time
import datetime

# This runs once when the container is initialized (cold start)
INIT_TIME = time.time()

def lambda_handler(event, context):
    start = time.time()
    # Measure how long the container has been alive
    uptime_since_init = start - INIT_TIME

    # Simulate a small workload
    time.sleep(0.1)

    end = time.time()
    execution_duration = end - start

    return {
        "statusCode": 200,
        "timestamp": datetime.datetime.utcnow().isoformat() + "Z",
        "execution_duration": f"{execution_duration:.4f} seconds",
        "uptime_since_init": f"{uptime_since_init:.4f} seconds",
        "message": "Hello from Lambda!"
    }

What the Code Does ?

• INIT_TIME is set once when the Lambda container starts , this is how we detect cold starts.

• uptime_since_init tells us how long the container has been alive:

  • Near 0 → cold start

  • Large number → warm start

• execution_duration measures how long the function itself takes to execute.

• We also add a small time.sleep(0.1) to simulate a minimal workload.

Step 2 : Deploy Two Versions of the Lambda

To compare performance:

1. Lambda A : Outside VPC: Create a Lambda and leave the VPC setting as “No VPC.”

2. Lambda B : Inside VPC: Create another Lambda in a private subnet of a VPC.

Both functions used 128 MB memory and the same code above.

Step 3 : Invoke Both Functions 10 Times

Invoke each function 10 times at 30-second intervals. We focus on the first invocation (cold start) and subsequent ones (warm starts).

Results : Outside VPC

First invocation (cold start):

Init Duration: 114.98 ms
execution_duration: ~0.1001 seconds
uptime_since_init: ~0.0058 seconds

Subsequent invocations (warm):

Init Duration: — (not shown)
execution_duration: ~0.1001 seconds
uptime_since_init: increasing

Outside a VPC, cold starts were about 115 ms, and warm invocations stayed around 102 ms consistently

Results : Inside VPC.

First invocation (cold start):

Init Duration: 820 ms
execution_duration: ~0.100 s
uptime_since_init: ~0.006 s

Subsequent invocations (warm):

Init Duration: — (not shown)
execution_duration: ~0.100 s
uptime_since_init: increasing

Inside a VPC, cold starts were about 820 ms, and warm invocations stayed around 102 ms consistently

Side-by-Side Comparison

Observation

Lambda functions outside a VPC start up nearly 7× faster during cold starts. Once warm, both perform about the same but that cold start latency can significantly impact real-world applications.

Q)Why the Gap Exists ?

The big gap is almost entirely due to ENI setup inside a VPC. AWS must create and attach a network interface before the function can run. That adds several hundred milliseconds of delay especially if the Lambda has been idle or deployed to a new subnet.

Even with AWS’s Hyperplane ENIs and reuse improvements, ENI creation still happens under certain conditions (e.g. first invocation, long idle times), and you’ll feel that delay.

Step 4 : Reducing Cold Start Latency Even More

If your Lambda must run inside a VPC (for example, to reach a database), here’s how to reduce the impact:

1. Enable Provisioned Concurrency

Provisioned concurrency keeps execution environments warm and ready:

aws lambda put-provisioned-concurrency-config 
--function-name MyVpcLambda 
--qualifier prod 
--provisioned-concurrent-executions 5

This eliminates most cold start delays, even inside a VPC.

2. Use VPC Endpoints

If your Lambda calls other AWS services (e.g., S3, DynamoDB), create VPC endpoints to avoid NAT Gateway latency:

3. Increase Memory Size

Higher memory allocation gives your Lambda more CPU power, reducing cold start times. Even increasing from 128 MB → 512 MB can cut cold start time by 30–40%.

Key Takeaways

• Cold starts happen when AWS spins up a new container for your Lambda.

• Lambdas outside a VPC are significantly faster (~100–150 ms) because they don’t require ENI setup.

• Lambdas inside a VPC can be 5–8× slower (~600–1200 ms) during cold starts due to ENI creation.

• Improvements like Hyperplane ENIs help, but they don’t eliminate the problem.

• Provisioned Concurrency, VPC Endpoints, and memory tuning can help reduce latency inside a VPC.

Conclusion

If low latency is critical, keeping Lambda functions outside a VPC is the best choice. They start faster, respond more quickly, and avoid the overhead of ENI setup. Running Lambda inside a VPC is necessary only when connecting to private resources but you should expect and plan for slower cold starts in that scenario. Even with AWS’s improvements, ENI creation can still add hundreds of milliseconds to startup time.

This blog explains how to deploy a containerized app to ECS with GitHub Actions CI/CD and why OIDC is the modern, secure way to let your pipeline assume AWS IAM roles.

System Architecture

Step 1 : Build and Containerize the App

Why: ECS runs containers; your app must be packaged into one. So, lets create minimal Flask app.

#app.py
from flask import Flask

app = Flask(__name__)

@app.route("/")
def home():
    return "🚀 Hello from my-app on Amazon ECS using Github actions auto deployment!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=80)

#requirements.txt
flask

#Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 80
CMD ["python", "app.py"]

Step 2 : Set Up AWS Infrastructure

We need a place to store images (ECR), a cluster to run them (ECS), and a network.

2.1 Create ECR Repository

aws ecr create-repository --repository-name my-app

Gives a URI like:

XXXXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/my-app

ECR is a private Docker registry. ECS will pull images from here.

2.2 Create ECS Cluster

aws ecs create-cluster --cluster-name my-app-cluster --region <aws-region>

A cluster is just a logical group of capacity (in Fargate you don’t manage servers).

2.3 Define Task

Create ecs-task-def.json:

{
  "family": "my-app-task",
  "executionRoleArn": "arn:aws:iam::<ACCOUNT_ID>:role/ecsTaskExecutionRole",
  "networkMode": "awsvpc",
  "containerDefinitions": [
    {
      "name": "my-app",
      "image": "<ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/my-app:latest",
      "cpu": 256,
      "memory": 512,
      "essential": true,
      "portMappings": [{ "containerPort": 80 }]
    }
  ],
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "256",
  "memory": "512"
}

Register:

aws ecs register-task-definition --cli-input-json file://ecs-task-def.json

The execution role lets the task pull from ECR and send logs. The image is a placeholder; GitHub Actions will later push and update it.

2.4 Networking (Default VPC)

# Get default VPC
aws ec2 describe-vpcs --filters "Name=isDefault,Values=true" --query "Vpcs[0].VpcId" --output text
# Get subnets in that VPC
aws ec2 describe-subnets --filters "Name=vpc-id,Values=<VPC_ID>" --query "Subnets[*].SubnetId" --output text
# Get default SG
aws ec2 describe-security-groups --filters "Name=vpc-id,Values=<VPC_ID>" --query "SecurityGroups[?GroupName=='default'].GroupId" --output text
# Open HTTP if needed
aws ec2 authorize-security-group-ingress --group-id <SG_ID> --protocol tcp --port 80 --cidr 0.0.0.0/0

2.5 Create Service

aws ecs create-service \
  --cluster my-app-cluster \
  --service-name my-app-service \
  --task-definition my-app-task \
  --desired-count 1 \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[subnet-aaa,subnet-bbb],securityGroups=[sg-xxx],assignPublicIp=ENABLED}"

Service keeps 1 running copy, updates when a new task definition revision arrives.

Step 3 : Configure OpenID Connect (OIDC)

Why: avoid putting permanent AWS keys in GitHub.

3.1 Add GitHub as OIDC Identity Provider

• In IAM → Identity providers → Add provider

• Type: OpenID Connect

• URL: https://token.actions.githubusercontent.com

• Audience: sts.amazonaws.com

Attach a trust policy like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringLike": {
          "token.actions.githubusercontent.com:sub": "repo:your-username/ecsapp:ref:refs/heads/main"
        },
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}

Attach only what’s needed (I am giving full access but you can give only the required access):

• AmazonECS_FullAccess , AmazonEC2ContainerRegistryFullAccess

3.3 How OIDC Works at Runtime

1. GitHub runner asks token.actions.githubusercontent.com for a JWT.

2. JWT includes:
   iss=https://token.actions...
   sub=repo:you/ecsapp:ref:refs/heads/main
   aud=sts.amazonaws.com

3. Action configure-aws-credentials calls AssumeRoleWithWebIdentity with JWT.

4. AWS STS validates signature + claims + trust policy.

5. STS returns temporary keys (valid ~1h).

6. These keys are used by the workflow for ECR/ECS.

Step 4 : GitHub Actions Workflow

Create .github/workflows/deployment.yaml:

name: Deploy my-app to Amazon ECS

on:
  push:
    branches: [ master ]

jobs:
  deploy:                   
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::943653636298:role/GitHub_Actions_Role
          aws-region: us-east-1

      - name: Login to Amazon ECR
        id: ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Build and push Docker image
        run: |
          IMAGE_URI=\({{ steps.ecr.outputs.registry }}/my-app:\){{ github.sha }}
          docker build -t $IMAGE_URI .
          docker push $IMAGE_URI

      - name: Render Amazon ECS task definition
        id: taskdef
        uses: aws-actions/amazon-ecs-render-task-definition@v1
        with:
          task-definition: ecs-task-def.json
          container-name: my-app
          image: \({{ steps.ecr.outputs.registry }}/my-app:\){{ github.sha }}

      - name: Deploy Amazon ECS task definition
        uses: aws-actions/amazon-ecs-deploy-task-definition@v2
        with:
          task-definition: ${{ steps.taskdef.outputs.task-definition }}
          service: my-app-service
          cluster: my-app-cluster
          wait-for-service-stability: true

Push to GitHub:

git add .
git commit -m "add CI/CD"
git push origin master

Step 5 : First Deployment Flow

1. Push → triggers workflow.

   

2. GitHub gets OIDC token → AWS STS returns temporary creds.

   

3. Docker image builds and is pushed to ECR.

   

4. ECS task definition updated with new image tag.

   

5. ECS service does a rolling deployment → new container goes live.

   

6. Check ECS console → Cluster → Service → Task → copy Public IP → open in browser.

   

Conclusion

With this setup, every push to your GitHub repository triggers a secure, automated deployment to Amazon ECS. Using OpenID Connect, your workflow gets short-lived AWS credentials on demand, eliminating static keys and reducing risk. ECS Fargate runs your containers without managing servers, while ECR stores immutable images for each build.This approach is simple, scalable, and secure  a modern DevOps pattern that combines fast CI/CD with least-privilege, keyless authentication.

Table of Contents

1. Introduction

2. Bash Scripting Basics

3. Variables and User Input

4. Conditional Statements

5. Loops and Iterations

6. Functions

7. Working with Files

8. Scripting Best Practices

9. Conclusion

1. Introduction

Bash scripting is a powerful tool for automating tasks in Linux and macOS environments. It enables users to write programs in the Bash shell to perform system tasks such as file manipulation, process control, and automating repetitive tasks. This guide will walk you through the basics of Bash scripting, starting from the fundamentals and progressing to more advanced topics.

2. Bash Scripting Basics

What is Bash?

Bash (Bourne Again SHell) is the default shell in many Unix-like operating systems. It allows users to write scripts for automating tasks and processing commands directly in the terminal.

Creating Your First Bash Script

To create a Bash script, simply write commands in a text file and save it with a .sh extension. Here’s the basic structure of a script:

#!/bin/bash
# This is a comment
echo "Hello, World!"

• #!/bin/bash: This line specifies the path to the Bash interpreter.

• echo "Hello, World!": This command outputs the text "Hello, World!" to the terminal.

Sample Output:

Hello, World!

Making a Script Executable

Before you can run your script, you need to make it executable:

chmod +x script-name.sh

To run the script, use:

./script-name.sh

3. Variables and User Input

Defining Variables

Variables in Bash are used to store data. To assign a value to a variable, use the following syntax:

name="Anup"
echo "Hello, $name"

Sample Output:

Hello, Anup

User Input

To get input from the user, you can use the read command:

echo "Enter your name:"
read name
echo "Hello, $name!"

Sample Output:

Enter your name:
Anup
Hello, Anup!

4. Conditional Statements

Conditional statements allow you to execute code based on certain conditions.

If Statements

The if statement checks if a condition is true. If it is, the code inside the if block is executed.

number=12
if [ $number -gt 10 ]; then
  echo "The number is greater than 10"
fi

Sample Output:

The number is greater than 10

Else and Elif

You can extend an if statement with else and elif (else if) to handle multiple conditions:

number=10
if [ $number -gt 10 ]; then
  echo "The number is greater than 10"
elif [ $number -eq 10 ]; then
  echo "The number is equal to 10"
else
  echo "The number is less than 10"
fi

Sample Output:

The number is equal to 10

Case Statements

The case statement is useful when you have many conditions to check:

choice="apple"
case $choice in
  "apple")
    echo "You selected apple"
    ;;
  "banana")
    echo "You selected banana"
    ;;
  *)
    echo "Invalid selection"
    ;;
esac

Sample Output:

You selected apple

5. Loops and Iterations

Loops are essential for repeating tasks, especially when dealing with lists or conditions.

For Loop

A for loop repeats a command for a specified number of times or through a list:

for i in {1..5}; do
  echo "Iteration $i"
done

Sample Output:

Iteration 1
Iteration 2
Iteration 3
Iteration 4
Iteration 5

While Loop

A while loop continues to execute as long as a given condition is true:

count=1
while [ $count -le 5 ]; do
  echo "Iteration $count"
  ((count++))
done

Sample Output:

Iteration 1
Iteration 2
Iteration 3
Iteration 4
Iteration 5

Until Loop

An until loop is the opposite of a while loop—it runs until the condition becomes true:

count=1
until [ $count -gt 5 ]; do
  echo "Iteration $count"
  ((count++))
done

Sample Output:

Iteration 1
Iteration 2
Iteration 3
Iteration 4
Iteration 5

6. Functions

Functions help organize your code into reusable blocks.

Defining a Function

To define a function, use the following syntax:

greet() {
  echo "Hello, $1"
}
greet "Anup"

Sample Output:

Hello, Anup

Returning Values from Functions

You can also return values from functions using echo and capture them using command substitution:

add() {
  result=$(( $1 + $2 ))
  echo $result
}

sum=$(add 5 7)
echo "The sum is $sum"

Sample Output:

The sum is 12

7. Working with Files

Bash provides several commands for file manipulation. Here are a few common operations:

Creating Files

To create an empty file, use the touch command:

touch newfile.txt

Reading Files

You can read the contents of a file using cat:

cat file.txt

Sample Output:

This is a file for blog.anupkafle.com.np.

Redirecting Output to a File

You can redirect the output of a command to a file using >:

echo "Welcome to blog.anupkafle.com.np" > output.txt

Appending to Files

To append data to a file, use >>:

echo "New line" >> output.txt

8. Scripting Best Practices

Use Comments

Always add comments to explain your code. This makes it easier to understand and maintain.

# This script prints a greeting
echo "Hello, World!"

Sample Output:

Hello, World!

Use Meaningful Variable Names

Choose variable names that describe their purpose. Avoid single-letter variables unless absolutely necessary.

user_name="Anup"

Error Handling

Use proper error handling to ensure your script behaves as expected even when something goes wrong. The exit command can be used to terminate a script with an exit status:

 if [ ! -f "file.txt" ]; then
  echo "File not found!"
  exit 1
fi

Sample Output:

File not found!

Test Your Scripts

Always test your scripts with different inputs and edge cases to ensure they work as expected.

9. Conclusion

Bash scripting is an essential skill for automating tasks and simplifying repetitive operations on Unix-based systems. By understanding the basics such as variables, conditionals, loops, and functions, you can write efficient and powerful scripts to manage your system more effectively.

Remember to always test your scripts, use comments, and follow best practices to ensure your code is clean, readable, and maintainable.

Happy scripting!

Starting with Version Control: Understanding Git Basics

Q. What is Git?

Git is a distributed version control system that allows developers to track changes, collaborate on code, and manage multiple versions of a project efficiently.

Q. What is a Version Control System (VCS)?

A Version Control System (VCS) is a software tool that helps developers manage changes to source code or other files over time. It enables collaboration, tracks changes, and provides a history of modifications, allowing teams to work on projects efficiently while minimizing conflicts.

Installing and Setting Up Git: A Guide for Windows and Linux

Installing Git on Windows

Step 1: Download Git

• Visit the official Git website and download the latest version for Windows.

• Choose the appropriate version for your system (32-bit or 64-bit).

Step 2: Install Git

1. Open the downloaded installer and follow the setup wizard.

2. Choose your preferred editor for Git (default is Vim, but you can select Notepad++ or VS Code).

Step 3: Verify Installation

• Open Command Prompt or PowerShell.

• Type the following command to verify the installation:

    git --version
  

• You should see the installed Git version.

Step 4: Configure Git

• Set your name and email:

    git config --global user.name "Your Name"
    git config --global user.email "hello@anupkafle.com.np" // Enter your Email
  

Installing Git on Linux

Step 1: Update Your System

• Before installing Git, ensure your system is up-to-date.

    sudo apt update && sudo apt upgrade
  

Step 2: Install Git

• The installation commands vary based on your Linux distribution:

  For Ubuntu/Debian:

    sudo apt install git
  

  For Fedora/RHEL:

    sudo dnf install git
  

  For Arch Linux:

    sudo pacman -S git
  

Step 3: Verify Installation

• Check if Git is installed and its version:

    git --version
  

Step 4: Configure Git

• Similar to Windows, set your global username and email:

    git config --global user.name "Your Name"
    git config --global user.email "hello@anupkafle.com.np" // Enter your Email
  

Check Your Git Configuration

After configuring your name and email in Git, it’s important to verify the settings to ensure everything is set up correctly. These details are crucial as they are attached to every commit you make and help identify who made the changes.

Check Your Global Configuration

Run the following command to view your global Git configuration, including your name and email:

git config --global --list

You should see output similar to this:

msi@anup:~$ git config --global --list
user.name=anupkafle
user.email=hello@anupkafle.com.np

Check Configuration for a Specific Repository

If you want to check the configuration for a specific repository, navigate to the repository folder and use:

git config --list

This will show both global and repository-specific configurations. If you’ve overridden the global configuration for the repository, the repository-specific settings will appear here.

Git Commands with Detailed Explanations and Examples

1. Initializing a Repository

git init

What It Does:

Creates a new Git repository in the current directory by initializing a .git folder.

Example:

mkdir my-project
cd my-project
git init

This initializes an empty Git repository in the my-project directory.

2. Cloning a Repository

Command:

git clone <repository-url>

What It Does:

Copies a repository (and its history) from a remote location to your local machine.

Example:

git clone https://github.com/anupkafle/testrepo.git

This clones the repository at https://github.com/anupkafle/testrepo.git into a local folder named anupkafle.

3. Checking the Status of Your Repository

Command:

git status

What It Does:

Shows the status of your working directory and staging area, including:

• Files modified but not staged.

• Files staged but not committed.

• Untracked files.

Example:

git status

Output might look like:

msi@msi:~/Desktop/my-project$ git status
On branch master

No commits yet

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    file1.txt

nothing added to commit but untracked files present (use "git add" to track)

4. Adding Files to the Staging Area

Command:

git add <file-name>    # Add a specific file
git add .              # Add all files

What It Does:

Moves changes from the working directory to the staging area, marking them for the next commit.

Example:

git add file1.txt
git add .

This stages the file1.txt file or all files in the directory, respectively.

5. Committing Changes

Command:

git commit -m "Your commit message"

What It Does:

Saves the changes in the staging area to the repository with a descriptive message.

Example:

git commit -m "Added a new feature to the project"

This commits all staged changes with the message "Added a new feature to the project."

Output might look like:

[master (root-commit) 2e0d35b] Added a new feature to the project
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 file1.txt

6. Viewing Commit History

Command:

git log

What It Does:

Shows the commit history for the repository, including commit hashes, author details, dates, and messages.

Example:

git log

Output:

commit 2e0d35bf87987f7bdcd8bd9cc6ac6051e8b12ab6 (HEAD -> master)
Author: anupkafle <hello@anupkafle.com.np>
Date:   Fri Nov 29 22:15:56 2024 +0100

    Added a new feature to the project

7. Creating and Switching Branches

Commands:

git branch <branch-name>       # Create a new branch
git checkout <branch-name>     # Switch to an existing branch
git checkout -b <branch-name>  # Create and switch to a new branch

What They Do:

• Branch creation: Creates a new branch to isolate work on a specific feature or bug.

• Switching branches: Moves you to another branch in the repository.

Example:

git checkout -b feature/new-feature

This creates and switches to a branch named feature/new-feature.

Output:

Switched to a new branch 'feature/new-feature'

8. Merging Branches

Command:

git merge <branch-name>

What It Does:

Combines changes from another branch into the current branch.

Example:

git checkout master
git merge feature/new-feature

This merges the feature/new-feature branch into the main branch.

9. Pushing Changes to a Remote Repository

Command:

git push origin <branch-name>

What It Does:

Uploads local commits from the specified branch to a remote repository.

Example:

git push origin master

Pushes the main branch to the remote repository. To proceed, you must first create a repository and link it to a remote URL.

To link your local repository to a remote repository, use the following command:

git remote add origin <remote-repository-url>

10. Pulling Changes from a Remote Repository

Command:

git pull

What It Does:

Fetches and merges changes from the remote repository into the current branch.

Example:

git pull

Updates the local branch with the latest changes from the remote branch.

11. Viewing Differences

Command:

git diff

What It Does:

Shows changes between the working directory and the repository (or between branches).

Example:

git diff

Displays line-by-line differences for modified files.

Output:

msi@msi:~/Desktop/my-project$ git diff
diff --git a/file1.txt b/file1.txt
index e69de29..537e61e 100644
--- a/file1.txt
+++ b/file1.txt
@@ -0,0 +1 @@
+hi this is test file

The git diff output shows the differences between the current working directory and the latest commit. In this case, the file file1.txt was modified, transitioning from an empty state (indicated by e69de29) to containing a single line, hi this is test file. The + symbol marks the addition of this line in the file.

12. Discarding Changes

Commands:

git checkout -- <file-name>  # Reverts changes to a file
git reset <file-name>        # Removes a file from staging

What They Do:

• Reverting: Discards changes in the working directory.

• Resetting: Removes changes from the staging area without deleting them.

Example:

msi@msi:~/Desktop/my-project$ cat > file1.txt 
Hello
msi@msi:~/Desktop/my-project$ cat file1.txt 
Hello
msi@msi:~/Desktop/my-project$ git checkout -- file1.txt
msi@msi:~/Desktop/my-project$ cat file1.txt 
msi@msi:~/Desktop/my-project$

13. Deleting a Branch

Command:

git branch -d <branch-name>

What It Does:

Deletes a branch that is no longer needed.

Example:

msi@msi:~/Desktop/my-project$ git branch
  feature/new-feature
* master
msi@msi:~/Desktop/my-project$ git branch -d feature/new-feature
Deleted branch feature/new-feature (was 2e0d35b).
msi@msi:~/Desktop/my-project$ git branch
* master
msi@msi:~/Desktop/my-project$

Deletes the feature/new-feature branch.

14. Checking Remote Repositories

Command:

git remote -v

What It Does:

Lists remote repositories linked to your local repository.

Example:

git remote -v

Output:

msi@msi:~/Desktop/my-project$ git remote -v
origin    https://github.com/anupkafle/testrepo.git (fetch)
origin    https://github.com/anupkafle/testrepo.git (push)

15. Tagging Releases

Command:

git tag -a <tag-name> -m "Tag message"

What It Does:

Marks a specific commit with a version number or release label.

Example:

msi@msi:~/Desktop/my-project$ git tag -a v1.0.0 -m "Version 1.0.0"
msi@msi:~/Desktop/my-project$ git log
commit 2e0d35bf87987f7bdcd8bd9cc6ac6051e8b12ab6 (HEAD -> master, tag: v1.0.0)
Author: anupkafle <anupkafle24@gmail.com>
Date:   Fri Nov 29 22:15:56 2024 +0100

    Added a new feature to the project

Creates and pushes a v1.0.0 tag to the remote repository.

16. Stashing Changes

Git stash is a feature that allows you to temporarily save changes in your working directory that you don’t want to commit yet. This is useful when you need to switch branches or perform other tasks without losing your uncommitted changes. The changes are saved in a stack-like structure, and you can reapply them later.

Command:

git stash

What It Does:

Temporarily saves changes without committing.

Example:

git stash

To apply stashed changes later:

git stash apply

17. Undoing Changes

Command:

git reset --hard <commit-hash>

What It Does:

Resets the repository to a specific commit, discarding all changes after it.

Example:

git reset --hard 1a2b3c4d5e6f7g8h9i0j

Rolls back to the specified commit.

18. Help and Documentation

Command:

git help <command>

What It Does:

Provides detailed documentation for any Git command.

Example:

git help commit

Displays help information for the commit command.

19.Git Rebase

Command:

git rebase <branch-name>

What It Does:

• Integrates changes from one branch into another by moving the base of your branch to the latest commit on the target branch.

• Keeps a linear commit history by replaying your changes on top of the target branch.

Example:

git rebase main

Rebases your current branch on top of the main branch.

Use Case:

When you want to synchronize your feature branch with the latest updates from the main branch while avoiding merge commits.

20. Git Cherry-Pick

Command:

git cherry-pick <commit-hash>

What It Does:

• Applies a specific commit from one branch onto the current branch.

Example:

git cherry-pick abc1234

Applies the commit with hash abc1234 onto your current branch.

Use Case:

When you need a specific change from another branch without merging the entire branch.

Conclusion

Git is an indispensable tool for modern software development, enabling teams to collaborate efficiently, maintain code integrity, and streamline project management. Its powerful commands, such as git commit, git branch, git rebase, and git stash, provide flexibility and control over your codebase, making it easier to manage complex projects. By mastering Git’s workflows—whether it’s basic version control, branch management, or advanced rebasing techniques—you can significantly enhance your productivity and contribute to a cleaner, more maintainable project history.

Incorporating Git into your development process not only helps you work better individually but also ensures seamless teamwork, especially when paired with platforms like GitHub or GitLab. Understanding Git’s capabilities and best practices is a crucial step in your journey toward becoming a proficient developer or DevOps engineer. With a strong foundation in Git, you are equipped to tackle challenges in version control, collaborate effectively, and deliver high-quality software.

FAQs

1. What is the difference between git pull, git fetch, and git clone?

   git pull, git fetch, and git clone serve different purposes in Git workflows. git pull combines git fetch and git merge, fetching changes from the remote repository and immediately merging them into your current branch, making it ideal for quick synchronization. git fetch, on the other hand, only downloads changes from the remote repository without merging, allowing you to review updates before integrating them. Finally, git clone is used to create a local copy of an entire remote repository, typically when setting up a repository for the first time. Each command has its unique use case: pull for immediate updates, fetch for careful review, and clone for starting fresh.

The evolution from DevOps to NoOps represents a shift toward eliminating traditional IT operations through extensive automation, thus reducing the operational burden on developers and allowing them to focus more on delivering higher-value aspects of software development. Companies like Netflix, Airbnb, Spotify, and Slack are at the forefront of this movement, adopting various NoOps principles to streamline operations.

The Evolution from DevOps to NoOps:

DevOps has transformed the collaboration between development and operational teams, optimizing both the creation and deployment phases of software. NoOps extends this philosophy to its logical endpoint: the automation of operational tasks to such an extent that the operational activities are almost invisible.

Core Principles of NoOps:

• Full Automation: Everything from resource allocation to application deployment and network adjustments is automated.

• Proactive Monitoring and Self-Healing: Systems are not only monitored automatically but can correct themselves without human intervention.

• Cloud-native Infrastructure: Emphasizes using solutions like serverless computing to minimize traditional operational management.

Purpose:

1. The main purpose of this article is to perform performance comparision between NoOps and DevOps using AWS Services.

Background:

In this section we will be familiar with DevOps and NoOps cloud model in Amazon Web Service(AWS)

DevOps Based Model

In the AWS ecosystem, DevOps methodologies are greatly enhanced by services like Amazon EC2 and Amazon EKS, which cater to distinct needs while supporting rapid and efficient application development and deployment. Amazon EC2 (Elastic Compute Cloud) is a cornerstone of AWS, offering flexible, virtual computing environments that users can customize and control. EC2 allows for the configuration of processing power, memory, and storage that suits diverse application needs, supported by various pricing options including On-Demand, Reserved, and Spot Instances, which help optimize costs based on usage patterns. Additionally, EC2's integration with AWS security and networking services ensures robust protection and scalability for applications.

On the other hand, Amazon EKS (Elastic Kubernetes Service) streamlines the deployment, scaling, and management of containerized applications using Kubernetes, without the need for installing or operating Kubernetes control planes. This service is especially beneficial for modern applications designed around microservices architectures that require dynamic scaling and high availability. EKS seamlessly integrates with essential AWS services such as Elastic Load Balancing, Amazon VPC, and IAM, enhancing both the security and functionality of container deployments. Furthermore, although EKS introduces an additional cost for managing the Kubernetes infrastructure, it significantly reduces the complexity and overhead associated with manual Kubernetes management.

NoOps Based Model

The NoOps (no operations) model, exemplified by AWS Lambda and Function as a Service (FaaS), represents a paradigm shift in software development, minimizing or eliminating traditional operational tasks through automation and abstraction. AWS Lambda, a serverless computing service, allows developers to run code in response to events without managing servers, perfectly aligning with the NoOps approach by automating scaling, patching, and managing of compute resources. Lambda supports multiple programming languages, triggers functions via events from AWS services or third-party apps, and automatically scales to match demand. This model drastically reduces operational management, enabling rapid deployment and significant cost efficiency due to its pay-forwhat-you-use pricing structure. Lambda is ideal for a variety of applications, from web backends to real-time data processing and IoT services, facilitating a focus on creating business value rather than infrastructure management, thus accelerating innovation and efficiency in development processes.

Methodology

To understand the performance differences between the DevOps and NoOps deployment strategies we will create an app that calculates the prime number between some intervals and deploy using AWS Lambda and AWS Elastic Kubernetes Service and test the load using Jmeter, an API testing tool by Apache.

Prime Number Calculator:

To effectively compare the performance and various metrics of NoOps and DevOps approaches, I utilized Amazon Web Services (AWS) to demonstrate each model. Specifically, AWS Elastic Kubernetes Service (EKS) was employed as an example of a DevOps-oriented service, and AWS Lambda was used to illustrate the NoOps model.

The application chosen for this comparison was a simple prime number calculator that identifies all prime numbers within a given range, specified by a start and an end interval. This application was developed using Flask, a lightweight web framework in Python.

Here’s how the deployment proceeded:

• Development of the Flask Application: The application was coded in Python and designed to accept two parameters: 'start' and 'end'. It calculated and returned all prime numbers within this interval.

• Deployment on AWS Lambda (NoOps): The Flask application was packaged and deployed on AWS Lambda, which abstracts away most server management tasks. This deployment was used to evaluate how the NoOps approach simplifies operations, particularly in terms of server provisioning, scaling, and management.

• Deployment on AWS EKS (DevOps): The same Flask application was containerized using Docker and deployed on a Kubernetes cluster managed by AWS EKS. This setup highlighted the level of control and flexibility provided by a DevOps approach, which can be essential for handling complex application architectures that require fine-tuned scaling and management.

• Performance Metrics and Comparison: After deploying the application on both platforms, key performance metrics such as deployment time, scalability, response time, and resource utilization were analyzed and compared. This comparison aimed to illustrate the practical impacts of choosing NoOps versus DevOps in a cloud environment.

Figure: API to calculate prime number

Table: Different metrics for lambda api on /api/primes endpoint

Figure: Aggregate Graph for Lambda Endpoint

Table: Different metrics for Kubernetes api on /api/primes endpoint

Figure: Aggregate Graph for Lambda Endpoint

Performance Metrics:

1. Load Time, Connect Time, and Latency:

   • Lambda API: Load time and latency for the Lambda API are nearly identical, suggesting that the major component of load time is the actual processing (latency). There is a noticeable increase in both metrics as the number of primes (End value) increases, peaking at 100,000 with a load time and latency of 2146 ms.

   • Kubernetes API: Similarly, Kubernetes shows increased load time and latency with the number of primes, but notably, the values spike drastically at 100,000 primes to 24150 ms, suggesting a significant degradation in performance under high computational load.

2. Connect Time

   • The connect time for both Lambda and Kubernetes is relatively low compared to load times, indicating that the overhead for establishing a connection is minimal. Kubernetes does show slightly lower connection times, which could be due to differences in network configuration or the persistent nature of containers compared to the stateless deployment of Lambda.

3. Error Rates

   • The Lambda API shows an error rate of 2.97%, which suggests some failures under certain conditions. In contrast, the Kubernetes API shows a 0.00% error rate, indicating more stable performance in handling requests without failures.

4. Throughput

   • The Lambda API shows a higher throughput of 54.5 requests per second compared to Kubernetes' 3.3 requests per second. This significant difference suggests that Lambda can handle a greater number of requests concurrently, likely benefiting from AWS's auto-scaling capabilities inherent to Lambda services.

Analysis

• Scalability: Lambda demonstrates better scalability with higher throughput, suggesting it is more capable of handling spikes in traffic without manual intervention for scaling.

• Performance Under Load: Kubernetes API performance deteriorates significantly when dealing with a large number (100,000) of primes, indicating potential issues in handling heavy computational tasks efficiently.

• Reliability: Kubernetes shows a better error rate, indicating it might be more reliable for consistent performance, particularly in scenarios where error rates are critical.

• Cost Implications: While not explicitly mentioned, Lambda's pricing model (based on requests and compute time) could potentially be more cost-effective for applications with variable traffic. Kubernetes may incur higher costs due to continuous running of cluster nodes.

Conclusion

The performance metrics comparison between AWS Lambda and AWS Kubernetes (EKS) for the /api/primes Flask API reveals distinct advantages and disadvantages depending on the specific use case and operational requirements:

• AWS Lambda excels in scalability and throughput, demonstrating robust performance particularly under variable load conditions. It is capable of handling a high number of requests per second, indicating efficient use of resources and rapid auto-scaling. This makes Lambda an excellent choice for applications with unpredictable traffic patterns and those that do not require a persistent state. However, there is a concern with a higher error rate, which suggests that for extremely sensitive applications, more robust error handling or configuration may be needed.

• AWS Kubernetes (EKS), on the other hand, provides a more stable error-free operation as indicated by the zero error rate in the tests, but struggles with performance under heavy computational loads, as seen with the significant latency increase at 100,000 primes. This points to Kubernetes being more suitable for applications where consistent performance and more extensive customization are required. Additionally, it might be better for long-running processes where the overhead of starting up a new instance, as in the case of Lambda, could be detrimental.

• Cost-effectiveness will vary: Lambda may generally be more cost-efficient for workloads with high peaks and low troughs of activity due to its pay-per-use model. Kubernetes may entail higher baseline costs due to the need to run some resources continuously but could potentially offer cost savings through more detailed control over resource allocation and usage.

• Throughput and Performance Under Load: Lambda's superior throughput makes it suitable for high-demand applications, while Kubernetes may require additional optimizations and resource allocation strategies to handle similar loads effectively.

User authentication is a critical component of modern web and mobile applications. Implementing secure and user-friendly authentication can be a complex task. However, with the powerful combination of Amazon Cognito and Google integration, developers can streamline the authentication process while leveraging the security and convenience of Google accounts. In this blog, we will explore how to integrate Google Sign-In with Amazon Cognito, allowing you to enhance your application's user experience and security.

Why Choose Amazon Cognito?

Amazon Cognito is a fully managed service by AWS that provides authentication, authorization, and user management for your applications. It offers several benefits:

1. Scalability: Amazon Cognito can handle millions of users, ensuring your application scales effortlessly as your user base grows.

2. Security: It supports industry-standard protocols, including OpenID Connect and OAuth 2.0, ensuring secure authentication and authorization flows.

3. Flexibility: Amazon Cognito supports various authentication methods, including social identity providers like Google, enabling you to offer multiple login options to your users.

4. User Management: It provides comprehensive user management features, such as user registration, user profile management, and password resets, reducing the development effort required for these functionalities.

Steps to Integrate Google Sign-In with Amazon Cognito:

Step1: Set Up Google Developer Account:

• Create a project in the Google Developers Console.

Enable the Google Sign-In API for your project. Set up the OAuth consent screen to configure and register the application.

Select the External and click on CREATE.

App information, App logo, App domain, Developer contact information, Test users must be configured as per the requirement for registering the app.

Configure the authorized JavaScript origins and redirect URIs for your application.

• Setup the Credentials to access the enabled APIs with OAuth client ID

In the application type, multiple options are available, as of now select Web application.

In the next click CREATE OAuth client ID. Authorized redirect URIs will be set up later.

After this Client ID and Client Secret will be provided which will be used to set up google as federated identity provider in the step later.

Step2: Create an Amazon Cognito User Pool:

• Create a user pool in Amazon Cognito to manage user registration and authentication.

  Search the Amazon Cognito service and click on Create user pool

In the authentication providers select Federated Identity providers

• In the Cognito user pool sign-in option choose the attribute that will be used to sign in. In the federated sign-in options select Google

• Configure the user pool settings, such as password policies, email verification, user attributes, sign-up experience, and message delivery.

• In the Connect federated identity provider set up Google federation with this user pool. Provide the client ID and client secret obtained from the Google Developers Console. Authorized scopes can be selected as per choice for instance openid email profile.

Now map the attributes between Google and user pool

• Integrate App and set up client:

Provide the favorable user pool name

Set up the app client and choose whether to generate client secret or not.

Provide the domain name for Hosted UI and OAuth 2.0 endpoints. The domain name must be unique.

Set up the callback URL to redirect the user back after authentication,=.

In the advanced app client setting select identity provider and OAuth 2.0 grant types. The Implicit Grant is an OAuth 2.0 authorization flow used in web applications. It's suitable for JavaScript-based applications running in web browsers or environments where client secrets cannot be securely stored. In this flow, the client application directly obtains the access token from the authorization server without a separate token exchange step. It involves redirecting the user to the authorization server, authentication, and granting permission.

Now, at last click on Create user pool

Step3:Set up Authorised redirect URI's.

Go to the Google developer console in the Authorised redirect URI's and provide the URI's as:

Copy

https://yourDomainPrefix.auth.region.amazoncognito.com/oauth2/idpresponse

Replace yourDomainPrefix and region accordingly from the values of user pool. and SAVE.

Step4: Check the Hosted UI

Select User pool created above and Click on App integration

At the bottom select the corresponding app client name you created earlier

In the hosted click on View Hosted UI

The output can be seen as below where sign in with google is available.

Successfully User Authentication with Amazon Cognito and Google Integration is done. This can also be integrated with own hosted UI's in own website.

I experienced this firsthand while working on a shoe e-commerce platform that catered to customers looking for the latest trends in footwear. During regular days, the traffic was steady and manageable. However, the traffic would surge unpredictably whenever we ran promotional events, offered discounts, or had flash sales. Initially, we tried to handle this by provisioning more servers manually before each event, but this approach could have been more efficient and effective. We often found ourselves over-provisioning and wasting resources or under-provisioning and facing downtimes, which frustrated customers and led to lost sales.

The need for a dynamic solution that automatically adjusts resources based on real-time demand became apparent. This is where AWS Auto Scaling came into play, transforming how we handled traffic spikes and ensuring our platform could meet users' needs, no matter the load.

The Challenge: Unpredictable Traffic and Performance Bottlenecks

Our platform was designed to handle several hundred users concurrently during non-peak hours. But as soon as we announced a sale on popular shoe brands, the number of concurrent users would skyrocket, sometimes even reaching tens of thousands. This led to a range of issues:

• Server Overload: Our EC2 instances struggled to manage the increased load, resulting in slow response times and, in some cases, server crashes.

• Manual Intervention: To prevent crashes, we manually added more instances before every significant event. This was labor-intensive and prone to errors, as predicting the resources needed required a lot of work.

• Cost Inefficiency: After the traffic spike subsided, the additional instances would sit idle, consuming resources unnecessarily and increasing our operational costs.

We needed a scalable solution that could adapt to traffic in real-time, ensuring optimal performance without manual intervention or wasted resources.

The Solution: Implementing AWS Auto Scaling

After researching various solutions, we decided to implement AWS Auto Scaling. This Feature automatically adjusts the number of Amazon EC2 instances based on the current demand, ensuring that the application can handle the load efficiently while optimizing costs.

Here's how we implemented AWS Auto Scaling to solve our scalability challenges:

1. Configuring Auto Scaling Groups: We created an Auto Scaling group (ASG) for our EC2 instances. The ASG allowed us to define the minimum and maximum number of instances running simultaneously. We set a minimum of two instances to ensure redundancy and a maximum of 20 instances, sufficient to handle our highest anticipated load.

   

   figure : Amazon EC2 Auto Scaling

2. Setting Up Scaling Policies: The next step was to define scaling policies based on CPU utilization. For example, we set a policy to add an EC2 instance when the average CPU utilization across our instances exceeded 70%. Similarly, if CPU utilization dropped below 30%, the ASG would terminate unnecessary instances to reduce costs. These policies ensured that our infrastructure could automatically scale in response to real-time demand without human intervention.

3. Integrating Elastic Load Balancing (ELB): To distribute incoming traffic evenly across all available instances, we integrated Elastic Load Balancing (ELB) with our Auto Scaling group. ELB automatically routes traffic to the healthiest instances, ensuring no single instance is overwhelmed with requests. This improved our application's reliability and enhanced the user experience by providing faster response times.

   

   figure: ELB with Auto Scaling

4. Monitoring and Optimization with CloudWatch: We used Amazon CloudWatch to monitor the performance of our Auto Scaling setup. CloudWatch provided real-time insights into CPU utilization, memory usage, and response times. Analyzing these metrics allowed us to fine-tune our scaling policies, ensuring our infrastructure was constantly optimized for performance and cost.

   

   Image source : plainenglish

5. Scaling Across Multiple Availability Zones: To further enhance availability and fault tolerance, we configured our Auto Scaling group to launch instances across multiple Availability Zones (AZs). This ensured that even if one AZ experienced issues, our application would remain operational and continue serving users from other AZs.

6. Utilizing Spot Instances for Cost Savings: To optimize costs further, we leveraged Spot Instances for non-critical workloads. Spot Instances are available at a significant discount compared to On-Demand Instances, allowing us to save on costs without compromising performance. We set up a mixed instances policy within our Auto Scaling group, combining On-Demand and Spot Instances to balance cost efficiency and availability.

   

figure: AWS EC2 Spot Instances

The Outcome: Seamless Scaling and Cost Efficiency

Implementing AWS Auto Scaling was a game-changer for our shoe e-commerce platform. The results were immediate and impactful:

• Automatic Scaling: During our next promotional event featuring discounts on popular shoe brands, we witnessed the true power of AWS Auto Scaling. As traffic surged, the Auto Scaling group automatically added more EC2 instances to handle the load. Once the event ended and traffic returned to normal, the group scaled down, ensuring we only paid for the needed resources.

• Improved Performance: With Auto Scaling and ELB in place, our application handled peak traffic without downtime or slow response times. This led to a better user experience, higher customer satisfaction, and increased successful transactions.

• Cost Optimization: We significantly reduced our infrastructure costs by automating the scaling process and incorporating Spot Instances. We no longer had to over-provision resources "just in case" or worry about idle instances draining our budget.

• Operational Efficiency: The automation provided by AWS Auto Scaling freed up our team to focus on other critical tasks. We no longer had to manually adjust infrastructure before and after events, which reduced the risk of human error and allowed us to invest more time in developing new features and improvements for our platform.

Lessons Learned and Best Practices

Through this experience, we learned several valuable lessons and developed best practices for using AWS Auto Scaling:

1. Start Small and Scale: Start with conservative scaling policies and closely monitor performance. Adjust policies gradually based on real-world data to avoid unnecessary costs or under-provisioning.

2. Leverage Multiple Metrics: While CPU utilization is a common trigger for scaling, consider other metrics like memory usage, disk I/O, or request count, depending on your application's behavior.

3. Use Mixed Instances for Cost Efficiency: Combining On-Demand and Spot Instances can provide the best balance between availability and cost. However, ensure that your application can handle the potential interruption of Spot Instances.

4. Regularly Review and Optimize: AWS environments are dynamic, and so should your scaling policies. Review CloudWatch metrics regularly and adjust your Auto Scaling configuration to ensure it remains aligned with your business needs.

5. Test Under Load: Before going live with a significant event, simulate high traffic conditions to test your Auto Scaling setup. This will help you identify bottlenecks or configuration issues before they impact real users.

Conclusion: Future-Proofing Your Applications with AWS Auto Scaling

Scalability is a fundamental requirement for any successful application, especially in today's competitive landscape, where user expectations are high, and downtime can result in significant financial losses. AWS Auto Scaling provided a robust, automated solution to effortlessly scale our shoe e-commerce platform, ensuring we could handle any traffic spike while optimizing costs and maintaining high performance. For any developer or organization looking to future-proof their applications, AWS Auto Scaling is an essential tool in your arsenal. By embracing automation, you can build systems that are not only scalable but also resilient, cost-effective, and capable of delivering an excellent user experience regardless of the demand.

The Problem: Multiple ALBs Leading to High Costs

The client had a microservices architecture spread across multiple regions, and each microservice was behind its own ALB. While this setup ensured isolation and flexibility, it also came with a hefty price tag, especially in the development environment where cost optimization is often overlooked.

Each ALB incurs a base cost, plus additional charges based on the amount of traffic processed. With 14-15 ALBs per region, the costs quickly added up, making it clear that a more efficient solution was needed.

Figure: Architecture for Deployed Microservices

The Solution: A Single ALB with Host-Based Routing

To address this, I proposed and implemented the use of a single ALB per region, utilizing host-based routing to manage traffic to the various microservices. This approach not only simplified the architecture but also significantly reduced costs. Here’s how you can do it too.

Step-by-Step Implementation

1. Setup the Application Load Balancer

   • First, create a single ALB in the AWS region where your microservices are hosted. This ALB will handle traffic for all the microservices in that region.

     

     

2. Configure Listeners

   • ALBs operate on Layer 7, allowing them to inspect the HTTP/HTTPS headers. Set up a listener on port 80 (HTTP) and redirect to 443(HTTPS) and also port 443 (HTTPS).

     

     

     

• Forward incoming traffic to the appropriate target groups based on host headers.

  

  

  

  

  

  • Repeat the same process for website 2 and others if any

    

    

3. DNS Configuration

   • Update your DNS settings to point the different subdomains (e.g., service1.example.com) to the ALB. This can be done using Route 53 or any other DNS service you’re using.

     

     

4. Testing and Validation

   • After setting up the ALB and routing rules, thoroughly test the setup to ensure that traffic is correctly routed to the respective microservices. Validate that each service is accessible through its designated domain.

     

     

     The Results: Significant Cost Savings

     After implementing this single ALB setup, the client saw a substantial reduction in their AWS bill. By consolidating the load balancers, we eliminated the redundant costs associated with maintaining multiple ALBs per region. The simplified architecture also made it easier to manage and monitor the environment, leading to operational efficiencies.

   • Conclusion

     If you're managing a microservices architecture on AWS and are looking for ways to reduce costs, consider consolidating your Application Load Balancers using host-based routing. This approach not only cuts down on unnecessary expenses but also simplifies your architecture, making it easier to maintain and scale.

     This solution worked wonders for my client, and I’m confident it can do the same for others facing similar challenges. By sharing this implementation strategy, I hope to help others optimize their AWS environments and save on their cloud bills.

Prerequisites

• An AWS account with access to EC2.

• An existing EC2 instance running a Linux-based operating system.

• SSH access to your EC2 instance using a key pair.

Step-by-Step Guide

1. Connect to Your EC2 Instance

   First, you need to connect to your EC2 instance using SSH. Use the terminal (Linux/macOS) or an SSH client like PuTTY (Windows).

    ssh -i /path/to/your-key.pem ec2-user@your-instance-public-dns
   

   Replace /path/to/your-key.pem with the path to your SSH key, and ec2-user with your instance's appropriate username (e.g., ubuntu for Ubuntu instances).

2. Switch to the Root User

Once logged in, switch to the root user to ensure you have the necessary permissions to make configuration changes.

sudo su -

3. Edit the SSH Configuration File

Open the SSH configuration file using a text editor like vi or nano.

nano /etc/ssh/sshd_config

4. Modify SSH Configuration for Password Authentication

Find the following line in the sshd_config file:

PasswordAuthentication no

Change no to yes:

PasswordAuthentication yes

Additionally, ensure that the following line is uncommented and set to yes:

ChallengeResponseAuthentication no

5. Set a Password for the User

You need to set a password for the user you wish to enable password authentication for. For example, to set a password for the ec2-user, run:

passwd ec2-user

You'll be prompted to enter and confirm a new password.

6. Restart the SSH Service

To apply the changes, restart the SSH service:

service sshd restart

7. Update Security Groups (Optional)

Ensure your EC2 instance's security group allows inbound SSH (port 22) traffic. You can do this through the AWS Management Console:

1. Navigate to EC2 Dashboard > Instances.

2. Select your instance and click on the Security tab.

3. Click on the Security Groups link.

4. Add or ensure an inbound rule exists for SSH with Source set to your preferred IP range.

8. Test Password Authentication

Disconnect from the instance and attempt to reconnect using the password:

ssh ec2-user@your-instance-public-dns

Enter the password you set earlier when prompted.

Important Security Considerations

• Security Risks: Enabling password authentication increases the risk of brute-force attacks. Consider using complex passwords and limit the source IP range for SSH access.

• Alternative Authentication: Consider using Multi-Factor Authentication (MFA) or a bastion host to improve security.

• Logging and Monitoring: Enable logging and monitoring to detect unauthorized access attempts.

Conclusion

While enabling password authentication on AWS EC2 instances is straightforward, it is crucial to understand the security implications and follow best practices to mitigate potential risks. Whenever possible, prefer using SSH key pairs for secure and efficient authentication.

By following the steps outlined in this guide, you can enable password authentication on your EC2 instances and ensure you have proper security measures.

Amazon Elastic Compute Cloud (EC2) is a powerful and flexible cloud computing service that allows users to run virtual servers in the cloud. EC2 instances are secured using key pairs, which consist of a public key that is stored on the instance and a private key that the user securely keeps. Losing access to the private key can be challenging, but fear not – Amazon Web Services (AWS) provides a straightforward process to recover from this predicament. In this blog post, we will guide you through recovering a lost EC2 key pair by creating a new one.

Step1: Launch a New Temporary Instance:

• In your AWS Management Console, create a new temporary instance.

  

  Step2: Create a New Key Pair:

  • Generate a new key pair and give it a name during the creation process.

    

    

Step3: Stop the Old Instance:

• Navigate to your old instance (blog.anupkafle.com.np) and stop it.

  

Step4: Go to your Volume:

• After the instance is stopped, go to storage, and click on the volume ID.

  

Step5: Rename the volume Volume (optional):

• Rename your volume so that we can recognize it easily.

  

Step6: Detach Old Volume:

• In the Volume, go to Actions, and detach it.

  

  

Step7: Attach Volume to Temporary Instance:

• Attach the detached volume to your new temporary instance.

  

  

Step8: Connect to Temporary Instance:

• Utilize AWS Instance Connect or SSH from your terminal to connect to the temporary instance.

  

  

Step9: Prepare for Disk Operations:

• Create a directory:

  mkdir -p /var/anupblog-disk

  

Step10: Mount Old Disk:

• Mount the old disk to the temporary instance.

    mount -o nouuid /dev/xvdf1 /var/anupblog-disk
  

  

• Use lsblk to confirm the volume attachment.

  

Step11: Copy New Public Key to Mounted Disk:

• Copy the new public key:

    cat /home/ec2-user/.ssh/authorized_keys >> /var/anupblog-disk/home/ec2-user/.ssh/authorized_keys
  

Step12: Unmount the Disk:

• Safely unmount the disk:

    umount /var/anupblog-disk
  

Step13: Detach Volume from Temporary Instance:

• In the AWS Console, detach the volume from the temporary instance.

  

Step14: Attach Volume to Old Instance:

• Attach the volume to the old instance, ensuring the device name remains the same.

  

  

Step15: Start the Old Instance:

• Start your old instance.

  

Step16: SSH into the Instance:

• Using the key created for the temporary instance, SSH into your old instance.

Conclusion:

Losing access to an EC2 instance due to a lost key pair can be a nerve-wracking experience, but AWS provides a clear and effective process for recovery. By following the step-by-step guide outlined in this blog post, you can create a new key pair, associate it with your EC2 instance, and regain control of your virtual server. Remember to maintain secure practices for managing and storing your key pairs to prevent future issues.

In the ever-evolving landscape of cloud service providers, finding a balance between user-friendliness and comprehensive support is crucial for meeting the diverse needs of users. This blog explores the experiences of users with Hostman, a cloud service provider lauded for its friendly interface, while also acknowledging feedback regarding documentation gaps and a somewhat limited service offering.

Features and Strengths:

1. User-Friendly Interface:

   • Hostman boasts a user-friendly interface, making it an ideal choice for beginners and startups entering the realm of cloud computing. The platform's simplicity facilitates easy onboarding and management of cloud resources, providing a seamless experience for users with varying levels of technical expertise.

     

2. Cloud Servers with Global Presence:

   • The platform offers dedicated computing resources through Cloud Servers, strategically located in Poland and the Netherlands. With plans to expand to additional regions, Hostman ensures reliable performance and accessibility for users across the globe.

3. Ready-Made Setups for Various Purposes:

   • Hostman provides over 25 ready-made setups with pre-installed environments and software, simplifying the setup process for different purposes. This feature allows users to deploy configurations tailored to their specific needs without extensive technical knowledge

     .

4. Instant Setup for Popular Databases:

   • The platform supports instant setup for popular database management systems, including MySQL, PostgreSQL, MongoDB, and Redis. This seamless integration enables efficient database management for applications and services.

     

5. Support for Various Development Frameworks:

   • Hostman supports various web development frameworks, such as React, Angular, Vue, Next.js, Ember, etc. Users can connect their repositories from platforms like Github, Gitlab, or Bitbucket, providing flexibility in testing and deploying projects.

     

6. "Refer a Friend" Program:

   Hostman sweetens the deal with its "Refer a Friend" program, where simplicity meets generosity. Share your unique referral link, and when your friend signs up and adds $10 or more to their account, they receive an extra $50, and you earn $100 – a win-win! There's no cap on the number of friends you can invite, so the more, the merrier. Just remember to fuel your account with a genuine $10 for those bonuses, use them across services, and note that they don't stack with other ongoing promos. Ready to amplify your Hostman experience? Start referring and unlocking rewards effortlessly!

   

   Feel free to use my referral link: https://hostman.com/r/vk99958 – a win-win opportunity awaits us both!

   Feedback and Areas for Improvement:

   1. Limited Documentation:

      • Users express concern about the lack of extensive documentation, potentially hindering their ability to troubleshoot issues independently. Clear and comprehensive documentation could empower users to address common problems without solely relying on customer support.

   2. Dependency on Support for Assistance:

      • Users note a need to contact support for assistance, indicating a potential reliance on customer support for issue resolution. Establishing a more self-service-oriented approach through enhanced documentation could reduce dependence on support for routine queries.

   3. Limited Service Offering:

      • Hostman offers a limited range of services. Expanding the service offerings could attract a broader user base with diverse requirements, positioning Hostman as a more comprehensive solution for various business needs.

Conclusion:

Hostman presents a compelling case as a user-friendly cloud service provider with strengths in simplicity, global accessibility, and ready-made setups. Addressing feedback regarding documentation and expanding service offerings can further enhance the platform's appeal, providing users with a more self-sufficient and versatile experience. Hostman's journey illustrates the delicate balance required to cater to both beginners seeking simplicity and experienced users requiring comprehensive features in the dynamic landscape of cloud services.

References:

1. Hostman Official Website. (https://www.hostman.com/)

2. User Feedback from Hostman Community Forums and Social Media Platforms.

3. Industry Analyst Reports on Cloud Service Providers.

Introduction

A Comprehensive Look at Enabling New Workflows, Enhancing Observability, and Boosting Productivity In today's data-driven world, organizations constantly seek new ways to optimize their operations, enhance productivity, and extract deeper insights from their data. Amazon Q, a generative AI-powered assistant, emerges as a game-changer, revolutionizing how we interact with and optimize our work processes.

Let's delve into scenarios where Amazon Q can significantly impact your organization.

1. Unleashing New Workflow Horizons

Amazon Q empowers users to establish entirely new workflows, transcending the limitations of traditional approaches. Its natural language processing (NLP) capabilities enable users to interact with it in plain English, asking questions and receiving immediate, actionable responses. This intuitive interface paves the way for creating novel workflows that were previously unimaginable.

A practical example: Imagine our marketing team utilizing Amazon Q to generate personalized email campaigns tailored to specific customer segments. By analyzing customer data and preferences, Amazon Q can craft email content that resonates with each individual, enhancing the campaign's effectiveness.

2.Unveiling Workflow Blind Spots

Amazon Q introduces a paradigm shift in workflow observability, giving users unprecedented visibility into their work processes. By seamlessly integrating with various AWS services, Amazon Q can monitor workflow execution, identify potential bottlenecks, and proactively alert users to anomalies. This real-time visibility enables users to optimize workflows, minimize disruptions, and maximize efficiency.

Envision this scenario: Consider a financial services organization utilizing Amazon Q to monitor its loan processing workflow. Amazon Q can identify patterns in loan applications, detect potential fraud attempts, and alert loan officers to potential issues. This proactive approach ensures timely interventions, prevents errors, and enhances customer satisfaction.

3.Unleashing Productivity Potential

Amazon Q catalyzes productivity enhancement, transforming current workflows and enabling users to achieve more quickly. By automating repetitive tasks, providing real-time insights, and facilitating seamless collaboration, Amazon Q streamlines work processes, eliminates redundancies and empowers users to focus on higher-value activities. Imagine a customer service team employing Amazon Q to handle routine inquiries and resolve common issues. Amazon Q can answer customer questions, provide step-by-step troubleshooting guides, and even escalate complex issues to the appropriate personnel. This automation frees customer service representatives to focus on more complex cases, improving customer satisfaction and overall team productivity.

Conclusion: Embracing the Future of Work

Amazon Q stands at the forefront of AI-powered workflow automation, offering a transformative solution for organizations seeking to enhance efficiency, gain deeper observability, and boost productivity. By enabling new workflows, providing real-time insights, and streamlining repetitive tasks, Amazon Q empowers organizations to achieve unprecedented operational excellence. As AI continues to evolve, Amazon Q is poised to revolutionize how we work, paving the way for a future of unparalleled productivity and innovation. Amazon Q represents a significant step in our journey towards a more efficient, data-driven, and human-centered work environment. By harnessing the power of AI, Amazon Q is empowering organizations to break free from traditional limitations and embrace a new era of work optimization and productivity.

References

1. https://aws.amazon.com/q/

2. https://aws.amazon.com/blogs/aws/introducing-amazon-q-a-new-generative-ai-powered-assistant-preview/

3. https://aws.amazon.com/blogs/aws/amazon-q-brings-generative-ai-powered-assistance-to-it-pros-and-developers-preview/

Amazon S3 (Simple Storage Service) is a scalable, secure, and highly durable object storage service that Amazon Web Services (AWS) provides. It allows you to store and retrieve large amounts of data efficiently. One feature that makes Amazon S3 powerful is multipart upload, particularly useful when dealing with large files or when network interruptions or errors are common. This blog will explore multipart upload, its advantages, and how to implement it on Amazon S3.

What is Multipart Upload?

Multipart upload is a feature of Amazon S3 that enables the efficient and reliable uploading of large objects by breaking them into smaller parts. These smaller parts are uploaded in parallel and assembled to create the final thing. This approach offers several benefits:

1. Resumable Uploads: If an error occurs during the upload process, you can retry uploading only the failed parts instead of the entire file. This is crucial for large files and unreliable network connections.

2. Improved Throughput: Multipart uploads can significantly improve upload speeds as you can upload multiple parts simultaneously.

3. Optimal Memory Usage: Uploading large files as a single object might consume a lot of memory. Multipart uploads allow you to work with smaller parts, reducing the memory footprint.

4. Metadata Updates: You can set object metadata on individual parts, especially for custom metadata or permissions.

When to Use Multipart Upload?

Multipart upload is particularly useful in the following scenarios:

1. Large Files: When uploading files larger than 100 MB, Amazon S3 recommends multipart upload.

2. Unreliable Network ConnUnreliable Network Connections:ections: In situations where network interruptions are common, multipart upload minimizes the risk of data loss and allows you to retry uploading specific parts.

3. Streaming Data: For applications that require streaming data directly to Amazon S3, multipart upload can be a more efficient choice.

4. Custom Metadata: If you need to set specific metadata or access control settings for different parts of an object, multipart upload provides this flexibility.

Best Practices for Multipart Upload

Here are some best practices to follow when implementing multipart upload on Amazon S3:

1. Part Size: Choose an optimal part size based on your use case. Smaller parts are better for unreliable connections, while larger parts can improve upload efficiency.

2. Error Handling: Implement robust error handling, as failures can occur during any part of the multipart upload process.

3. Optimal Concurrency: Consider the number of parts to upload in parallel depending on your available resources. AWS SDKs often provide concurrency options to fine-tune this.

4. Monitoring and Logging: Use AWS CloudWatch and Amazon S3 access logs to monitor and log your multipart uploads for tracking and troubleshooting.

5. Lifecycle Policies: Implement lifecycle policies to manage incomplete multipart uploads, ensuring that you don't leave unfinished uploads consuming storage

How to Implement Multipart Upload on Amazon S3

Let's go through the steps to implement a multipart upload on Amazon S3:

1. Split the Video File: Split the video file using the split command into smaller parts.

   

1. Create an S3 Bucket: If you haven't already, create an S3 bucket where you want to upload the video parts. Replace multipartupload-demo-oct with your desired bucket name. Remember that S3 bucket names must be globally unique.

    aws s3api create-bucket --bucket multipartupload-demo-oct
   

2. Initiate a Multipart Upload: Initiate the multipart upload for the video file. You will receive an upload-id, which you will need for subsequent steps.

    aws s3api create-multipart-upload --bucket your-unique-bucket-name --key yourfilename
   

   

   This command will return an upload-id like nVQBtP3g0teF_D5JuOz16emWZAhS97y8wyuQx6Q0GRNSn7Ogaz6hkCyphgx4lhinf_2iAX5iWUwQeZo8JHLSBA--

   1. Upload the Parts: Now, you'll upload the two video parts, xaa and xab, as individual parts of Amazon S3. You must specify the part number and provide the UploadId obtained in the previous step.

      (I) Upload the first part (xaa):

      This will return an ETag for the uploaded part, something like."9d15c7757230b2f88a47906bdc254c07"

    aws s3api upload-part --bucket DOC-EXAMPLE-BUCKET --key large_test_file --part-number 1 --body large_test_file.001 --upload-id

(II) Upload the second part (xab):

This will also return an ETag for the second part, something like "b104d08a3208c1b16ce4dbccca9d8d34".

aws s3api upload-part --bucket your-unique-bucket-name --key videoplayback.mp4 --part-number 2 --body xab --upload-id nVQBtP3g0teF_D5JuOz16emWZAhS97y8wyuQx6Q0GRNSn7Ogaz6hkCyphgx4lhinf_2iAX5iWUwQeZo8JHLSBA--

1. List Etag: you can use the aws s3api list-parts command to list the ETags for the multipart upload parts. Here's the command you can use:

    aws s3api list-parts --bucket your-unique-bucket-name --key videoplayback.mp4 --upload-id nVQBtP3g0teF_D5JuOz16emWZAhS97y8wyuQx6Q0GRNSn7Ogaz6hkCyphgx4lhinf_2iAX5iWUwQeZo8JHLSBA--
   

   

2. Create a JSON file and keep the details in the JSON file

   

1. Complete the Multipart Upload: After uploading all parts, you must request Amazon S3 to complete the multipart upload. Use the upload-id and a list of ETags (ETags from the two parts) to indicate which parts belong to this upload.

    aws s3api complete-multipart-upload --multipart-upload file://multipart.json --bucket multipartupload-demo-oct --key videoplayback.mp4 --upload-id nVQBtP3g0teF_D5JuOz16emWZAhS97y8wyuQx6Q0GRNSn7Ogaz6hkCyphgx4lhinf_2iAX5iWUwQeZo8JHLSBA--
   

   This command will complete the upload, and your video file, videoplayback.mp4, will be available in your S3 bucket.

   

That's it! You've successfully performed a multipart upload for your video file using the AWS CLI. This approach ensures the efficient and reliable transfer of extensive data to Amazon S3, even when network interruptions occur.

Conclusion:

Multipart upload on Amazon S3 is valuable for anyone working with large files, ensuring a reliable, efficient, and flexible data transfer process. By understanding the concept and following best practices, you can harness the full power of multipart upload, ensuring the secure storage of your data in the cloud, even under challenging circumstances.

Prerequisites

Before we dive into the deployment process, make sure you have the following prerequisites in place:

1. AWS Account: You need an AWS account to access AWS services.

2. Terraform Installed: Download and install Terraform from the official website.

3. AWS CLI: Install the AWS Command Line Interface (CLI) and configure it with your AWS credentials. You can install it from here.

4. Static Website Files: Prepare the static website files (HTML, CSS, JavaScript, images, etc.) you want to deploy. Place them in a directory.

Steps to Deploy a Static Website in AWS S3 Using Terraform

Let's start by setting up your Terraform configuration. Create a file named main.tf and add the following code:

Step 1: AWS Provider Configuration

provider "aws" {
  region     = "us-east-1" # Change this to your desired region
  access_key = "your key" # Add your AWS access key ID here
  secret_key = "your secret key" # Add your AWS secret access key here
}

This block configures the AWS provider, specifying the AWS region and your access key and secret key. It tells Terraform which cloud provider to interact with and how to authenticate.

Step 2: S3 Bucket Resource

resource "aws_s3_bucket" "my-static-website" {
  bucket = "mydemostaticwebsite-2324413" # give a unique bucket name
  tags = {
    Name = "my-demo-static-website"
  }
}

Here, you define an S3 bucket resource using the aws_s3_bucket resource type. It creates an S3 bucket with the specified name and sets tags to provide metadata for the bucket.

Step 3: S3 Bucket Website Configuration

resource "aws_s3_bucket_website_configuration" "my-static-website" {
  bucket = aws_s3_bucket.my-static-website.id

  index_document {
    suffix = "index.html"
  }

  error_document {
    key = "error.html"
  }
}

This block configures the S3 bucket to act as a static website. It specifies the index document (the default file to load when accessing the website's root) and the error document (the page to display 404 errors).

Step 4: S3 Bucket Ownership Controls

resource "aws_s3_bucket_ownership_controls" "my-static-website" {
  bucket = aws_s3_bucket.my-static-website.id
  rule {
    object_ownership = "BucketOwnerPreferred"
  }
}

This block sets up ownership controls for the S3 bucket, ensuring that the bucket owner is preferred for objects within the bucket.

Step 5: S3 Bucket Public Access Block

resource "aws_s3_bucket_public_access_block" "my-static-website" {
  bucket = aws_s3_bucket.my-static-website.id

  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}

Here, you configure the public access block settings for the S3 bucket. In this example, public access is allowed for various locations. You can adjust these settings based on your security requirements.

Step 6: S3 Bucket ACL

resource "aws_s3_bucket_acl" "my-static-website" {
  depends_on = [
    aws_s3_bucket_ownership_controls.my-static-website,
    aws_s3_bucket_public_access_block.my-static-website,
  ]

  bucket = aws_s3_bucket.my-static-website.id
  acl    = "public-read"
}

This block sets the bucket access control list (ACL) for the S3 bucket, allowing public read access. It depends on the previous two blocks to ensure proper ownership controls and public access settings.

Step 7: Create an index.html and upload the index.html File to S3 Bucket

Keep index.html in the same directory where main.tf is located.

resource "aws_s3_object" "index_html" {
  bucket       = aws_s3_bucket.my-static-website.id
  key          = "index.html"  # The name you want for the file in the S3 bucket
  source       = "index.html"  # The path to your local index.html file
  content_type = "text/html"

  # Make the object publicly accessible
  acl = "public-read"
}

This block uploads the index.html file from your local directory to the S3 bucket. It also sets the content type and makes the object publicly accessible.

Step 8: S3 Static Website URL Output

output "website_url" {
  value = "http://${aws_s3_bucket.my-static-website.bucket}.s3-website.us-east-1.amazonaws.com"
}

This block defines an output variable that displays the URL of your static website after Terraform applies the configuration. The URL is constructed using the bucket name and the S3 website endpoint.

Step 9: S3 Bucket Policy

resource "aws_s3_bucket_policy" "bucket-policy" {
  bucket = aws_s3_bucket.my-static-website.id

  policy = <<POLICY
{
  "Id": "Policy",
  "Statement": [
    {
      "Action": [
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::mydemostaticwebsite-2324413/*",
        "arn:aws:s3:::mydemostaticwebsite-2324413"
        ],
      "Principal": "*"
    }
  ]
}
POLICY
}

This block defines an S3 bucket policy that allows various actions on objects within the bucket and sets the principal to "*" (anyone). This policy ensures public access to the bucket's contents.

Step 10: Overall Code

The overall above code looks like this.

# main.tf

# Configure the AWS provider with your credentials and desired region
provider "aws" {
  region     = "us-east-1" # Change this to your desired region
  access_key = "your key"  # Add your AWS access key ID
  secret_key = "your secret key"  # Add your AWS secret access key here
}

# Create an S3 bucket and website configuration
resource "aws_s3_bucket" "my-static-website" {
  bucket = "mydemostaticwebsite-2324413" # give a unique bucket name
  tags = {
    Name = "my-demo-static-website"
  }
}

resource "aws_s3_bucket_website_configuration" "my-static-website" {
  bucket = aws_s3_bucket.my-static-website.id

  index_document {
    suffix = "index.html"
  }

  error_document {
    key = "error.html"
  }
}

resource "aws_s3_bucket_ownership_controls" "my-static-website" {
  bucket = aws_s3_bucket.my-static-website.id
  rule {
    object_ownership = "BucketOwnerPreferred"
  }
}

resource "aws_s3_bucket_public_access_block" "my-static-website" {
  bucket = aws_s3_bucket.my-static-website.id

  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}

resource "aws_s3_bucket_acl" "my-static-website" {
  depends_on = [
    aws_s3_bucket_ownership_controls.my-static-website,
    aws_s3_bucket_public_access_block.my-static-website,
  ]

  bucket = aws_s3_bucket.my-static-website.id
  acl    = "public-read"
}

# Upload index.html from the current directory to the S3 bucket and make it public
resource "aws_s3_object" "index_html" {
  bucket       = aws_s3_bucket.my-static-website.id
  key          = "index.html"  # The name you want for the file in the S3 bucket
  source       = "index.html"  # The path to your local index.html file
  content_type = "text/html"

  # Make the object publicly accessible
  acl = "public-read"
}

# S3 static website URL
output "website_url" {
  value = "http://${aws_s3_bucket.my-static-website.bucket}.s3-website.us-east-1.amazonaws.com"
}

# S3 bucket policy
resource "aws_s3_bucket_policy" "bucket-policy" {
  bucket = aws_s3_bucket.my-static-website.id

  policy = <<POLICY
{
  "Id": "Policy",
  "Statement": [
    {
      "Action": [
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::mydemostaticwebsite-2324413/*",
        "arn:aws:s3:::mydemostaticwebsite-2324413"
        ],
      "Principal": "*"
    }
  ]
}
POLICY
}

Step 11: Deploying Your Static Website

1. Open your terminal and navigate to the directory containing your main.tf file.

2. Run the following Terraform commands:

    terraform init
   

   terraform init primarily initializes the AWS provider, installs the required provider plugin, validates the configuration files, sets up a local backend for state management, and prepares the project directory for further Terraform commands like terraform plan and terraform apply.

   

    terraform apply --auto-approve
   

   In the provided Terraform code, running terraform apply --auto-approve initiates applying infrastructure changes defined in the configuration without requiring manual confirmation. It automates the approval of changes, executes the planned actions, and provides the output, including the URL of the static website, as specified in the configuration. Use this flag cautiously and ensure you've reviewed your configuration carefully before applying changes, especially in production environments.

    Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
   
    Outputs:
   
    website_url = "http://mydemostaticwebsite-2324413.s3-website.us-east-1.amazonaws.com"
   

   Once the deployment is complete, Terraform will display the URL of your static website. You can access your website using this URL.

   

   On clicking the website_url, you can access your webpage.

   Conclusion

   In this blog post, you've learned how to create a static website on Amazon S3 using Terraform. By automating the infrastructure setup with Terraform, you can quickly deploy and manage your static websites on AWS. This approach provides a cost-effective and scalable solution for hosting your web content. Start building and deploying your static websites on AWS today!

Benefits of Amazon Cognito:

• It's easy to use: Amazon Cognito provides a simple API that makes it easy to add user authentication and access control to your apps.

• It's secure: Amazon Cognito uses industry-standard security practices to protect your users' data.

• It's scalable: Amazon Cognito can handle millions of users and is designed to scale with your app. Support for multiple identity providers

• User pools: Amazon Cognito user pools are a great way to manage user accounts for your app. User pools provide a central place to store user data, such as passwords, email addresses, and phone numbers.

• Federated identities: Amazon Cognito federated identities allow your app to access AWS resources without the user having to sign in to your app.

  Let’s dive into the Demonstration :

Step 1: Go to the AWS Management Console Search for Cognito And Click on it.

Step 2: Click on Create user pool.

Step 3: Under the Authentication Providers:

• Choose Cognito User pool

• Select Email from Cognito user pool sign-in options.

• Click on Next.

  

Step 4: From the Password policy, select the Cognito defaults. You can also create your custom password policy.

Step 5: Make sure that you Enable self-service account recovery.

• From the Delivery method for user account recovery messages, choose Email

• Click on Next.

Step 6: From the Multi-factor authentication

• choose No MFA

• select on Enable self-service account recovery

• Select email only from Delivery method for user account recovery messages.

  

Step 7: Click on Next

Step 8: Under Configure Sign-up experience, Enable self-registration

Step 9: From the Configure message delivery

• Select send email with cognito

• Click on Next

  

Step 10: Under Integrate your app

• Give a name to the user pool, For now we say it “demo-app-user-pool”

• Select the Use the Cognito hosted UI

• From Domain, Select the Use a cognito domain and enter the domain prefix, let's say https://mywebsite

  

Step 11: Under the Initial app client,

• choose Public client from App type

• Give it a app client name let’s say “DemoApp”

  

Step 12: In the Client Secret we must provide the Callback URls so, For now

https://localhost:800/logged_in.html

Note: We will create a file name logged_in.html that’s why we are using it at localhost. When we enter our credentials we will be on this logged_in.html page.

Step 13: Click on Next

Step 14: From the Review and Create go to the end and Click on Create user pool.

Step15: You will see the User pool named “Demo-app-user-pool” Click on it.

• Select the App integration

  

Step 16: Scroll down at the end you will see App Client list, and there you’ll find App client name “DemoApp”

• Click on it.

  

Step 17: Under the “DemoApp” you will see the “Hosted UI” and on the right side you’ll find “View Hosted UI”, Click on it.

You will redirect to the web browser. Now copy the Link of this login form page.

Step 18: Open the VS code and make a folder and inside it create two files named index.html and logged_in.html Paste the following code in the index.html file

<body>
   <h3>Welcome to my Website</h3>
   <a href="#">Register|Login </a>
</body>
</html

• On the href =”< paste the link copied from the login page form>”

  

• In the logged_in.html Paste the following code

    <html>
    <body>
       <h1>Congratulations!!</h1>  
       <p>You are logged in...</p>
    </body>
    </html>
  

Step 19: Now open the terminal and type the following command.

• python3 -m http.server

• Now, click on the (http://)

  

• You will be redirected to this page.

  

• Click on the Register|Login and you will be redirected to the login and sign up page.

• Since you are trying to access it for the first time you must Click on Sign up and you will receive a confirmation code in your mail and verify it.

  

Step 19:After you verify you can navigate back to the “Demo-app-user-pool” and under “Users” you can see the user with status confirmed.

Step 20: when you enter your credentials you will redirect to the Logged_in.html page.

Finally, You have successfully completed this demonstration. Now, you will be able to connect your website with Aws Cognito to add user sign-up, sign-in, and access control to your web and allow only authenticated users.

Challenge #1: Service Discovery

In a microservices architecture, services need to communicate with each other, but as the number of services grows, it becomes challenging to manage and discover them. Service discovery is crucial for scaling microservices, and AWS solves this problem with Amazon Elastic Container Service (ECS). ECS is a container orchestration service that allows developers to manage, deploy, and scale Docker containers. ECS includes Amazon ECS Service Discovery, which allows services to discover and communicate with each other easily. ECS Service Discovery automatically registers new services and updates DNS records, making it easy to scale microservices without worrying about the underlying infrastructure.

Challenge #2: Load Balancing

Load balancing is another critical challenge when scaling microservices. AWS offers Elastic Load Balancing (ELB), a managed load balancing service that automatically distributes incoming application traffic across multiple targets, such as EC2 instances, containers, and IP addresses. ELB includes Application Load Balancer (ALB), which allows developers to route traffic based on content, URL, or IP address. ALB also provides features like SSL termination, health checks, and sticky sessions, making it easy to scale microservices without worrying about load balancing

.

Challenge #3: Auto Scaling

Auto Scaling is a critical challenge when scaling microservices. AWS offers Auto Scaling, a service that automatically adjusts the number of EC2 instances or containers in response to changes in demand. Auto Scaling allows developers to set minimum and maximum thresholds for the number of instances or containers. It can automatically scale up or down based on CPU utilization, network traffic, or application metrics.

.

Challenge #4: Monitoring and Logging

Monitoring and logging are essential for scaling microservices. AWS provides CloudWatch, a monitoring and logging service that allows developers to collect and track metrics, collect and monitor log files, and set alarms. CloudWatch provides detailed insights into application performance and helps developers diagnose and troubleshoot issues quickly. With CloudWatch, developers can confidently scale microservices, knowing they can monitor and log applications at scale.

Challenge #5: Security

Security is another critical challenge when scaling microservices. AWS provides several security services, including Amazon Virtual Private Cloud (VPC), which allows developers to create a virtual network isolated from the internet and other AWS resources. VPC allows developers to set up security groups and network access control lists (ACLs) to control inbound and outbound traffic to and from their microservices. AWS also offers AWS Identity and Access Management (IAM), which provides granular control over who can access AWS resources and what actions they can perform.

Conclusion

Scaling microservices can be challenging, but with the tools and services provided by AWS, it can be made much more manageable. This blog post discussed some challenges when scaling microservices and how AWS can help overcome them. By leveraging AWS services such as ECS, ELB, Auto Scaling, CloudWatch, and VPC, we developers can confidently scale microservices, knowing they have the tools they need to ensure their applications perform reliably and securely.

AWS Amplify is a set of tools and services for building and deploying cloud-powered applications. It includes a library of pre-built UI components, a command-line interface for building and deploying your applications, and a backend platform for hosting and scaling your app. AWS Amplify makes it easy to build and deploy applications quickly, with a focus on improving the developer experience.

Pricing

AWS Amplify is free to use, and many of the services and features it provides are also free. For example, the AWS Amplify CLI is free to use, as are the pre-built UI components and the backend platform.

However, some of the services and features provided by AWS Amplify do have costs associated with them. For example, if you use the storage or database features of AWS Amplify, you may be charged for the use of underlying AWS services like Amazon S3 or Amazon DynamoDB. Similarly, if you use the authentication or authorization features of AWS Amplify, you may be charged for the use of underlying AWS services like Amazon Cognito.

Steps for Hosting

• Sign in to the AWS Management Console and navigate to the Amplify service.

  

• Click the "Get started" button to start a new Amplify project.

  

• Build your static website using your preferred tools and frameworks.

• Upload your static website files to the Amplify hosting environment by Connecting the Git repository or to your source code or Upload the files to deploy your website.

  

• Authorize AWS Amplify to access to your Git provider account

  

• Choose the repository and branch where your code resides.

  

• Give the App name then save and deploy.

  

• Allow some time for the website to be deployed.

  

• Click on the link shown and browse your static website

  

Conclusion

That's it! You should now have a static website hosted on AWS Amplify. If you need more detailed instructions or run into any issues, you can refer to the AWS Amplify documentation for more information.

Amazon WorkMail is a secure, managed business email and calendar service with support for existing desktop and mobile email client applications. Users can access their email, contacts, and calendars using the client application of their choice, including Microsoft Outlook or any client application supporting the IMAP protocol.

Pricing

Amazon WorkMail costs $4.00 per user per month and includes 50 GB of mailbox storage for each user. You can get started with a 30-day free trial for up to 25 users.

Note: If a user is created after the first of a month, then the monthly fee for that mailbox will be adjusted on a pro-rata basis from the first day it was active to the end of that month. If a user is terminated or deleted before the end of a month, then the monthly fee for that user will still apply through the end of the month.

Let's get started..

step1: Login to your AWS account and search for WorkMail

Step2: Click on Create organization

Note: There are several options, but I am using my domain parked on Route53 . Don't worry, you can later add the external domain as well if you don't have domain now.

• Select an Existing Route 53 domain

• Choose Route53 hosted zone

• Setup alias

• Create organization

  

• You will see the dashboard like below:

Step3: Adding custom domain

• Click on your organization name

You will see a dashboard like below.

• Click on Domains on the dashboard left bar .

  Note: I have already added my domain shoeasy.me but you will see domain like ailas.awsapps.com

  

• Click on Add domain (optional if you don't want custom domain email)

  

• Choose the domain and click Add domain

  

• Click on Update all records in route 53

  This will add all the DNS records required for WorkMail automatically. After records are added you can set your domain as default domain .

  

Step4: Setup Mobile policy (optional)

Here you can set password policies and other mobile policies. I kept all of the defaults, but you can change them to suit your needs.

Step5: Creating users

• Click Users from the left menu of your organization dashboard. You see a page like the below

  

• Click on Create user and create a user by filling in the required details.

  

• On email address choose your custom domain (If you havenot added custom domain go with domain provided by aws) , choose password and create user

You have sucessfully created a user . You can add more user according to your needs.

Step6:User login

• You will find a login link on your organization dashboard (say https://shoeasymail.awsapps.com/mail) go to the address.

  

• Use the credentials you used before while creating mail to sign in

  

• You are all done you can receive and send mails now

  

Enjoy !!!!