创建cloud 9环境

创建Cloud 9

在这一节,我们将在AWS帐号下配置Cloud9 IDE,用于后续实验连接到MSK并执行操作。

在AWS控制台上,搜索Cloud9服务:

image-20210430170852514

在Cloud9控制台,点击Create environment:

image-20210430172234510

输入环境的名称和描述:

image-20211220214258391

点击进入下一步。在Configure settings步骤, 机型型号选择m5.large,然后点击Next step:

image-20220103233720169

点击Create Environment创建Cloud9, 整个创建过程会持续大概2-3分钟。

环境配置

打开Cloud9的一个新terminal页面:

image-20211220214709264

默认cloud9已经安装好java11和python3,但还需要安装其他的一些工具:

sudo yum install maven nc jq -y

下载并安装kafka:

sudo su
wget https://archive.apache.org/dist/kafka/3.0.0/kafka_2.13-3.0.0.tgz
tar -xzf kafka_2.13-3.0.0.tgz
mv kafka_2.13-3.0.0 /usr/local/
# 把.sh给去掉
for i in /usr/local/kafka_2.13-3.0.0/bin/*.sh; do
	mv $i ${i%???}
done;
ln -sfn /usr/local/kafka_2.13-3.0.0 /usr/local/kafka
exit
echo 'export PATH=/usr/local/kafka/bin:$PATH' >> /home/ec2-user/.bashrc

source ~/.bashrc

sudo su
cp /usr/local/kafka_2.13-3.0.0/bin/kafka-run-class /usr/local/kafka_2.13-3.0.0/bin/kafka-run-class.sh 
exit

现在可以在命令行执行kafka的各种命令:

image-20211222224041285

调整Cloud 9 磁盘大小

默认Cloud 9的EBS卷是10GB,后面的实验要下载很多docker镜像,很容易占满。使用脚本将其扩容到50GB。

将以下内容保存为ebs-resize.sh

#!/bin/bash

# Specify the desired volume size in GiB as a command-line argument. If not specified, default to 50 GiB.
SIZE=${1:-50}

# Get the ID of the environment host Amazon EC2 instance.
INSTANCEID=$(curl http://169.254.169.254/latest/meta-data//instance-id)

# Get the ID of the Amazon EBS volume associated with the instance.
VOLUMEID=$(aws ec2 describe-instances \
  --instance-id $INSTANCEID \
  --query "Reservations[0].Instances[0].BlockDeviceMappings[0].Ebs.VolumeId" \
  --output text)

# Resize the EBS volume.
aws ec2 modify-volume --volume-id $VOLUMEID --size $SIZE

# Wait for the resize to finish.
while [ \
  "$(aws ec2 describe-volumes-modifications \
    --volume-id $VOLUMEID \
    --filters Name=modification-state,Values="optimizing","completed" \
    --query "length(VolumesModifications)"\
    --output text)" != "1" ]; do
sleep 1
done

if [ $(readlink -f /dev/xvda) = "/dev/xvda" ]
then
  # Rewrite the partition table so that the partition takes up all the space that it can.
  sudo growpart /dev/xvda 1

  # Expand the size of the file system.
  sudo resize2fs /dev/xvda1

else
  # Rewrite the partition table so that the partition takes up all the space that it can.
  sudo growpart /dev/nvme0n1 1

  # Expand the size of the file system.
  sudo xfs_growfs -d /
fi

执行:

chmod +x ebs-resize.sh
./ebs-resize.sh

执行完成后,查看磁盘空间进行确认:

image-20220104000248837

放通MSK安全组

我们的cloud 9环境在后面的实验中会访问MSK集群并执行命令,所以应该让MSK集群的安全组放行cloud 9所在的安全组:

image-20211223093102748

选择对应的安全组并保存