在这一节,我们将在AWS帐号下配置Cloud9 IDE,用于后续实验连接到MSK并执行操作。
在AWS控制台上,搜索Cloud9服务:
在Cloud9控制台,点击Create environment:
输入环境的名称和描述:
点击进入下一步。在Configure settings
步骤, 机型型号选择m5.large
,然后点击Next step:
点击Create Environment
创建Cloud9, 整个创建过程会持续大概2-3分钟。
打开Cloud9的一个新terminal页面:
默认cloud9已经安装好java11和python3,但还需要安装其他的一些工具:
sudo yum install maven nc jq -y
下载并安装kafka:
sudo su
wget https://archive.apache.org/dist/kafka/3.0.0/kafka_2.13-3.0.0.tgz
tar -xzf kafka_2.13-3.0.0.tgz
mv kafka_2.13-3.0.0 /usr/local/
# 把.sh给去掉
for i in /usr/local/kafka_2.13-3.0.0/bin/*.sh; do
mv $i ${i%???}
done;
ln -sfn /usr/local/kafka_2.13-3.0.0 /usr/local/kafka
exit
echo 'export PATH=/usr/local/kafka/bin:$PATH' >> /home/ec2-user/.bashrc
source ~/.bashrc
sudo su
cp /usr/local/kafka_2.13-3.0.0/bin/kafka-run-class /usr/local/kafka_2.13-3.0.0/bin/kafka-run-class.sh
exit
现在可以在命令行执行kafka的各种命令:
默认Cloud 9的EBS卷是10GB,后面的实验要下载很多docker镜像,很容易占满。使用脚本将其扩容到50GB。
将以下内容保存为ebs-resize.sh
#!/bin/bash
# Specify the desired volume size in GiB as a command-line argument. If not specified, default to 50 GiB.
SIZE=${1:-50}
# Get the ID of the environment host Amazon EC2 instance.
INSTANCEID=$(curl http://169.254.169.254/latest/meta-data//instance-id)
# Get the ID of the Amazon EBS volume associated with the instance.
VOLUMEID=$(aws ec2 describe-instances \
--instance-id $INSTANCEID \
--query "Reservations[0].Instances[0].BlockDeviceMappings[0].Ebs.VolumeId" \
--output text)
# Resize the EBS volume.
aws ec2 modify-volume --volume-id $VOLUMEID --size $SIZE
# Wait for the resize to finish.
while [ \
"$(aws ec2 describe-volumes-modifications \
--volume-id $VOLUMEID \
--filters Name=modification-state,Values="optimizing","completed" \
--query "length(VolumesModifications)"\
--output text)" != "1" ]; do
sleep 1
done
if [ $(readlink -f /dev/xvda) = "/dev/xvda" ]
then
# Rewrite the partition table so that the partition takes up all the space that it can.
sudo growpart /dev/xvda 1
# Expand the size of the file system.
sudo resize2fs /dev/xvda1
else
# Rewrite the partition table so that the partition takes up all the space that it can.
sudo growpart /dev/nvme0n1 1
# Expand the size of the file system.
sudo xfs_growfs -d /
fi
执行:
chmod +x ebs-resize.sh
./ebs-resize.sh
执行完成后,查看磁盘空间进行确认:
我们的cloud 9环境在后面的实验中会访问MSK集群并执行命令,所以应该让MSK集群的安全组放行cloud 9所在的安全组:
选择对应的安全组并保存