Docker under LXD Linux Containers

I am now able to bring up zookeeper and kafka in my 5 docker nodes. Here is a zip file with the below bash scripts

These scripts all run on Ubuntu 16.04 LTS, with lxd init already completed, and the lxdbr0 network IP set to 192.168.192.1/24. Launch.sh creates 5 LXD containers, each running an instance of docker.

#!/bin/bash
dockercount=5
image="ubuntu-daily:16.04"
network="192.168.192."
function logcmd() {
echo "`date` running: $*"
$*
}
# launch containers if they don't exist
for id in `seq 1 $dockercount`
do
dhost=docker${id}
status=`lxc list -c ns $dhost | grep docker | cut -d' ' -f4`
if [ "$status" == "RUNNING" ]; then
echo "$dhost $status"
elif [ "$status" == "STOPPED" ]; then
echo lxc start $dhost
lxc start $dhost &
elif [ "$status" == "" ]; then
echo lxc launch $image $dhost -p default -p docker
lxc launch $image $dhost -p default -p docker &
fi
done
echo "waiting until all $dockercount nodes are up with an address on network $network"
count=0
while true
do
sleep 5
count=`lxc list -c 4 | grep $network | cut -d' ' -f2 | wc -l`
echo "$count docker nodes up"
test $count -lt $dockercount || break
done
echo "installing software"
cat <setup.sh
exec >/root/setup.log 2>&1
#!/bin/bash
(
set -x
apt-get update
apt-get dist-upgrade -y
apt-get install -y docker.io docker-compose
) 2>&1 | tee /root/setup.log
EOS
dhostlist=`lxc list -c n docker | grep docker | cut -d' ' -f 2`
for dhost in $dhostlist
do
lxc file push setup.sh $dhost/root/setup.sh
lxc exec $dhost bash /root/setup.sh &
done
wait
# jain them all into a swarm
swarmcount=0
for dhost in $dhostlist
do
if [ $swarmcount -eq 0 ]
then
command="lxc exec $dhost -- docker swarm init"
echo "$command"
output="`$command`"
token="`echo $output | sed -e 's/^.*--token //' -e 's/ .*$//'`"
swarmmaster="`echo $output | sed -e 's/^.*${network}/${network}/' -e 's/ .*$//'`"
swarmcount=$(( $swarmcount + 1 ))
else
command="lxc exec $dhost -- docker swarm join --token $token $swarmmaster"
echo "$command"
output="`$command`"
swarmcount=$(( $swarmcount + 1 ))
fi
done
echo "$swarmcount nodes in the swarm"

zookeeper.sh launches 3 zookeeper containers, under docker1, docker2 and docker3.

#!/bin/bash
# your LXD network number
network="192.168.192"
image="jeygeethan/zookeeper-cluster"
function main_routine() {
cd $HOME || exit 1
servers=""
for id in `seq 1 3`
do
ipaddr=`lxc list -c 4 docker$id | grep $network | cut -d' ' -f2`
servers="${servers} server.${id}=${ipaddr}:2888:3888"
done
servers="`echo $servers | sed -e 's/^ *//'`"
echo "servers=\"$servers\""
start_script=start-zookeeper.sh
for id in `seq 1 3`
do
echo "launching node $id"
cat > $start_script << EOF
#!/bin/bash
test -d /var/log/zookeeper || mkdir -p /var/log/zookeeper
set -x
docker run --name zookeeper --detach --restart=always \
-p 2181:2181 -p 2888:2888 -p 3888:3888 \
-e ZOO_MY_ID=$id \
-e ZOO_SERVERS="$servers" \
-v /var/log/zookeeper:/var/log/zookeeper \
$image
EOF
lxc file push $start_script docker$id/root/$start_script
lxc exec docker$id -- bash /root/$start_script &
done
wait
rm -f $start_script
}

main_routine

kafka.sh launches five distinct kafka 0.9 containers, one under each docker host.

#!/bin/bash
# your LXD network number
network="192.168.192"
image="jeygeethan/kafka-cluster"
function main_routine() {
cd $HOME || exit 1
# list of LAN IP addresses of docker nodes, append port 2181
tmplist=`lxc list -c 4 docker | grep ${network} | cut -d' ' -f2 | sed -e "s/$/:2181/" | head -3`
# make it a comma separated list
zoolist=`echo $tmplist | sed -e "s/ /,/g" -e "s/^,*//" -e "s/,*$//"`
start_script=start-kafka.sh
for id in `seq 1 5`
do
cat > $start_script << EndOfScript
#!/bin/bash
test -d /var/lib/kafka || mkdir -p /var/lib/kafka
test -d /var/log/kafka || mkdir -p /var/log/kafka
set -x
docker run --detach --name kafka --restart=always \
-p 9092:9092 \
-e KAFKA_ID=$id \
-e KAFKA_HOST=docker$id \
-e KAFKA_PORT=9092 \
-e ZOOKEEPER_CONNECT="$zoolist" \
-v /var/lib/kafka:/var/lib/kafka \
-v /var/log/kafka:/var/log/kafka \
$image
EndOfScript
lxc file push $start_script docker$id/root/$start_script
lxc exec docker$id -- bash /root/$start_script
done
rm -f $start_script
}

main_routine

The d script lets you just type “./d docker -ps -a” and see everything that is running

#!/bin/bash
command="$*"
dhlist=`lxc list -c n docker | grep docker | cut -d' ' -f2`
for dh in $dhlist
do
echo lxc exec $dh -- $command
lxc exec $dh -- $command
done