Docker – just don’t quite get the whole ecosystem yet

I get all the technology. I have plenty of experience with Linux Containers, I get all the tools, and how to get them to work. My only confusion seems to be how is an engineer supposed to learn how to build an application, beyond starting from alpine or centos or archos and installing and configuring packages? Is there a modern cloud bible that I’m missing?

For example, what is the benefit of all those docker repositories, anyway? For example, I see tons of openvpn containers out there. How am I supposed to review them, choose one to fit my needs? Kitematic shows “stars”, but it’s rare in a category of more than 10 items that more than 4 have any stars at all. Am I supposed to download all 75 of them, try each one out, figure out what the benefits of each are, what configuration/management/logging differences are setup for each one, or what? This is pure chaos, if you ask me. Docker is a toolkit, not a functioning application ecosystem, not yet anyway.

I also fail to understand how to setup persistent storage for a service. Let’s say I want to run a DHCP service. Ton’s of prebuilt dhcp containers out there I can start with, or I can build my own. How do I keep my enterprise data stored, backed up, live somewhere? If the container host goes down and I re-launch my container elsewhere, all the data learned since the container was built is lost.
Kafka requires local storage for all the objects it’s maintaining. Splunk indexers need disk space to store the data they’re indexing. Mysql needs disks to store the data. It’s simple to setup a container that has some disk space, but how do you persist past any once instance. There must be a chapter in a book I’ve not read yet. It would make more sense if I could figure out how to mount iSCSI or CEPH volumes, or anything other than simple NFS file servers for persistent storage for a cluster node.

It would make a whole lot more sense if docker repositories published the docker-compose.yml, Dockerfile and other config files people used to create their versions of the applications. Seems like lots of recipies are on github, I need to look into that. I just want to see the steps they used to build their service, not only the final result. Let me see how to configure each package, how to setup a load balancer, how to setup a schema, how someone else solved a complex problem like mine.

The docker-registry is perfect for deploying pre-built containers locally to publish production ready containers for your operations department to launch on amazon or in the private cloud, but I really don’t want to only be able to download various people’s pre-built containers. If I can’t trust how it was built, if it’s not reproducible, modifiable, then I don’t want it.