Chatbots? Really?

The new host trend in Silicon Valley is chatbots. Everybody and their brother is rushing to produce an API so businesses can create their own chatbots that interact with people via text (or voice?) conversation.

Satya Nadella of Microsoft calls it “conversation as a service”. Facebook and the media echo chamber that follow them around call it the next big revolution of the Internet. No more user-interface issues (as if chatting isn’t a user-interface, and everybody can speak and spell intelligibly).

The only reason I can see for this push is it will be harder to block ads that appear (or are read to you) in the middle of a chat session.

Pylint

The past few weeks I’ve been exploring Python and related libraries and packages to build an application at work. I’m an old Perl hacker, so Python and it’s strict indentation format limitations really rail on my nerves. But it’s a really powerful language, and all the tools I’m trying to work with support Python or REST APIs.

Friday, I decided the most annoying part of Python programming is it’s inability to detect common mistakes, like typos, and undefined variable names until you actually exercise the code that uses that mistyped variable name.

Back in my C programming language days, we used a program called ‘lint’ to locate obvious errors in source code before running a whole compiler. Friday, I googled python lint, and found out about ‘pylint’, which is exactly what I needed. The default configuration of pylint, enforces a strict coding style I was not used to previously, so it took me a good 5-6 hours to get my software to pass pylint with a minimum of warnings and no errors. but I find it really works well, and helps find all the issues I mostly worry about – not logic issues, I can deal with those, but syntax issues, like lines indented incorrectly (usually due to some extra tab characters where you don’t want them) or simple typos or fat fingers.

Pylint will speed up my app development, I guarantee it.

We stand at the precipice

US Vice-President Henry Wallace, April 9, 1944.

“The American fascists are most easily recognized by their deliberate perversion of truth and fact. Their newspapers and propaganda carefully cultivate every fissure of disunity… They claim to be super-patriots, but they would destroy every liberty guaranteed by the Constitution. They demand free enterprise, but are the spokesmen for monopoly and vested interest. Their final objective toward which all their deceit is directed is to capture political power so that, using the power of the state and the power of the market simultaneously, they may keep the common man in eternal subjection.”

LXC Linux Containers

I’ve been using LXC Linux Containers to host a number of OpenStack admin processes, as well as other things, like playing with Galera+MariaDB. Anyway, I thought I’d share my Bash shell scripts for container creation, management, etc. Your mileage may vary.

You’ll find them at http://whistl.com/files/cnutils.shar in a shell archive.
Just run it to extract the scripts, and examine them or use them at your own risk. They do assume an LVM environment with the vgname vg1, and a btrfs filesystem mounted at /var/logdata on the lxc host.

The script cncreate.sh creates a container, given the name, ip, mac address and lvm volume size (eg 20G). It’ll create a btrfs subvolume of /var/logdata and mount it as /var/log inside the container, for centralized logging. This script uses the base1.config file as a template for /var/lib/lxc/<containerneme>/config, and common.conf is a copy of my /etc/lxc/common.conf

The script cnclone.sh is kind of like cncreate.sh, except it uses the LVM snapshot feature to clones another “base” container instead of creating one from scratch. LVM snapshots are copy-on-write clones, so only things added or changed after the snapshot consume any additional disk space. You can create a base container, install 100GB of software, then make snapshots of that base, and each snapshot would have full access to all those packages, but only consume additional disk space for it’s config files, log files, and writable data files. This can be a huge space savings, but it’s saves time on configuring. If your base container already knows how to authenticate with your LDAP cluster, you might only need modify a few config files post snapshot, if individual accounts or certificates were required.

The script cnls.sh lists all configured containers in /var/lib/lxc, and if they are running, some additional info.

Then there’s cnman.sh, which is just a general management utility which takes an argument of stop, start, restart or status and operates on ALL containers. Nice when you need to fix something. Only if a container has lxc.start.auto = 1 will it be started by cnman.sh start, but it will shut any down it finds for stop.

Enjoy,

whistl
<>

MariaDB + Galera

I got a bit distracted. Just learned about MariaDB 10.1, which now includes the Galera multi-master database package built-in. I had to make it work, of course, and using three containers, it wasn’t all too hard. The documentation is a little weak at this time, because they want to sell their support contracts, but it’s not too bad.

This is what I had to add to /etc/my.cnf.d/server.conf on CentOS 7:


[galera]
wsrep_on=ON
# this varies on different versions and OSs
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_provider_options="gcache.size=512M"
# your cluster's name
wsrep_cluster_name="cluster1"
# who belongs to the cluster
# use the next line on the first server only, the very first time only
#wsrep_cluster_address="gcomm://"
# after the first time, and on every other server, use the whole cluster
wsrep_cluster_address="gcomm://192.168.1.67,192.168.1.68,192.168.1.74"
# define method of sync (requires rsync package be installed, the default is mysqldump)
wsrep_sst_method=rsync
# you need to create this user with this passwd on node 1,
# after bring it up the first time and before joining node2
# to the cluster, for wsrep to work
wsrep_sst_auth=wsrepuser:wsrep16secretpass
# put the node's unique shortname here
wsrep_node_name="mariadb1"
#wsrep_node_name="mariadb2"
#wsrep_node_name="mariadb3"
# this is where the wsrep server will bind
# put the node's storage network IP address here
wsrep_node_address="192.168.1.67"
#wsrep_node_address="192.168.1.68"
#wsrep_node_address="192.168.1.74"
# this is where mysql service will be offered
# put the node's production IP address here
bind-address="192.168.1.67"
#bind-address="192.168.1.68"
#bind-address="192.168.1.74"
query_cache_size=0
binlog_format=row
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
query_cache_type=0

# Then you had to start the first node, the first time only, with:

/etc/init.d/mysql start --wsrep-new-cluster

# The you set things up:
$ mysqladmin -u root password
# enter the new root password twice

# do it again for the public interface
$ mysqladmin -u root -h mariadb1 password

$ mysql -u root -p
mysql> use mysql;
mysql> delete from user where User='';
mysql> create user wsrepuser;
mysql> grant all on *.* to 'wsrepuser'@'%' identified by 'wsrep16secretpass';

# check your wsrep process
mysql> show status like 'wsrep_cluster_size';
$ less /var/log/mysql/mysqld.log

# join node two and three (same config, uncomment that node's name and ip addresses)
# just start them like normal, then check the cluster size again and their mysqld.logs
# change the gcomm line in node 1, and restart it normally. You're up!