Page 1 of 3

My First Hackweek Was All about Shippping


Saturday, December 19, 2015

Last week I had the chance to participate in my first Hackweek. I never had such an experience in any other company I’ve ever worked for and between my colleagues’ reports about previous experiences and my own expectations I was very excited to see what was all the fuzz about.

These type of events are not unique to SUSE, as a matter of fact Twitter and a bunch of other companies were also having their Hackweeks at the same time and I’m glad this is the case because after having the chance to participate in one I realize it’s a great way to promote creativity.

A hackweek is basically a week were you get to work on anything you want to work on. You are not expected to deliver anything but instead encouraged to experiment and explore with anything you think is worth spending time on.

In order to make the most out of Hackweek I decided to join a project and not start one of my own so I could do some pairing. This kind of interactions always make it a lot of fun for me plus I get to learn a ton. That’s how I joined Cornelius Schumacher to work on Yes Ship It! This is a project he had already started on his own so we were not doing everything from scratch.

The approach of yes_ship_it is different from the typical release script. It doesn’t define a series of steps which are executed to make a release. It defines a sequence of assertions about the release, which then are checked and enforced.

The first thing we decided to do together was a Rails App which allows you to track successful software releases. Since it was going to be 100% related to Yes Ship It! we decided to call it Yes It Shipped!. Let me show you how trivial it is to add it to a project like the formstack-api gem.

  1. Install the yes_ship_it gem

    1
    $ gem install yes_ship_it
    
  2. Add a yes_ship_it.conf file

    1
    $ yes_ship_it init
    
  3. Release!

    1
    $ yes_ship_it
    

By default yes_ship_it will check if:

  • you are in the right release branch (by default master) and the code was pushed.
  • the working directory is not missing to commit anything.
  • the version was update
  • the changelog was updated
  • a tag was added and published
  • a new version of the gem was built and published

The aim is to make it as generic as possible so you can adapt it to any project you have. For starters you can remmove any check in the process and soon enough you will be able to add checks of your own.

What I like the most about it is that I can run yes_ship_it at any time. I don’t need to remember or make sure what was the last step I did because that’s exactly what it will do for me.

What do you think? Leave your comments below and remember to release early and release often!


Dockerizing Your Infrastructure with Machinery


Thursday, October 15, 2015

Docker offers to make it easy to deploy applications no matter what you infrastructure is. The problem is that you cannot just switch from one technology to another overnight just because it will help you solve certain issues. Moving to the new technology is an issue on its own, plus there is no technology that doesn’t come with its own issues. With Machinery we want to help you make the transition from VMs to containers as smooth as possible.

This post was originally delivered as a presentation in LinuxCon Dublin 2015.

Let’s assume you have a server running a web application that uses Rails, Apache and MariaDB. We will use Machinery to create a System Description from this server which we will manually review and then allow machinery to automatically detect the respective workloads. Finally we will use machinery to verify that the see the differences between the original system and the resulting containers.

On my case I will use Portus as my Rails application.

Our work flow will look more or less like this:

Work flow

Creating a system description

First of all you will need to make sure you can access the server without requiring a password by copying your ssh keys into the server.

1
ssh-copy-id root@192.168.1.100

This is required because machinery will open an ssh connection and run a set of commands in order to evaluate the information inside the server. Then go ahead and inspect the server in order to create a system description.

1
machinery inspect --name portus --extract-files 192.168.1.100

The option --name is the name we want to give to our system description. The option --extract-files will make a copy of every file except those that are the default from an RPM package.

Once machinery has finished a folder named wordpress will be created inside /home/USERNAME/.machinery. If you have a pick into this folder you will find a file called manifest.json which is the physical representation of the system description and a files directory containing the extracted files.

You can easily open the system description with your favorite text editor but machinery also provides a show command for easy access.

1
machinery show portus --html

The additional --html option will tell machinery to start a web server on your local machine which will help us have a more interactive look about how this server has been set up. By omitting this option you will get access to the same information as text through the CLI.

Your default browser will open up with all the information about the wordpress system description. From here you can manually see in detail every configuration file, package and service that was running in order to write your Dockerfile.

HTML View

This feature on its own will help you out a lot because you can translate all this information not only into a Dockerfile but also into a configuration management system.

Containerizing

For the particular case of Rails (using MariaDB and Apache2) we’ve implemented a really nice feature into machinery that will allow you to only run a command and get the Dockerfiles created for you and also the docker-compose files so you can orchestrate your application once it has been containerized.

In order to do this you need to tell machinery to containerize a system description whose files have been extracted (in our case the wordpress system description)

1
machinery containerize --output-dir /tmp/containers portus

The required --output-dir tells machinery where you want your containerized version of the server to be saved.

Now we can go to /tmp/containers/portus and have a look. Machinery has created a set of files for us to run a containerized version of our application. The details on how to set it up and get it running will be inside the README.md file.

The first thing we need to do is to run the setup script.

1
./setup.rb

This script is unique depending on the application that was containerized. In the case of Portus it sets up a database container so that our web container can connect to it.

1
docker-compose up

Docker compose will start an instance of all the required containers. For Portus this means a web, a db and a registry container and allow you to access them from your local machinery through port 3000.

Voila! You just got yourself a containerized version of your application. How cool is that?

Introspection

Now this sound very nice but once you’ve done all this (specially if you are writing your own Dockerfiles or CMS recipes, modules, etc) you might want to check what is inside your resulting containers.

1
machinery inspect-container portus_db

At this point I could just show the portus_db system description and have a look if everything is the way I expect it to be but we can go one step further and compare it to the portus system description. This way I can make sure what the differences are between the container and the VM.

1
machinery compare portus portus_db

I could have added the --html option but I won’t so you can see the cli view of machinery. It was designed to play well with your other *nix commands so you could easily do something like this, to determine if there is a difference in the MariaDB package.

If there were no differences machinery would ommit this information. For this reason I’ve created a contianer with a different MariaDB version

1
2
3
4
machinery compare portus portus_db --no-pager | grep mariadb
* mariadb (version: 10.0.21 <> 10.0.20)
* mariadb-client (version: 10.0.21 <> 10.0.20)
* mariadb-errormessages (version: 10.0.21 <> 10.0.20)

You can easily spot the differences here. Now it’s up to you to change your Dockerfile or CMS files in order to add the right version of the package you’d like to install.

Summary

As you can see Machinery can be very useful to start deploying your applications as docker containers. Inspect an existing system and understand how it was configured using the inspect and show commands respectively. Write your Dockerfiles and build your docker images which you can then also inspect and compare to the original system using the inspect-container and compare commands.

While doing this remember to have a lot of fun and share back how you used machinery in your own project.


Ally Skills Workshop Review


Thursday, October 8, 2015

Last week I had the chance to attend the Ally Skills Workshop with Valerie Aurora and can’t recommend it enough. This was an enlightening experience on learning how to support women in their workplaces and communities and it should be given at every tech event or better yet be part of the training in tech companies.

Valerie is a very cool person. She presented herself very modestly but after reading more about her I think she could have bragged much more about her development skills. She gave a 30 minute introduction on the class and put us in groups so we could work on multiple scenarios.

Each scenario was about real life encounters that women have had at their workplaces and made them feel uncomfortable. We as men got the chance to express how we think we should deal with the situation in order to give our support.

There were also women in our groups which made everything more realistic because they could give us immediate feedback and share similar experiences. Once we were done we would share back with Valerie and she would give us additional feedback or correct us when necessary.

I’ve always felt happier and more productive when working in teams were people respect one another and I believe this kind of workshop provides the necessary tools and hands on experience to build such environments for women.


Test Drive Machinery Using a Docker Container


Thursday, September 10, 2015

In my last post I shared how to install Machinery. The process assumes that you are running openSUSE 13.2 but this of course might not be the case. You might be running a different Linux distro or even another OS. That’s were Docker comes very handy. In this post I will show you how to easily test drive Machinery on a container.

Machinery will save all its data in a .machinery folder at your home directory. In order to be valuable, this data needs to persist after you kill your Docker containers.

1
mkdir -p ~/.machinery

Now we can pull the image from the Docker Hub

1
docker pull mauromorales/machinery

Once the image is in your system all we need to do is to start a container every time we want to run machinery.

1
2
3
4
$ docker run -ti \
> -v ${HOME}/.machinery:/root/.machinery \
> -v ${HOME}.ssh:/root/.ssh \
> mauromorales/machinery  /bin/bash

Here we are telling docker to start a container in -ti interactive mode with tty and to -v map a volume from the folder we just created to the /root/.machinery this is root’s home directory because root is running Machinery inside our container. Additionally we will also map our .ssh folder so machinery can use your ssh keys to access the remote servers (if you don’t want to share this folder with your container just map it to a different directory and generate new keys) mauromorales/machinery specifies the image we downloaded and finally we run bash.

Once your container has started you can start using Machinery :) happy hacking!


Installing Machinery


Tuesday, August 11, 2015

For the past three months I’ve been working on a really cool tool called Machinery and I’d like to share some of the neat things you can do with it.

In this first post I’ll show you how to install Machinery on openSUSE 13.2

First you need to add the systemsmanagement repo to your system. You can easily do it with zypper running the following command as root:

1
2
3
zypper ar -f http://r.opensu.se/systemsmanagement:machinery/13.2 \
systemsmanagement
zypper refresh

You will get asked about a key for the repo. At this point you need to trust temporarily or trust always.

Immediatelly afterwards we should be able to install Machinery by running:

1
zypper in machinery

When asked if you want to install the respective packages you’ll need to answer with a “y” (yes)

Now let’s double check that machinery was propperly installed by running the version option.

1
machinery --version

At the moment of writing the version is:

1
machinery version 1.11.2 (system description format version 4)

Learn more about how to use Machinery on the wiki or by reading the man page.

1
man machinery

Have fun! And if you spot any issues please let us know by opening an issue and if you have any questions write us to our mailing list