Developing reliable integration tests requires you to control every part of your execution environment, from the operating system up to the source code itself. Running your tests inside containers takes extra work up front, so why do it?
Could you tell me what packages are on your local machine? Could you tell me what your version of chrome is at this moment? Me neither. Could I tell you what I’m running in my containers?
In a heartbeat.
There's no conflicting node packages, no chance that the browser updates, no chance you installed a conflicting node package, you know exactly what is there.
A problem I used to get constantly was
(Webdriver Exception: Cannot find a matching set of capabilities)
.
This is a useless error message.
If you don’t know what the problem is (this error was trying to tell me my firefox, geckodriver and/or selenium versions were incompatible), it gives you no guidance on solving the problem.
When you run your tests in a pipeline you spin up a container. That container is defined.
How can your tests truly align to what’s in the pipeline if you can't even know your environment?
You can create a container that matches your pipeline.
As always when trying new things there are always hurdles. Here are some problems that I faced when I started this journey.
You need to get used to the concept of thinking about your environment. You can't take it for granted.
What version of Firefox do you want to use?
What version of python or node?
You should already know the answer to these, but until you do you can't easily containerize your tests.
The first image I used was fedora
. I picked this image because I thought it
was light weight and it would be similar to Ubuntu.
Unfortunately I didn’t know how to do things different in Fedora. It caused a few problems and slowed me down.
So if you are creating an image make sure you understand the environment you are using.
First you need to install docker.
Once you have installed Docker you should get to building your Dockerfile
. I
chose ubuntu for my base image but if you wanted to go somthing a little more
lightweight you could go alpine
or mint
.
DockerfileFROM ubuntu:16.04
Install your dependencies. Robot Framework
requires python
, so we need to install that first.
DockerfileRUN apt-get update && apt-get install --quiet --assume-yes \python3 python3-pip unzip
Next install robot
via pip3
.
DockerfileRUN pip3 install --upgrade robotframework
Last step is to copy your tests into the container. In this case, copy
everything in the same directory as the Dockerfile
.
DockerfileCOPY . /robot
Your Dockerfile
should look like this. Name it Dockerfile
and place in the
base directory.
DockerfileFROM ubuntu:16.04RUN apt-get update && apt-get install --quiet --assume-yes \python3 python3-pip unzip firefox wgetRUN pip3 install --upgrade robotframeworkCOPY . /robot
This blog isn't about robot
, so I'm going to just supply a basic test that
builds and passes right away.
Create a file called my-first-test.robot
and place it next to the Dockerfile
you just made.
robotmy-first-test.robot*** Variables ***${test} Pass*** Test Cases ***TestPass Execution ${test}
Finally with all this done you can build and run the container.
bash# Build the image and tag it as myfirstdockertest$ docker build -t myfirstdockertest .# Start the container with the image we just built# We're also mounting our current directory$ docker run --name myfirstdockertest -td -v ${PWD} myfirstdockertest# Jump into the container and run the robot test$ docker exec -t myfirstdockertest robot robot/my-first-test.robot# Stop the container$ docker stop myfirstdockertest# Remove the container$ docker container rm myfirstdockertest
After it runs, you should see this in the console.
bash===========================================================================My-First-Test===========================================================================Test | PASS |Pass---------------------------------------------------------------------------My-First-Test | PASS |1 critical test, 1 passed, 0 failed1 test total, 1 passed, 0 failed===========================================================================
Thats a pass!
If you want to understand some more of what's going on here I have compiled a list of further reading of things that I found interesting.