Managed Apache OpenWhisk as IBM Cloud Functions

Automate web forms with Selenium and Serverless Python

This blog post is about using Selenium as the automation vehicle with Serverless Python actions.

Recently I was asked to support a client with building a scaffold for a software bot based on Serverless actions. The task involved using a complex Python code with Apache OpenWhisk provided in the free tier of IBM Cloud.

Please find the details on doing it here: https://github.com/blumareks/openwhisk-python

For the purpose of the automation, and custom libraries used (Chromium) you want to expand the code from the above mentioned GIT HUB repository. You might want to research the details on creating a custom actions with Docker based images for Apache OpenWhisk, and IBM Cloud Functions. JeanCarl Bisson suggested me to use the following StackOverflow: https://stackoverflow.com/questions/59778963/selenium-run-on-alpine-3-6-container in order to make this attempt successful.

Here are the steps to create the Serverless Action, that you would leverage in order to include custom binaries in the custom Docker image:

1. building a Dockerfile
2. publishing an image in Dockerhub
3. creating a Serverless action
4. activating the action via command call

Building a Dockerfile

In order to get a Docker image that is supported by Apache Openwhisk you might need to follow this recipe.

You might want to start with the Alpine linux based image for Python:

FROM openwhisk/python3action:latest
RUN apk update

Then you might want to add the required Chromium, Chrome (chromedriver), and Selenium libraries:

RUN apk add chromium && \
    apk add chromium-chromedriver
RUN apk add --upgrade --no-cache pango-dev gdk-pixbuf gtk+3.0-dev
RUN apk add --update py-pip
RUN pip install -U selenium

The entire Dockerfile looks like this:

FROM openwhisk/python3action:latest
RUN  apk update
RUN  apk add chromium && \
     apk add chromium-chromedriver
RUN  apk add --upgrade --no-cache pango-dev gdk-pixbuf gtk+3.0-dev
RUN  apk add --update py-pip
RUN  pip install -U selenium

Now you are ready to build the image. Issue the following command in a terminal from where the Dockerfile is located (you might want to docker login first):

$ docker build -t python3action:selenium.chromium.5 .

The command would result hopefully with successful completion.

=> exporting to image                           0.2s
=> => exporting layers                          0.1s
=> => writing image sha256:<some_chars&digits>  0.0s
=> => naming to docker.io/library/python3action:selenium.chromium.5                                                                          0.0s

publishing an image in Dockerhub

You might want to tag the image with the version, and push it then to Dockerhub, or other repository, that would be accessible from IBM Cloud Functions.

$ docker tag python3action:selenium.chromium.5 blumareks/python3action:selenium.chromium.001

You might want to check if your image works. Issue the following command (and when done exit, by writing exit):

$ docker run -ti blumareks/python3action:selenium.chromium.001
/bin/bash
bash-5.0# chromium-browser --version
Chromium 81.0.4044.113 
bash-5.0# exit

Screenshot: checking up if the chromium works in the image.

After the above test is successful feel free to push the image to the Dockerhub (or other image repository of your choice). Please replace blumareks with your Docker name:

docker push blumareks/python3action:selenium.chromium.001

creating a Serverless action

You can now create a Serverless Action that leverages the Docker image.

The python action might look like this:

file: action.py
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
def main(args):
    chrome_options = Options()
    chrome_options.add_argument("--no-sandbox")
    chrome_options.add_argument("--headless")
    driver = webdriver.Chrome(options=chrome_options)
    driver.get("http://www.python.org")
    assert "Python" in driver.title
    elem = driver.find_element_by_name("q")
    elem.clear()
    elem.send_keys("pycon")
    elem.send_keys(Keys.RETURN)
    assert "No results found." not in driver.page_source
    mypage = driver.page_source
    driver.close()
    return {
      "page": mypage
    }

Now you are ready to bind the above action with the Docker and create the Apache Openwhisk action in IBM Cloud.

You might want to follow the first steps shown in the github repo referenced above:

sign up to IBM Cloud
You might want to sign in/up to IBM Cloud. After creating an account on http://cloud.ibm.com use this link to download a CLI: https://cloud.ibm.com/functions/learn/cli

Please mind that I will be using CF based Namespace for the following actions (as they are required for Serverless Swift to operate).

The CF based Namespace look like something this: serverless.swift@roboticsind.com_dev

login from IBM Cloud CLI

The easiest way to login from CLI is using your browser’s existing session in IBM Cloud. Simply type:

ibmcloud login -a cloud.ibm.com -o “serverless.swift@roboticsind.com” -s “dev” — sso

This should open a new window, and the session token should be presented to you. Copy it and paste in the terminal window.

Get a one-time code from https://identity-1.us-south.iam.cloud.ibm.com/identity/passcode to proceed.
Open the URL in the default browser? [Y/n] > 
One-time code > 
Authenticating…
OK
Targeted account Serverless Swift …

Also you need to target the environment:

$ ibmcloud resource groups
…some groups here…
$ ibmcloud target -g Default

connect to IBM Cloud Functions
You would need to dowload the CF plugin:

ibmcloud plugin install cloud-functions

And now you can test if the connection works (if not, check my book — Serverless Swift — for hints):

ibmcloud fn list
ibmcloud fn package create test
$ ibmcloud fn action update test/docker-selenium --docker blumareks/python3action:selenium.chromium.001 action.py 

activating the action via command line call

As soon as the action is ready you should be ok to run the Serverless action from the command line. Test your action:

ibmcloud fn action invoke test/docker-selenium --blocking

Let me know if that worked for you — just clap! 👏 That would make me very happy.

And last, but not least:

You might want to checkout my book on Serverless Swift — https://www.apress.com/us/book/9781484258354

Thank you for reading this blog post/book. Follow me on Twitter: http://twitter.com/blumareks