ui-browser-test: Adding UI browser testing

Ui browser test developed on top of Robot frame work.
Test runs chromium, chrome and firefox test cases.

On LAVA we have access to serial console and we have to run ui tests
from this serial console, this is possible by changing user from
root to linaro and exporting the DISPLAY=:0. because linaro owns the
X display process.

Change-Id: I43bda407f468120839b0a4c8248777303945529e
Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>

11 files changed, 390 insertions, 0 deletions

diff --git a/automated/lib/sh-test-lib b/automated/lib/sh-test-lib
index fca1153..3edecb8 100755
--- a/automated/lib/sh-test-lib
+++ b/automated/lib/sh-test-lib
@@ -114,3 +114,22 @@ install_deps() {
         esac
     fi
 }
+
+validate_check_sum() {
+    if [ "$#" -ne 2 ]; then
+        warn_msg "The number of parameters should be 2"
+        error_msg "Usage: validate_check_sum filename known_sha256sum"
+        return 1
+    fi
+    OUTPUT_FILE_NAME="$1"
+    SHA256SUM_CHECK="$2"
+    # Get sha256sum of output_file
+    GET_SHA256SUM=$(sha256sum ${OUTPUT_FILE_NAME} | awk '{print $1}')
+    echo "GET_SHA256SUM is "${GET_SHA256SUM}""
+    if [ "${SHA256SUM_CHECK}" = "${GET_SHA256SUM}" ] ; then
+        return 0
+    else
+        echo "checksum did not match"
+        return 1
+    fi
+}
diff --git a/automated/linux/ui-browser-test/README.rst b/automated/linux/ui-browser-test/README.rst
new file mode 100644
index 0000000..f2b05be
--- /dev/null
+++ b/automated/linux/ui-browser-test/README.rst
@@ -0,0 +1,53 @@
+=====================
+Robot framework tests
+=====================
+Robot Framework is a generic test automation framework for acceptance testing
+and acceptance test-driven development (ATDD). It has easy-to-use tabular test
+data syntax and it utilizes the keyword-driven testing approach. Its testing
+capabilities can be extended by test libraries implemented either with Python
+or Java, and users can create new higher-level keywords from existing ones
+using the same syntax that is used for creating test cases. reference: [1]
+
+[1] http://robotframework.org/
+
+Requirements
+============
+- Linux (Debian / Ubuntu / Openembedded / Fedora based)
+- Python 2.7
+- python-pip
+- robotframework
+- robotframework-selenium2library
+- Web-Browser (firefox, google-chrome or chromium)
+- chromedriver
+- google-chrome / chromium / firefox
+
+Installation and Run
+=====================
+If you are on Debian or Ubuntu please run
+
+be a root
+# ./install-on-debian.sh
+# ./ui-browser-test.sh -u linaro -s false
+
+Basic Usage
+===========
+robot testcase-name.robot
+
+Examples
+--------
+robot chrome-test.robot
+robot chromium-test.robot
+robot firefox-test.robot
+robot login-lava.robot
+robot youtube-play-bkk16.robot
+robot youtube-play.robot
+
+Run all tests in the current directory
+python -m robot .
+
+NOTES
+=====
+Ensure you have right PATH exported before running tests
+
+For more information on usage:
+https://github.com/robotframework/robotframework/blob/master/INSTALL.rst
diff --git a/automated/linux/ui-browser-test/install-on-debian.sh b/automated/linux/ui-browser-test/install-on-debian.sh
new file mode 100755
index 0000000..7be1cf7
--- /dev/null
+++ b/automated/linux/ui-browser-test/install-on-debian.sh
@@ -0,0 +1,66 @@
+#!/bin/bash
+#
+# Install robotframework and chromedriver
+# Copyright (C) 2016 Linaro Limited
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+#
+# Author: Naresh Kamboju <naresh.kamboju@linaro.org>
+
+set -eu
+. ../../lib/sh-test-lib
+# pre-requirements
+pre_requirements () {
+    pip install robotframework
+    pip install robotframework-selenium2library
+}
+
+# For armv7
+install_chromedriver_armv7l () {
+    google_chrome_softlink
+    apt-get -y install chromedriver
+    cp /usr/lib/chromium/chromedriver /usr/bin/
+    chmod 777 /usr/bin/chromedriver
+}
+
+# For aarch64
+install_chromedriver_aarch64 () {
+    google_chrome_softlink
+    apt-get -y install chromedriver
+    cp /usr/lib/chromium/chromedriver /usr/bin/
+    chmod 777 /usr/bin/chromedriver
+}
+
+# In case of 96boards we pre install chromium on linaro builds
+# Officially chromium not supported by robot framework.
+# Hack, created a soft link as google-chrome from chromium.
+# In the test case we have to mention browser name as chrome.
+# But test case opens chromium browser
+google_chrome_softlink () {
+    cd /usr/bin/
+    ln -sf chromium google-chrome
+    cd -
+}
+
+pre_requirements
+ARCH=$(uname -m)
+type install_chromedriver_"${ARCH}"
+if [ $? -ne 0 ]; then
+    echo "Not supported architecture ${ARCH}"
+    echo " $0 : failed"
+    exit 1
+else
+    install_chromedriver_"${ARCH}"
+fi
diff --git a/automated/linux/ui-browser-test/robot-results-parser.py b/automated/linux/ui-browser-test/robot-results-parser.py
new file mode 100755
index 0000000..ad0e4d4
--- /dev/null
+++ b/automated/linux/ui-browser-test/robot-results-parser.py
@@ -0,0 +1,42 @@
+#!/usr/bin/python
+#
+# Robot framework test results parser
+#
+# Copyright (c) 2016 Linaro Limited
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+#
+# Author: Naresh Kamboju <naresh.kamboju@linaro.org>
+
+import sys
+import xml.etree.ElementTree as ET
+
+input_file = sys.argv[1]
+tree = ET.parse(input_file)
+root = tree.getroot()
+
+for statistics in root.findall('statistics'):
+    for suite in statistics.findall('suite'):
+        for stat in suite.findall('stat'):
+            name = stat.get('name')
+            if 'Robot-Test-Scripts' == name:
+                status = 'pass'
+                print name, " ", status
+            else:
+                if '1' == stat.get('pass'):
+                    status = 'pass'
+                else:
+                    status = 'fail'
+                print name, " ", status
diff --git a/automated/linux/ui-browser-test/robot-test-scripts/chrome-test.robot b/automated/linux/ui-browser-test/robot-test-scripts/chrome-test.robot
new file mode 100644
index 0000000..dd205b2
--- /dev/null
+++ b/automated/linux/ui-browser-test/robot-test-scripts/chrome-test.robot
@@ -0,0 +1,14 @@
+*** Settings ***
+Library  Selenium2Library
+
+*** Variables ***
+
+*** Test Cases ***
+Open-Google-Chrome-Browser-Test
+    [Documentation]  Test program to open and close google-chrome browser
+    [Tags]  smoketest
+    Open Browser  http://www.linaro.org  chrome
+    sleep    5s
+    Close Browser
+
+*** Keywords ***
diff --git a/automated/linux/ui-browser-test/robot-test-scripts/chromium-test.robot b/automated/linux/ui-browser-test/robot-test-scripts/chromium-test.robot
new file mode 100644
index 0000000..3f56bcd
--- /dev/null
+++ b/automated/linux/ui-browser-test/robot-test-scripts/chromium-test.robot
@@ -0,0 +1,14 @@
+*** Settings ***
+Library  Selenium2Library
+
+*** Variables ***
+
+*** Test Cases ***
+Open-Chromium-Browser-Test
+    [Documentation]  Test program to open and close chromium browser
+    [Tags]  smoketest
+    Open Browser  http://www.linaro.org  chrome
+    sleep    5s
+    Close Browser
+
+*** Keywords ***
diff --git a/automated/linux/ui-browser-test/robot-test-scripts/youtube-play-linaro-connect.robot b/automated/linux/ui-browser-test/robot-test-scripts/youtube-play-linaro-connect.robot
new file mode 100644
index 0000000..c316bfc
--- /dev/null
+++ b/automated/linux/ui-browser-test/robot-test-scripts/youtube-play-linaro-connect.robot
@@ -0,0 +1,23 @@
+*** Settings ***
+Library           Selenium2Library
+
+*** Variables ***
+${Browser}        chrome
+${SiteUrl}        https://youtu.be/HpvZ2HwL4zI
+${DashboardTitle}    Linaro Connect BKK16 - Welcome and Introduction - YouTube
+${Delay}          10s
+
+*** Test Cases ***
+Youtube-Play-Linaro-Connect-Test
+    Open Browser to the Youtube Page
+    sleep    ${Delay}
+    Assert Youtube Video Title
+    [Teardown]    Close Browser
+
+*** Keywords ***
+Open Browser to the Youtube Page
+    open browser    ${SiteUrl}    ${Browser}
+    Maximize Browser Window
+
+Assert Youtube Video Title
+    Title Should be    ${DashboardTitle}
diff --git a/automated/linux/ui-browser-test/robot-test-scripts/youtube-play.robot b/automated/linux/ui-browser-test/robot-test-scripts/youtube-play.robot
new file mode 100644
index 0000000..c38e96e
--- /dev/null
+++ b/automated/linux/ui-browser-test/robot-test-scripts/youtube-play.robot
@@ -0,0 +1,19 @@
+*** Settings ***
+Library           Selenium2Library
+
+*** Variables ***
+${Browser}        chrome
+${SiteUrl}        https://youtu.be/HpvZ2HwL4zI
+${DashboardTitle}    Linaro Connect BKK16 - Welcome and Introduction - YouTube
+${Delay}          10s
+
+*** Test Cases ***
+Youtube-Play-Video-Test
+    Open Browser to the Youtube Page
+    sleep    ${Delay}
+    [Teardown]    Close Browser
+
+*** Keywords ***
+Open Browser to the Youtube Page
+    open browser    ${SiteUrl}    ${Browser}
+    Maximize Browser Window
diff --git a/automated/linux/ui-browser-test/run-robot-tests.sh b/automated/linux/ui-browser-test/run-robot-tests.sh
new file mode 100755
index 0000000..1996889
--- /dev/null
+++ b/automated/linux/ui-browser-test/run-robot-tests.sh
@@ -0,0 +1,34 @@
+#!/bin/bash
+#
+# Run robot framework test cases
+#
+# Copyright (C) 2016 Linaro Limited
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+#
+# Author: Naresh Kamboju <naresh.kamboju@linaro.org>
+
+set -eu
+export DISPLAY=:0
+echo "env"
+echo "==="
+env
+echo "who"
+echo "==="
+who
+UI_BROWSER_TEST_OUTPUT="ui-browser-test-output"
+mkdir -p "${UI_BROWSER_TEST_OUTPUT}"
+python -m robot -d "${UI_BROWSER_TEST_OUTPUT}" robot-test-scripts/ || true
+exit 0
diff --git a/automated/linux/ui-browser-test/ui-browser-test.sh b/automated/linux/ui-browser-test/ui-browser-test.sh
new file mode 100755
index 0000000..9d7b377
--- /dev/null
+++ b/automated/linux/ui-browser-test/ui-browser-test.sh
@@ -0,0 +1,81 @@
+#!/bin/bash
+#
+# ui browser tests by using Robot framework
+#
+# Copyright (C) 2016 Linaro Limited
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+#
+# Author: Naresh Kamboju <naresh.kamboju@linaro.org>
+
+set -eu
+. ../../lib/sh-test-lib
+WD="$(pwd)"
+OUTPUT="$(pwd)/output"
+RESULT_FILE="${OUTPUT}/ui-browser-test-results.txt"
+UI_BROWSER_TEST_OUTPUT="ui-browser-test-output"
+
+usage()
+{
+    echo "Usage: $0 [-u username] [-s <true|false>]" 1>&2
+    echo "the user should own X process and DISPLAY=:0"
+    exit 1
+}
+
+
+while getopts "s:u:" o; do
+  case "$o" in
+    # Skip package installation
+    s) SKIP_INSTALL="${OPTARG}" ;;
+    # Default user is linaro
+    u) TESTUSER="${OPTARG}" && id "${TESTUSER}" ;;
+    *) usage ;;
+  esac
+done
+
+if [  $# -ne 4 ]; then
+    usage
+fi
+
+
+# Test run.
+! check_root && error_msg "This script must be run as root"
+pkgs="python2.7 python-pip python-lxml"
+install_deps "${pkgs}" "${SKIP_INSTALL}"
+
+[ -d "${OUTPUT}" ] && mv "${OUTPUT}" "${OUTPUT}_$(date +%Y%m%d%H%M%S)"
+mkdir -p "${OUTPUT}"
+
+dist_name
+if [ "${dist}" = "Debian" ] || [ "${dist}" = "Ubuntu" ]; then
+    "${WD}"/install-on-debian.sh
+else
+    echo "Not a debian machine"
+fi
+
+# Copy robot test scripts to /tmp
+cp -a robot-test-scripts /tmp/
+# Tests should runs by linaro users because X owned by linaro user.
+# linaro user can not create output files in /root
+# so change directory to /tmp
+cd /tmp
+# Run as TESTUSER
+su "${TESTUSER}" -c "${WD}"/run-robot-tests.sh
+# "${UI_BROWSER_TEST_OUTPUT}" directory created by TESTUSER from run-robot-tests.sh
+mv "${UI_BROWSER_TEST_OUTPUT}" "${OUTPUT}"
+mv robot-test-scripts "${OUTPUT}"
+# Parse test results
+python "${WD}"/robot-results-parser.py "${OUTPUT}"/"${UI_BROWSER_TEST_OUTPUT}"/output.xml >> "${RESULT_FILE}"
+cd -
diff --git a/automated/linux/ui-browser-test/ui-browser-test.yaml b/automated/linux/ui-browser-test/ui-browser-test.yaml
new file mode 100644
index 0000000..980ba3d
--- /dev/null
+++ b/automated/linux/ui-browser-test/ui-browser-test.yaml
@@ -0,0 +1,25 @@
+metadata:
+    name: ui-browser-test
+    format: "Lava-Test-Shell Test Definition 1.0"
+    description: "ui browser tests by using robot frame work
+                  Test runs chromium, chrome and firefox browser test cases."
+    maintainer:
+        - naresh.kamboju@linaro.org
+    os:
+        - debian
+    scope:
+        - functional
+    devices:
+        - hi6220-hikey
+        - apq8016-sbc
+
+params:
+    # Default user is linaro
+    TESTUSER: "linaro"
+    # Install deps packages
+    SKIP_INSTALL: "False"
+run:
+    steps:
+        - cd ./automated/linux/ui-browser-test/
+        - ./ui-browser-test.sh -u "${TESTUSER}" -s "${SKIP_INSTALL}"
+        - ../../utils/send-to-lava.sh ./output/ui-browser-test-results.txt