How to Deploy Selenium grid on AWS / Amazon EKS

  

                                    Selenium Grid is good for parallel execution but maintenance is a nightmare in an era where you see a frequent upgrades to browsers and corresponding drivers. No sooner the usage of automation framework / selenium grid increases, scalability and maintenance becomes a challenge. To address such issues, we do have solutions based on dockers, docker swarm etc. Having said that, there are some caveats in scaling, managing container health etc.

Below solution would try to address most of them. Major chunks of the solution include Selenium, Zalenium, Docker, Kubernetes and Amazon EKS.

This article would outline the process of deploying Selenium grid(Zalenium) on AWS (Amazon EKS) using Kubernetes and Helm.

What do we achieve with this setup..?

• Scalability: EKS can scale the nodes and pods as per the given configuration.
• Visibility: Zalenium provides a feature to view the live executions on the containers.
• Availability: Amazon EKS cluster makes selenium grid available all the time.
• Maintenance: Low maintenance as the containers are destroyed after each execution.

Pre-requisites:

• An active Amazon AWS account.
• IAM user is created in AWS account
• AWS CLI is connected to AWS account providing the user credentials using local powershell or any terminal
                                                              OR
• Use AWS cloudshell which is automatically connected to logged in account.
• Install AWS CLI (for local terminal), kubectl, helm in the given order.

Lets Get Started!

Once the above pre-requisites are met, next task to deploy any application on kubernetes is to create a kubernetes cluster. There are different ways to create a cluster on AWS, I'll brief couple of ways to achieve the same.

First, Create cluster from AWS GUI.

1. Create master node or cluster 

• Open Amazon EKS console
• Choose Create Cluster
• Provide details like cluster name, k8s version, role
• Select VPC, security groups, endpoint access
• Further steps as shown on GUI which will make 'master' ready.

2. Create worker nodes and connect to the above created cluster.

• Create Node Group (Amazon EC2) instances.
• Choose the cluster, to which the above node group should get attached.
• Select security group, resources etc.,
• Define min and max number no. of nodes.

Sounds complex?. No issues, there is another simple and efficient way to make the whole process look simple.

Second, Create cluster using eksctl (The official CLI for Amazon EKS)

The above complex task can be achieved with a single command.

How to Deploy Selenium grid on Google Cloud Platform using Kubernetes and Helm

                                          

                                        It is a open secret that Selenium Grid maintenance is a nightmare in an era where you see a frequent upgrades to browsers and corresponding drivers. No sooner the usage of automation framework / selenium grid increases, scalability and maintenance becomes a challenge. To address such issues, we do have solutions based on dockers, docker swarm etc. Having said that, there are some caveats in scaling, managing container health etc.

Below solution would try to address most of them. Major chunks of the solution include Selenium, Zalenium, Docker, Kubernetes and Google Cloud Platform.

This article would outline the process of deploying Selenium grid(Zalenium) on Google cloud platform using Kubernetes and Helm.

What do we achieve with this setup..?

• Scalability: GCP/GKE can scale the nodes and pods as per the given configuration.
• Visibility: Zalenium provides a feature to view the live executions on the containers.
• Availability: GKE kubernetes cluster makes selenium grid available all the time.
• Maintenance: Low maintenance as the containers are destroyed after each execution.

Pre-requisites:

• An active Google Cloud Platform account
• Enable Kubernetes engine by setting up a Billing Account.

How to setup Selenium Grid using Docker Desktop - Windows

                            Selenium Grid has made the test automation execution much faster and smarter. The excessive usage of selenium grid has its own hitches like too much of system resource utilization, all browsers to be installed on the system etc. 

With the introduction of docker images for selenium, made the life of testers lot easier. Out of multiple ways to bring up selenium grid with docker, I would choose the simple & quick way to bring up using docker-compose.

Pre-requisite: Docker desktop for windows is installed.

Steps to bring up Selenium Grid:

• Pull required version of selenium hub and nodes using the command "docker pull" as below

         docker pull selenium/node-chrome-debug:3.141.59 

• Create a yaml file with the below docker-compose instructions.

• run the command 

         docker-compose -f docker-compose.yml up -d

How to send Cookies from Web(GUI) to Web Service

Cookies: Browser Cookies are also referred as Internet Cookie,Web Cookie or HTTP Cookie. A cookie is a "small piece of data" sent from the website to the user's device while the user is accessing the website.
There are multiple ways these cookies are being used in modern world like authentication, browser state information, storing user personal information like name, address etc., tracking the user activity on web, etc., Different types of cookies satisfy different needs.

In certain cases, one might have to test web services which needs cookies to be passed as part of request. One way to address this, is to combine Web application plus web service test steps. 

Below code snippet would open the web application, capture the cookies stored by the application and pass it to web service using Rest Assured.


public class CookiesToWebServices {
static WebDriver driver;
static String driverPath = "C:/Drivers/chromedriver.exe";

public static void main(String[] args) {

System.setProperty("webdriver.chrome.driver",driverPath);
ArrayList cookieList = new ArrayList();

String url = "www.testurl.com/testpath/id";
HashMap headermap = new HashMap<>();
headermap.put("Content-Type", "application/json");
headermap.put(....

HashMap queryParammap = new HashMap<>();
queryParammap.put("id", "12345");
queryParammap.put(....

How to Connect Zephyr For Jira Programmatically (ZAPI)

Jira by default is a 'Defect Management Tool'. However, available add-on for Jira make it much more capable than just defect Management tool. Couple of such add-ons like 'Zephyr For Jira' & 'ZAPI' (API to access data from 'Zephyr For Jira' programmatically) make Jira capable for 'Test Management'. Both of these add-ons have different version for Server & Cloud accordingly. Let us look at, how do we work with ZAPI server version.

Use Case: Whenever the automation is run, it has to update the status of execution in test cycle.

Pre-requisite: Manual Tests are created in Jira with Issue Type - Test & Test cycle is created with tests added into it (Adding tests into test cycle creates execution ids for each test case in test cycle).

High Level Steps:

1. Login to Jira using proper credentials.
2. Provide the Cycle Id OR ProjectId,Version & Cycle Id of the Test cycle to ZAPI api.
3. Update the Test case status in test cycle based on the execution id of each test case.

Example to Update Status of Test case execution Using ZAPI API:

This example shows how to consume ZAPI REST APIs from Java using REST Assured libraries.

How to run Chrome & Firefox Webdriver in Headless mode

Headless Browser! What does it mean.? As name suggests, It is a browser without head.i.e., without GUI. 

Headless Browsers are a kind of simulation programs of real browsers, but without GUI. Having said that, these may not be 100% equal to real browsers as they have its own limitations. Here are few popular headless browsers:

• Headless Chrome
• Headless Firefox
• HtmlUnit Driver
• PhantomJS
• SlimmerJS

Headless browsers come with its own Pros & Cons.

Pros:

• Improves the execution speed,in turn increases the performance.
• Can be executed on Browser-less setups like Servers(without browsers).
• Useful while scraping the web.
• Simulate multiple browser versions on a single machine.

Cons:

• It doesn't mimic the real user.
• Debugging is quite difficult.

Below snippet would show you, How to make Chrome & Firefox work in Headless mode.

Compare Images in Selenium Test

Visual / UI Validation of web pages across different browsers & devices is very important these days as more and more applications are getting web enabled. And most of the websites are eyeing on 'Responsive Web Design' to create a single website across multiple resolutions.

This type of testing is usually performed manually. However, now you can automate these type of tests to an extent using different image comparison libraries like Thumbnailator, Slenium-Shutterbug, ImageMagick Applitools Eyes etc.

Using these libraries, both Functional Testing & Visual Testing can go hand-in-hand without having to maintain separate set of scripts.
One such example using Image-comparison.jar based on Thumbnailator library is as below:

Steps:

Capture Full Screenshot in Selenium using Shutterbug

Selenium with the new upgrade 2.0 to 3.* had changed the way it works in the background. Starting with this upgrade, FirefoxDriver has stopped taking the full screenshot. There are many instances where user needs complete screenshot of the web page.
Many teams have worked on it & provided few open source libraries like AShot, ShutterBug etc.

Below code snippet would enable you to get the full screenshot using ShutterBug java library.

STEPS:

Capture Full Screenshot in Selenium using AShot

Selenium with the new upgrade 2.0 to 3.* had changed the way it works in the background. Starting with this upgrade, FirefoxDriver has stopped taking the full screenshot. There are many instances where user needs complete screenshot of the web page.
Many teams have worked on it & provided few open source libraries like AShot, ShutterBug etc.

Below code snippet would enable you to get the full screenshot using AShot java library.

STEPS:

Native App Automation using Appium

In the Mobile world, we see multiple types of apps namely Native App, Hybrid App & the browser based application, Web App. Also, we see multiple types of operating systems like Android, iOS, Windows, Firefox OS etc.

Automated testing of these many variation of apps, is very tricky & difficult task. Not all of them are supported by single automation tool. Having said that, Appium which is built over Selenium webdriver supports most of the OS & apps with different configurations.

In this article, you would know a way to automate the Native App using AndroidDriver.

Prerequisites:

1. Android SDK is downloaded and installed. (ANDROID_HOME env. variable should be set)
2. Either Emulator or  Real device (USB debugging enabled) is available.
3. Appium desktop app or non GUI package is installed. 

Steps:

1. Create a java project in eclipse 'AppiumMobileTesting'.
2. Create a class 'TestNativeApp' to add the automation code.
3. Connect your device(USB debugging enabled), to system's USB port & accept if any confirmation pop ups.

Web App Automation using Appium

In the Mobile world, we see multiple types of apps namely Native App, Hybrid App & the browser based application, Web App. Also, we see multiple types of operating systems like Android, iOS, Windows, Firefox OS etc.

Automated testing of these many variation of apps, is very tricky & difficult task. Not all of them are supported by single automation tool. Having said that, Appium which is built over Selenium webdriver supports most of the OS & apps with different configurations.

In this article, you would know a way to automate the mobile web app using RemoteWebdriver.

Prerequisites:

1. Android SDK is downloaded and installed. (ANDROID_HOME env. variable should be set)
2. Either Emulator or  Real device (USB debugging enabled) is available.
3. Appium desktop app or non GUI package is installed. 

Steps:

1. Create a java project in eclipse 'AppiumMobileTesting'.
2. Create a class 'TestWebApp' to add the automation code.

How to Avoid Certificate Popups to enable smooth Automation

Certificate Popups are common in a secured environment or while accessing the internal applications. It is a basically a way to authenticate the users accessing the application.

Yes, I know..! This is annoying for automation testers.
To overcome this annoying feature and yet without compromising on security, we have to make certain settings on each browser. These settings enable us to have smooth automation.

The fix is very simple, let the browser know which certificate to select as default one. Thus stops asking you for a client certificate. Follow the settings mentioned below based on the browser:

NOTE: 

1. This approach will help when you have no or only single certificate added to browser.
2. This approach is tested on windows OS.

Mozilla Firefox:

Step 1: Import Certificate

1. Open Firefox > Options > Certificates > View Certificates > Your Certificates.
2. Click Import & follow instructions to import a certificate(*.pfx)
3. After Import, Success Message will be popped up.

Selenium Tips and Tricks

Get the DOM Object into View:

At times, few actions are not performed on a DOM object until it is visible in the view-port. Below is the small code snippet to get the DOM object into view using WebDriver's 'JavascriptExecutor' interface. 

WebElement element = driver.findElement(By.id("someId"));
JavascriptExecutor js= (JavascriptExecutor) driver;
js.executeScript("arguments[0].scrollIntoView(true);",element);

OR

((JavascriptExecutor) driver).executeScript("document.getElementById('text123').scrollIntoView(true);");

Selenium WebDriver Wait Commands:

For any automation scripts to work without any sync issues with application, wait statements are very important. Selenium WebDriver provide different types of Wait conditions to make automation scripts effective.

Below are different types of waits & its usage.

Integration of Selenium + Sikuli

Sikuli is a tool using Visual Recognition capability which automates anything seen on the screen of your computer running Windows, Mac, few Linux/Unix OS systems. It uses Image recognition powered by OpenCV to identify the GUI components.

The version of Sikuli is named as 'SikuliX'. It also comes with basic text recognition (OCR) feature to search text within images.Besides locating images on screen, it can handle mouse & keyboard events to interact with GUI elements. 

Selenium for Angular JS Pages

Selenium + NgWebDriver  =  Better AngularJS Support

Angular JS is a opensource web application framework based on Java-script. Any Angular JS implemented web page would have additional custom tag attributs like ng-app, ng-model, ng-bind, ng-repeat etc.

Automating the AngularJS based web application is possible using Selenium. However in certain cases where the web pages are built extensively using AngularJS  resulting in complex DOM structure, it is difficult to locate DOM elements with simple XPaths & CSS selectors.In this case, Protractor (an E2E test framework for Angular Applications) built on Java script version of Selenium makes it easy for identifying the locators easily.

Voice Driven Testing using Perfecto Mobile

Most of the applications today started supporting voice commands either it is a desktop or a mobile.More & more functionalities are being supported by voice commands.

Testing these functionalities is very difficult as it requires audio files which are hard to edit / maintain them whenever required. Perfecto has come up with a better way to test these functionalities.

Steps to test Voice Driven Functionalities:

1. Enter required text
2. Convert Text to Audio
3. Inject the generated Audio to application

Seamless Integration of Selenium + AutoIT

AutoIT is a freeware available to automate Windows GUI by simulating the combinations of mouse movements & key strokes. AutoIT has its own basic syntax, editor & control identification mechanism. However, to achieve the code level integration one can use 'AutoItX', packaged with AutoIT. This supports access to AutoIT functions using COM objects.

Extract Text & Images from PDF

Extracting text from PDF is not as simple as reading text from a text(.txt) file. One has to use suitable libraries to extract data from any pdf file. For example, Below code snippet shows one of the way to extract text or images from PDF file using Apache's PDFBox libraries.

Visual / UI Test Automation using Selenium and Applitools Eyes

Visual / UI Testing is a methodology for validating the multiple visual aspects of Websites, Mobile apps and Desktop apps across multiple browser, resolution and OS combinations.

Visual / UI Testing is very important these days as more and more applications are getting web enabled. (i.e more of the standalone applications are being converted into websites) & most of the websites are eyeing on 'Responsive Web Design' to create a single website across multiple resolutions.The mobile apps designed for a platform has to support multiple screen resolutions or devices.

This type of testing is usually performed manually. However, now you can automate these type of tests using different automation tools available in the market. One such automation tool is Applitools Eyes which has the advantage of validating visual aspects within the existing Selenium scripts.

JDBC-ODBC Bridge support in Java 8

JDBC-ODBC Bridge offered lot of flexibility in connecting a data source. Many had used this bridge to connect & query MS Excel / MS Access as a database. Starting from Java 8, Oracle has removed this feature. This created a major blocker for many to upgrade to new Java version & many applications which were built based on this have failed.

As an interim solution, one can build an extension from jdbc classes of previous java version & add it to Java 8 extensions.

Follow the below steps to enable the JDBC-ODBC Bridge support in Java 8.