Friday 4 March 2022

How to Deploy Selenium grid on AWS / Amazon EKS

  


                                    Selenium Grid is good for parallel execution but maintenance is a nightmare in an era where you see a frequent upgrades to browsers and corresponding drivers. No sooner the usage of automation framework / selenium grid increases, scalability and maintenance becomes a challenge. To address such issues, we do have solutions based on dockers, docker swarm etc. Having said that, there are some caveats in scaling, managing container health etc.

Below solution would try to address most of them. Major chunks of the solution include Selenium, Zalenium, Docker, Kubernetes and Amazon EKS.

This article would outline the process of deploying Selenium grid(Zalenium) on AWS (Amazon EKS) using Kubernetes and Helm.

What do we achieve with this setup..?

  • Scalability: EKS can scale the nodes and pods as per the given configuration.
  • Visibility: Zalenium provides a feature to view the live executions on the containers.
  • Availability: Amazon EKS cluster makes selenium grid available all the time.
  • Maintenance: Low maintenance as the containers are destroyed after each execution.

Pre-requisites:

  • An active Amazon AWS account.
  • IAM user is created in AWS account
  • AWS CLI is connected to AWS account providing the user credentials using local powershell or any terminal
                                                                OR
  • Use AWS cloudshell which is automatically connected to logged in account.
  • Install AWS CLI (for local terminal), kubectl, helm in the given order.

Lets Get Started!

Once the above pre-requisites are met, next task to deploy any application on kubernetes is to create a kubernetes cluster. There are different ways to create a cluster on AWS, I'll brief couple of ways to achieve the same.

First, Create cluster from AWS GUI.

1. Create master node or cluster 
  • Open Amazon EKS console
  • Choose Create Cluster
  • Provide details like cluster name, k8s version, role
  • Select VPC, security groups, endpoint access
  • Further steps as shown on GUI which will make 'master' ready.
2. Create worker nodes and connect to the above created cluster.
  • Create Node Group (Amazon EC2) instances.
  • Choose the cluster, to which the above node group should get attached.
  • Select security group, resources etc.,
  • Define min and max number no. of nodes.
Sounds complex?. No issues, there is another simple and efficient way to make the whole process look simple.

Second, Create cluster using eksctl (The official CLI for Amazon EKS)

The above complex task can be achieved with a single command.

Wednesday 23 February 2022

How to Deploy Selenium grid on Google Cloud Platform using Kubernetes and Helm

                                          



                                        It is a open secret that Selenium Grid maintenance is a nightmare in an era where you see a frequent upgrades to browsers and corresponding drivers. No sooner the usage of automation framework / selenium grid increases, scalability and maintenance becomes a challenge. To address such issues, we do have solutions based on dockers, docker swarm etc. Having said that, there are some caveats in scaling, managing container health etc.

Below solution would try to address most of them. Major chunks of the solution include Selenium, Zalenium, Docker, Kubernetes and Google Cloud Platform.

This article would outline the process of deploying Selenium grid(Zalenium) on Google cloud platform using Kubernetes and Helm.

What do we achieve with this setup..?

  • Scalability: GCP/GKE can scale the nodes and pods as per the given configuration.
  • Visibility: Zalenium provides a feature to view the live executions on the containers.
  • Availability: GKE kubernetes cluster makes selenium grid available all the time.
  • Maintenance: Low maintenance as the containers are destroyed after each execution.

Pre-requisites:

  • An active Google Cloud Platform account
  • Enable Kubernetes engine by setting up a Billing Account.

Friday 18 February 2022

How to setup Selenium Grid using Docker Desktop - Windows

                            Selenium Grid has made the test automation execution much faster and smarter. The excessive usage of selenium grid has its own hitches like too much of system resource utilization, all browsers to be installed on the system etc. 

With the introduction of docker images for selenium, made the life of testers lot easier. Out of multiple ways to bring up selenium grid with docker, I would choose the simple & quick way to bring up using docker-compose.

Pre-requisite: Docker desktop for windows is installed.


Steps to bring up Selenium Grid:

  • Pull required version of selenium hub and nodes using the command "docker pull" as below

         docker pull selenium/node-chrome-debug:3.141.59 

  • Create a yaml file with the below docker-compose instructions.
docker-compose file

  • run the command 
         docker-compose -f docker-compose.yml up -d

Thursday 23 September 2021

Dockers and Kubernetes Cheat sheet

Below are list of frequently used commands. This would come very handy while working with docker & kubernetes.

Docker & Kubernetes Commands 



Thursday 12 August 2021

How to create and execute Jmeter script using Java


Performance Test!. When we say this term, one of the first things that gets into our mind is 'Jmeter'.

           Jmeter is the go-to tool for the performance testing needs in open source community. It is built completely using Java, designed to perform load test and measure performance. It can simulate load on a server, group of servers, network to check the threshold limit and analyze performance under different types of loads. Vast list of plug-ins which extends jmeter capabilities and making it handle most of the performance test requirements.

Mostly the Jmeter GUI is used to create the scripts, configure the users, capture other details and execute the scripts. But, when it comes to integrate the performance tests to code driven automation frameworks, one has to switch to Jmeter GUI to create scripts & fallback to framework to execute the jmeter scripts. In order to make the integration seamless, Jmeter scripts can be created & executed during runtime using code driven framework. Below code snippet would lead you to achieve the same.

Below snippet will let you create a jmeter script (jmx) for a webservice by adding minimal elements to test plan. Typical hierarchy of a web request in a jmx would be as below:

 Test Plan 
à Thread Group à Sampler à Assertions à Listeners 

Steps:

  • Create a maven project through eclipse or any IDE.
  • Add the below Jmeter dependencies in your POM file:
    • ApacheJMeter_core
    • ApacheJMeter_components
    • ApacheJMeter_http
    • jorphan
    • ApacheJMeter_java
  • Create a class named "APITest" and copy the below code snippet
  • Change the service details & file locations accordingly.
  • DONE..! You are all set to create and run the jmeter script from java.

import java.io.File;
import java.io.FileOutputStream;
import org.apache.commons.io.FileUtils;
import org.apache.jmeter.config.Arguments;
import org.apache.jmeter.config.gui.ArgumentsPanel;
import org.apache.jmeter.control.LoopController;
import org.apache.jmeter.control.gui.LoopControlPanel;
import org.apache.jmeter.control.gui.TestPlanGui;
import org.apache.jmeter.engine.StandardJMeterEngine;
import org.apache.jmeter.protocol.http.control.gui.HttpTestSampleGui;
import org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy;
import org.apache.jmeter.report.config.ConfigurationException;
import org.apache.jmeter.report.dashboard.ReportGenerator;
import org.apache.jmeter.reporters.ResultCollector;
import org.apache.jmeter.reporters.Summariser;
import org.apache.jmeter.save.SaveService;
import org.apache.jmeter.testelement.TestElement;
import org.apache.jmeter.testelement.TestPlan;
import org.apache.jmeter.threads.ThreadGroup;
import org.apache.jmeter.threads.gui.ThreadGroupGui;
import org.apache.jmeter.util.JMeterUtils;
import org.apache.jorphan.collections.HashTree;

public class APITest {
	
    public void createAndExecute() {
		File jmeterHome = new File("C:/apache-jmeter");
		try {
			if (jmeterHome.exists()) {
				// JMeter Engine
				StandardJMeterEngine jmeter = new StandardJMeterEngine();
				setconfig(jmeterHome, "./htmlreportsdir");

				// JMeter Test Plan, basic all u JOrphan HashTree
				HashTree testPlanTree = new HashTree();
				// HTTP Sampler
				HTTPSamplerProxy httpSampler = new HTTPSamplerProxy();
				httpSampler.setName("HTTP Sampler");
				httpSampler.setProtocol("https");
		        httpSampler.setDomain("testjmeter.com");
		        httpSampler.setPort(8080);
		        httpSampler.setPath("/getservicepath");
		        httpSampler.setMethod("GET");
		        httpSampler.setProperty(TestElement.TEST_CLASS, HTTPSamplerProxy.class.getName());
		        httpSampler.setProperty(TestElement.GUI_CLASS, HttpTestSampleGui.class.getName());
		        httpSampler.setEnabled(true);
		        
		        httpSampler.addArgument("Arg1","val1");//Arguments
		        httpSampler.addArgument("Arg1","val1");//Arguments
		        
		        httpSampler.addNonEncodedArgument("", "serviceBody", "=");//payload
	        	httpSampler.setPostBodyRaw(true);
	        	
				//Loop Controller
				LoopController loopController = new LoopController();
		        loopController.setLoops(1);
		        loopController.setFirst(true);
		        loopController.setProperty(TestElement.TEST_CLASS, LoopController.class.getName());
		        loopController.setProperty(TestElement.GUI_CLASS, LoopControlPanel.class.getName());
		        loopController.initialize();
		        
				//Thread Group
				ThreadGroup threadGroup = new ThreadGroup();
		        threadGroup.setName("API Thread Group");
				threadGroup.setNumThreads(20); //Users
		        threadGroup.setRampUp(10); //Seconds
		        threadGroup.setSamplerController(loopController);
		        threadGroup.setProperty(TestElement.TEST_CLASS, ThreadGroup.class.getName());
		        threadGroup.setProperty(TestElement.GUI_CLASS, ThreadGroupGui.class.getName());
		        
		        threadGroup.setIsSameUserOnNextIteration(true);
		        threadGroup.setScheduler(false);
				
				//Test Plan
				TestPlan testPlan = new TestPlan("JMeter Script From Java Code");
		        testPlan.setProperty(TestElement.TEST_CLASS, TestPlan.class.getName());
		        testPlan.setProperty(TestElement.GUI_CLASS, TestPlanGui.class.getName());
		        testPlan.setUserDefinedVariables((Arguments) new ArgumentsPanel().createTestElement());
				
				//Construct Test Plan from previously initialized elements
				testPlanTree.add(testPlan);
				HashTree threadGroupHashTree = testPlanTree.add(testPlan, threadGroup);
				threadGroupHashTree.add(httpSampler);

				// save generated test plan to JMeter's .jmx file format
				String jmxFilePath = "./jmxfiles/TestAPI.jmx";
				SaveService.saveTree(testPlanTree, new FileOutputStream(jmxFilePath));

				// add Summarizer output to get test progress in stdout like:
				String jtlFilePath = ".jtlFiles/TestAPI.jtl";
				ReportGenerator reportGenerator = setReportInfo(testPlanTree, jtlFilePath);
				
				//Run Test Plan
				jmeter.configure(testPlanTree);
		        jmeter.run();
		        
				// Report Generator
				FileUtils.deleteDirectory(new File("./htmlreportsdir"));// delete old report
				FileUtils.deleteDirectory(new File("./reportsdir"));// delete old report
				reportGenerator.generate();

				System.out.println("Test completed. See " + jtlFilePath + " file for results");
				System.out.println("JMeter .jmx script is available at " + jmxFilePath);
				
			} else {
				System.out.println("Jmeter Home not found..");
			}

		} catch (Exception e) {
			System.out.println(e.getMessage());
		}
	}
    
	public void setconfig(File jmeterHome,String htmlrepDir){
		File jmeterProperties = new File(jmeterHome.getPath() +"/bin/jmeter.properties");
        //JMeter initialization (properties, log levels, locale, etc)
        JMeterUtils.setJMeterHome(jmeterHome.getPath());
        JMeterUtils.loadJMeterProperties(jmeterProperties.getPath());
        JMeterUtils.initLocale();
        
        //Set directory for HTML report
        JMeterUtils.setProperty("jmeter.reportgenerator.exporter.html.property.output_dir",htmlrepDir);
	}
	
	
	public ReportGenerator setReportInfo(HashTree testPlanTree,String jtlFilePath) throws ConfigurationException{
		Summariser summer = null;
        String summariserName = JMeterUtils.getPropDefault("summariser.name", "summary");
        if (summariserName.length() > 0) {
            summer = new Summariser(summariserName);
        }
        
        // Store execution results into a .jtl file
        File logFile = new File(jtlFilePath);
        //delete log file if exists
        if (logFile.exists()){
            boolean delete = logFile.delete();
            System.out.println("Jtl deleted: " + delete);
        }
        
        //Summary Report
        ResultCollector logger = new ResultCollector(summer);
        logger.setEnabled(true);
        logger.setFilename(logFile.getPath());
        //creating ReportGenerator for creating HTML report
        ReportGenerator reportGenerator = new ReportGenerator(jtlFilePath, logger); 
        testPlanTree.add(testPlanTree.getArray()[0], logger);
         
	return reportGenerator;
	}
		
}

Friday 4 June 2021

How to establish a connection to Remote Host & Copy file

Many times we do get the requirement, to connect a remote server & perform operations like copying a file from local to remote, vice versa and many others. But, have you ever thought how do we do that through java.? Yes, We can. Java has multiple libraries which can perform this task. One such java library which can perform this task is Jcraft's 'JSch (Java Secure Channel)' . 

Below example will let you know how to connect a remote host & perform copy operations.


public void copyFiletoRemotehost(String username,String pwd,String[] remotehosts, String port) {
        Session jschSession = null;
        int SESSION_TIMEOUT = 10000;
        final int CHANNEL_TIMEOUT = 5000;
        String REMOTE_HOST = "";
        int REMOTE_PORT = 22;
		
	String lFile = "C:/localhost/local.txt";
        String rFile = "/localtoRemote.txt";
        try {
            JSch jsch = new JSch();
            for(int i=0; i < remotehosts.length; i++) {
            	REMOTE_HOST = remotehosts[i].toString();
            	jschSession = jsch.getSession(username, REMOTE_HOST, REMOTE_PORT);

                //Remove known_hosts requirement
                java.util.Properties config = new java.util.Properties();
                config.put("StrictHostKeyChecking", "no");
                jschSession.setConfig(config);
                
                //authenticate using password
                jschSession.setPassword(pwd);
                jschSession.connect(SESSION_TIMEOUT);
                Channel sftp = jschSession.openChannel("sftp");
                sftp.connect(CHANNEL_TIMEOUT);

                ChannelSftp channelSftp = (ChannelSftp) sftp;
                channelSftp.put(lFile, rFile);
                channelSftp.exit();
                
                //Copy from one location to another with remote server
                copyFile(jschSession);
                jschSession.disconnect();
            }
        } catch (JSchException | SftpException e) {
            e.printStackTrace();
        } finally {
            if (jschSession != null) {
                jschSession.disconnect();
            }
        }
    }
    
    /**
    * This method will let you copy from one folder to another within the remote server
    */
    public static void copyFile(Session jschSession) throws JSchException {
	String cmd1 = "cp C:/username/localtoRemote.txt  C:/anotherfolder/localtoRemote.txt";
	Channel exec = jschSession.openChannel("exec");
		
	ChannelExec channelExec = (ChannelExec) exec;
	channelExec.setCommand(cmd1);
	exec.connect(CHANNEL_TIMEOUT);
	channelExec.setErrStream(System.err);
   }

References:
http://www.jcraft.com/jsch/

Saturday 18 July 2020

Hybrid App Automation using Appium

In the Mobile world, we see multiple types of apps namely Native App, Hybrid App & the browser based application, Web App. Also, we see multiple types of operating systems like Android, iOS, Windows, Firefox OS etc.

Automated testing of these many variation of apps, is very tricky & difficult task. Not all of them are supported by single automation tool. Having said that, Appium which is built over Selenium webdriver supports most of the OS & apps with various configurations.

In this article, you would know a way to automate the Hybrid App using AndroidDriver.

Prerequisites:

  1. Android SDK is downloaded and installed. (ANDROID_HOME env. variable should be set)
  2. Either Emulator or  Real device (USB debugging enabled) is available.
  3. Appium desktop app or non GUI package is installed. 

Steps:

  1. Create a java project in eclipse 'AppiumMobileTesting'.
  2. Create a class 'TestHybridApp.java'.
  3. Connect your device(USB debugging enabled), to system's USB port & accept if any confirmation pop ups.
  4. Start Appium Server. Once the device is connected, open the 'Appium Inspector Session' providing the suitable capabilities.
  5. Open the Hybrid app in the device and you see the same screen on inspector session.
  6. Click on the required control in inspector session & then the related locator details are shown on the right hand side in appium inspector.

Friday 6 March 2020

How to query or filter Json using RestAssured JsonPath


REST Assured(RA) is a framework built on Java to test the REST services. It supports any HTTP method but has explicit support for GET, POST, PUT, DELETE, HEAD, OPTIONS, PATCH and includes specifying & validating like Headers, Cookies etc.

For validating and querying the Webservice responses, it has couple of libraries 'JsonPath' and 'XMLPath' for parsing Json and XML responses accordingly.

Even-though we know that, we have to use these libraries but most of the times we get into situations where we forget or we are not sure of the actual syntax. Here are list of few scenarios and it's JsonPath syntax (XMLPath syntax are almost similar) for your reference.

All the Jsonpath examples in this post, use the following Json:

 
{
   "store":{
      "book":[
         {
            "category":"reference",
            "author":"Nigel Rees",
            "title":"Sayings of the Century",
            "price":8.95
         },
         {
            "category":"fiction",
            "author":"Evelyn Waugh",
            "title":"Sword of Honour",
            "price":12.99
         },
         {
            "category":"horror",
            "author":"Herman Melville",
            "title":"Moby Dick",
            "isbn":"0-553-21311-3",
            "price":8.99
         },
         {
            "category":"fiction",
            "author":"J. R. R. Tolkien",
            "title":"The Lord of the Rings",
            "isbn":"0-395-19395-8",
            "price":22.99
         }
      ],
      "bicycle":{
         "color":"red",
         "price":19.95
      }
   },
   "city":"Bangalore"
}



Following examples let you know,how to effectively use the JsonPath to extract the required values from the RESTful json in different scenarios:

Thursday 18 April 2019

How to send Cookies from Web(GUI) to Web Service


Cookies: Browser Cookies are also referred as Internet Cookie,Web Cookie or HTTP Cookie. A cookie is a "small piece of data" sent from the website to the user's device while the user is accessing the website.
There are multiple ways these cookies are being used in modern world like authentication, browser state information, storing user personal information like name, address etc., tracking the user activity on web, etc., Different types of cookies satisfy different needs.

In certain cases, one might have to test web services which needs cookies to be passed as part of request. One way to address this, is to combine Web application plus web service test steps. 

Below code snippet would open the web application, capture the cookies stored by the application and pass it to web service using Rest Assured.


public class CookiesToWebServices {
static WebDriver driver;
static String driverPath = "C:/Drivers/chromedriver.exe";

public static void main(String[] args) {

System.setProperty("webdriver.chrome.driver",driverPath);
ArrayList cookieList = new ArrayList();

String url = "www.testurl.com/testpath/id";
HashMap headermap = new HashMap<>();
headermap.put("Content-Type", "application/json");
headermap.put(....

HashMap queryParammap = new HashMap<>();
queryParammap.put("id", "12345");
queryParammap.put(....

Monday 31 December 2018

How to Connect Zephyr For Jira Programmatically (ZAPI)

Jira by default is a 'Defect Management Tool'. However, available add-on for Jira make it much more capable than just defect Management tool. Couple of such add-ons like 'Zephyr For Jira' & 'ZAPI' (API to access data from 'Zephyr For Jira' programmaticallymake Jira capable for 'Test Management'. Both of these add-ons have different version for Server & Cloud accordingly. Let us look at, how do we work with ZAPI server version.


Use Case: Whenever the automation is run, it has to update the status of execution in test cycle.

Pre-requisite: Manual Tests are created in Jira with Issue Type - Test & Test cycle is created with tests added into it (Adding tests into test cycle creates execution ids for each test case in test cycle).

High Level Steps:
  1. Login to Jira using proper credentials.
  2. Provide the Cycle Id OR ProjectId,Version & Cycle Id of the Test cycle to ZAPI api.
  3. Update the Test case status in test cycle based on the execution id of each test case.

Example to Update Status of Test case execution Using ZAPI API:


This example shows how to consume ZAPI REST APIs from Java using REST Assured libraries.

Friday 14 September 2018

How to run Chrome & Firefox Webdriver in Headless mode

Headless Browser! What does it mean.? As name suggests, It is a browser without head.i.e., without GUI. 

Headless Browsers are a kind of simulation programs of real browsers, but without GUI. Having said that, these may not be 100% equal to real browsers as they have its own limitations. Here are few popular headless browsers:

  • Headless Chrome
  • Headless Firefox
  • HtmlUnit Driver
  • PhantomJS
  • SlimmerJS

Headless browsers come with its own Pros & Cons.
Pros:

  • Improves the execution speed,in turn increases the performance.
  • Can be executed on Browser-less setups like Servers(without browsers).
  • Useful while scraping the web.
  • Simulate multiple browser versions on a single machine.

Cons:

  • It doesn't mimic the real user.
  • Debugging is quite difficult.

Below snippet would show you, How to make Chrome & Firefox work in Headless mode.

Friday 7 September 2018

Compare Images in Selenium Test

Visual / UI Validation of web pages across different browsers & devices is very important these days as more and more applications are getting web enabled. And most of the websites are eyeing on 'Responsive Web Design' to create a single website across multiple resolutions.

This type of testing is usually performed manually. However, now you can automate these type of tests to an extent using different image comparison libraries like Thumbnailator, Slenium-Shutterbug, ImageMagick Applitools Eyes etc.

Using these libraries, both Functional Testing & Visual Testing can go hand-in-hand without having to maintain separate set of scripts.
One such example using Image-comparison.jar based on Thumbnailator library is as below:


Steps:


Wednesday 5 September 2018

Capture Full Screenshot in Selenium using Shutterbug


Selenium with the new upgrade 2.0 to 3.* had changed the way it works in the background. Starting with this upgrade, FirefoxDriver has stopped taking the full screenshot. There are many instances where user needs complete screenshot of the web page.
Many teams have worked on it & provided few open source libraries like AShot, ShutterBug etc.

Below code snippet would enable you to get the full screenshot using ShutterBug java library.

STEPS:


Capture Full Screenshot in Selenium using AShot


Selenium with the new upgrade 2.0 to 3.* had changed the way it works in the background. Starting with this upgrade, FirefoxDriver has stopped taking the full screenshot. There are many instances where user needs complete screenshot of the web page.
Many teams have worked on it & provided few open source libraries like AShot, ShutterBug etc.

Below code snippet would enable you to get the full screenshot using AShot java library.

STEPS:


Wednesday 25 July 2018

Native App Automation using Appium

In the Mobile world, we see multiple types of apps namely Native App, Hybrid App & the browser based application, Web App. Also, we see multiple types of operating systems like Android, iOS, Windows, Firefox OS etc.

Automated testing of these many variation of apps, is very tricky & difficult task. Not all of them are supported by single automation tool. Having said that, Appium which is built over Selenium webdriver supports most of the OS & apps with different configurations.

In this article, you would know a way to automate the Native App using AndroidDriver.

Prerequisites:

  1. Android SDK is downloaded and installed. (ANDROID_HOME env. variable should be set)
  2. Either Emulator or  Real device (USB debugging enabled) is available.
  3. Appium desktop app or non GUI package is installed. 

Steps:

  1. Create a java project in eclipse 'AppiumMobileTesting'.
  2. Create a class 'TestNativeApp' to add the automation code.
  3. Connect your device(USB debugging enabled), to system's USB port & accept if any confirmation pop ups.