Dockerised Jmeter + Grafana + Influxdb

Jmeter is a widely used load testing tool from Apache. It is open source. Influxdb is a widely used time series database from InfluxData. It is open source. Grafana is a widely used graphing and analytics engine from Raintank inc. It is open source. Docker is a platform designed to containerize applications and services. It […]

Implementing Iteration Pacing in jMeter

Jmeter is an excellent tool for performance testing, especially given the price. It does however have a small number of limitations. One of those is the lack of a decent analysis, results and reporting engine – which I guess is why a lot of companies use Blazemeter as their testing component. Another limitation in my […]

Loadrunner Parameters in Blocks not updating?

I’ve encountered an issue recently where in a loadrunner script, I wanted to execute a number of requests of one type, followed by a number of requests of another type, followed by a number of requests of a third, and subsequently fourth and final type. The whole thing is four blocks looping in sequence inside […]

Centralised Testing Functions – Oversight and Governance

I’ve been working on a project over in Sweden, and have found it unlike anything I’ve ever seen before. Sadly, this is for all too familiar reasons. I work for an organisation with a small performance testing function centralised across all the projects that can be conjured up. As a result, there are often a […]

Python Functions

Declaration of functions in Python is nice and simple. def my_function(): print(“Hello From My Function!”) The call is also uncomplicated my_function() With Arguments def sum_two_numbers(a, b): return a + b sum_two_numbers(10,8)

First Steps with WATIR

July 21st, 2011 BlogAdmin

Watir is an open-source ruby-based browser automation tool. I first looked at it 6 months ago but with a new loadrunner role to involve myself with, I didn’t get far.

In any case, these are the steps I took to get started and to get things up and running.

  • 1. Install Ruby from ruby-lang.org. I chose version 1.9.2 as the most current stable build (despite a warning stating that Watir didn’t work on 1.9.2… we’ll see about that).
  • 2. Following the instructions here on watir.com, I updated the rubygems and installed Watir. I left the firefox addin as it’s not compatible with the latest version and I was loathe to downgrade my main browser. I have IE and Chrome too so I figured if one of them would run, that would be sufficient.
  • 3. Picked up the watir-webdriver whilst I was at it. Documentation can be found here at the excellent watirMelon site. I just used this command in a dos prompt:
  • gem install watir-webdriver
  • 4. I grabbed the chromedriver executable from here and dropped the executable in my ruby/bin/ folder as I know that’s in my path.
Read More »

Automated Testing Best Practices (Basic)

February 25th, 2011 BlogAdmin

I wanted to prepare an article that specifies the very basic best practices for Automated Testing, whether its functional or non-functional. It seems to me that all the tools, whether open-source or commercial allow for these essential practices, presumably because this is the way we should all be doing it… some of us aren’t, you know who you are and you’re very naughty. Also your testing isn’t as good as it should be…

This article is very much a work-in-progress as I braindump…

  1. All scripts should have a manual equivalent, because this is the foundation upon which the automation is built. It doesn’t have to be complicated, it doesn’t have to be as thorough as the automated script, but it does have to accurately specify the steps to follow.
  2. Some of the tools are best used in a record-edit-playback loop (Loadrunner for example). Others pretty much require you to edit them as you progress through your script, adding checkpoints and things as you go (QTP springs to mind). If its a pain to go and insert checks after the recording, that is just tough. Your test is only as good as the conditions it checks for.
  3. A good understanding of the tool is required before you begin any testing phase. Yes, we all have to start somewhere, and that somewhere should always be with training additional reading.
  4. All scripts have transactions, whether its a page click, a text entry, a mouseclick, whatever. Defining your test by these transactions makes debugging easier, makes the scripts more readable and ultimately makes you look professional. Alternatively, a multi-step business process should be mapped as a single logical transaction.
  5. All scripts should include checkpoints for every “transaction” regardless of whether there’s a condition defined. Again its not complicated but your checkpoints need to be sufficient that if something is broken AT ANY POINT your test can detect it.
    If you check for something unique to where you are in the AUT and set pass/fail criteria around that, that’s perfect. The sooner a test which has failed is known to have failed is the sooner the processor is moving on the next activity, saving time and money. And the sooner you’ll be debugging which is one of the fun bits in my opinion
Read More »

First Steps with QTP part 2

November 23rd, 2010 BlogAdmin

Ok, in place of the planned article on QTP automation which I promise I will get to eventually, this post will be about interacting with files through QTP.

Typically we’ll want to handle text files (for error logging and general message transmission) or Excel files. To begin with though, there are some standard declarations required.

Const ForReading = 1, ForWriting = 2, ForAppending = 8
Dim fso, f

Read More »

Basic Approaches to Performance Testing

November 3rd, 2010 BlogAdmin

In which I shall attempt to state that there are 3 and only 3 approaches to performance testing…

1. The comparative method.

We’ll assume you have an application called AUT v1.0, We’ll further assume you have a scenario built to test AUT v1.0 and to hit it sufficiently hard that it’s response times are less than perfect. Ideally it should be walking, not limping and certainly not sprinting along.

We’ll then suggest that v1.1 is coming out soon and that much of the functionality is unchanged. There is always new functionality, that’s the whole point, but it is the level of new-ness that dictates whether this approach can work.

Run your scenario against v1.0 as often as is necessary for consistent timings to be established. I maintain that 3 is the absolute minimum, and that more (often much more is better). Gather your results so that direct comparison between transaction times and runs is possible.

Run 1 2 3 4 5
Transaction A 1 1 1.1 2 1
Transaction B 2 2 2.2 4 2
Read More »

First Steps with QTP part 3

November 3rd, 2010 BlogAdmin

I’ve recently been tasked with inserting data into an 20-year-old 16-bit application to give us a benchmark data set for an updated alternative. In standard investment banking terms, there are deals, to which orders are attached, and there are orders which can have multiple tickets.

An API had been developed for inserting Orders and Tickets but Deals were still being entered manually. I am averse to doing anything repetitive manually, which is how and why I ended up automating in the first place.

As every techie knows all user-generated data starts life in a spreadsheet. It might get stored in a database eventually but for input and updates, it’s a spreadsheet.
Looking at the tools available I had a choice between QTP and writing a bespoke API.
Looking at the data, and armed with the knowledge that QTP and Excel will talk to each other with minimal fuss, I thought it might be worth giving QTP a trial run.
Looking at the application, I figured “there’s no chance of making it work, but I’m paid to solve problems so spend half-a-day on it, and see what happens”.

Read More »

Future Development

July 29th, 2010 BlogAdmin

I’ve been crazy-busy working, and have neglected the site for a while. I’m still crazy-busy but work on the site is continuing although most of it seems to be behind the scenes at the moment.

Currently I’m building a multi-form php application with a db to provide something to script against. Once that’s done I’ll be re-working and completing the article on beginning QTP.
I’ve a future article on end-to-end QTP automation to write, based on a project I’ve just completed at a former client. The software part of that is actually done although it needs to be anonymised. It uses scheduled tasks, vb scripts, and excel with vba to collate the results.

There is so much more to prepare and write posts on, I think it might finally be time to prepare a “to do” list.

1. Sort out the ad’s on this site so that they are pointing to genuine things…
2. Build some test app’s
3. Catch up on the selenium article

Read More »

Emulating Reality taken too far

February 22nd, 2010 BlogAdmin

So I’m working at a client who use, alongside the usual mix of workstations, a hand-held device running on Windows CE. They have 800 of these across the business (though most seem to be sitting with the engineers being repaired, they’re not particularly robust, and don’t react well to percussive maintenance).
To provide a little background, I’m here because the incumbent test/dev team have been advised their services are no longer required. The current Performance team have identified a number of issues, many of which are still outstanding and the focus has now switched to the hand-held terminals (HHT’s).
Long before I arrived, it was decided that performance testing is required for these devices (I agree) and that the best way would be a test script and a stopwatch. To say I disagree would be understating it on a massive level losartan 100mg tab.

I’m all for emulating the real business processes, its just that I know if you “measure” performance with a stopwatch, then you can cease calling it perfomance testing and start calling it “meaningless arbitrary timekeeping signifying nothing”. (Told you I disagreed).
Here at AutomationSolutions, I like to think we look for the better way, I certainly have no interest in wasting my afternoon or my clients money sitting about tapping buttons on an oversized calculator. I’m going to do some research into emulating hand-held operating systems. I’m going to do some research into capturing wireless packets, and simulating that process. I’m going to see if I can identify any way to bring a scientific method to this testing, and if I cannot I’m going to request that we move it immediately out-of-scope or at least admit that we’re more interested in ticking boxes than in accurately measuring performance.

(While I’m thinking about it, I might add a rant tag and some pretty CSS to the site. I suspect this will not be the last word on this topic).

Read More »

First Steps with QTP

February 2nd, 2010 BlogAdmin

Matt:
So I’ve got around 10 years experience with Loadrunner, am well-versed with Winrunner, can code in half a dozen languages and still I never got around to learning QTP. Well now I have no choice since we’re using it at my latest client site.
The aim of this article is to provide guidance in QTP, how to start, what to look at, how to record a simple script and parameterise it. The essentials, basically.

I recorded a simple script going from Google to AS.org, inserted one checkpoint as I was doing so. It seems that the insertion of checkpoint is best done at record-time. I’m not clear yet on how to insert one afterwards…

My script looks like:
Browser("Google").Page("Google").WebEdit("q") _
.Set "automationsolutions.org"
Browser("Google").Page("Google").WebEdit("q").Submit
Browser("Google").Page("automationsolutions.org") _
.Link("Automation Solutions").Click
Browser("Google").Page("Automation Solutions").Check _
CheckPoint("Automation Solutions")

It’s very readable I suppose and handily is built around Visual Basic, one of the languages I am familiar with.

Standard stuff like Browser, Page, WebEdit, (“q”, by the way is the name of the textbox object on Google’s homepage. Check with Firebug if you want confirmation) Link and Click are container objects or Actions for specific data, but I don’t think “Google”, “Automation Solutions” or that CheckPoint tell me enough about what’s going on – is “Google” www.google.co.uk?, is “Automation Solutions” a link to the site or a string to be pushed to the browser? And what is that checkpoint checking for?

The Object Repository

Right-Clicking on the keyword “Google” brings up a menu with Object Properties uppermost in the list. Clicking on that brings up this:

!img_placeholder!

I’m not happy with the default properties and although QTP might feel they’re sufficient to identify the object, I want more information so I click the green +.
I can see URL is available for selection (as openurl) so I select that. It has no effect on the code, but in the event I need to debug, I can see how knowing the url could be useful.

!img_placeholder!

Right-clicking on the checkpoint brings up:

!img_placeholder!

It shows what text is being searched for, and it also shows that it can be parameterised. Having added it to the global table, I am forced to wonder how best to data-drive a QTP test.

Conclusion

Using google to search for a data string, find a link based on that string and then check that its on the page we expected might seem trivial but it’s allowed us to see the basics of QTP

Read More »

First Steps with Selenium

January 14th, 2010 BlogAdmin

Matt:
So, Selenium is an open-source toolset for testing web-based applications. It can be found here.
I started working with Selenium recently and I though it’d be an interesting post to document how I went from novice to competent user.

One of the interesting features of Selenium is that is can be used for performance testing. It’s my experience that performance testing is a costly business, so to see an open-source tool that can do it is rare.

Lets begin by looking at what Selenium is comprised of:

  • Selenium IDE – the scripting environment.
  • Selenium RC – allows further scripting and execution with variable data
  • Selenium Grid – Allows for multiple tests (or instances of the same test) to be executed.

It seems to me that the IDE and RC are doing the job of Loadrunners VUGen and Grid is handling the tasks of the controller. More on this in a future post.

At the time of writing, Selenium IDE is available as an add-on for the Firefox browser only, though the tests can be replayed (via RC) on Firefox, IE and Chrome.

Installation

Installation is made easy with the availability of 3 zipped packages (core, rc and grid) and the inclusion of the IDE as a firefox extension, (xpi) which will automatically install into firefox. The folks over at Selenium HQ also recommend getting an element inspector for your browser. As all the Selenium scripting I’ve done is in Firefox, Firebug is highly recommended, and freely available as an add-on or from http://getfirebug.com/.

Start by installing the IDE into firefox, and unpack the zip files somewhere sensible.

Hello Selenium!

Under the tools menu, there should now be an entry for Selenium IDE that leads us to this:

Selenium IDE

More to come, this article is going to be much bigger than I first thought…

Read More »
Newer Entries »
Powered by WordPress and ThemeMag