So when I started at my current company I started in support calling and talking to clients and fixing various issues. I worked my way up to taking tougher clients then eventually moved over to become a Quality Assurance Analyst. When I first started we were 5 testers who tested the whole software as a whole. No one was a specialist in one area and no one had any say in any part of the software. We were even left out of important meetings and basically were the bottom of the barrel but we put in our bugs where ever we found them. Shortly after I started there was a plan to move more towards what Spotify does and have a bunch of groups that were made of programers, testers, and something called a Product Owner. Since at the time we were short on QA a lot of us were in multiple squads. As we kept with this new set up more QA was needed and we now number a dozen.
Now with the change QA has a voice in their part of the software. We specialize in testing only certain parts of the software and work closely with the developers and product owner to put out a better product. This has changed for the better as now QA has a voice in each squad and everyone is a equal in the squad. We get a say in fixes and designs. This has been a definite improvement to how things were before. However some old habits are still creeping up when admins take over our squad for random new projects and sometimes QA is left out again. This is happening less but over the years I hope it will fade. We are now getting more control with the product owners getting more responsibilities. So far agile is working really well for us but like most companies we have yet to embrace the full agileness.
Coming Soon…
Currently I am working on a podcast site with a fellow tester of mine called Test Reactor. This is going to be a podcast by geeks for geeks. There will be a segment every podcast about Software testing as well. This is to show the world that Geeks work too. I hope it can be under way soon and will update you when we get our first podcast going.
<div class="paragraph" style="text-align:left;">So a big debate that is heard a lot is whether to use Exploratory testing or Scripted. Exploratory testing is where you go in and you search the app specifically looking for bugs. You are going to search anywhere and everywhere based on the design. You will also try things out side the design. Basically think of it as you are on a safari hunting for bugs. You are exploring new territory looking for things that are out of the ordinary. This is a very widely used way of testing and also very popular.</div> <div><div class="wsite-image wsite-image-border-thin " style="padding-top:10px;padding-bottom:10px;margin-left:0;margin-right:0;text-align:center">
What is Scripted testing? Well script testing you have a set script based on the design and how QA will be testing the software. With this you can get through and make sure certain points are hit. If something needs to be tested then you add it to the script. This is what you use a test plan for and you can read more about that in my first couple blogs. Think of it as a play and if it does not go a written put in a bug.
What are the benefits of using both? There is more benefits to using both then there are to not using one or the other. Scripted is really good and getting a through test but may miss something. Exploratory will help you find what you are missing but is no where near as through. This in hopes makes our below pie chart have less not found.
A while ago I read an article on the future of agile testing. When reading through I saw this thing called Pair Testing so I looked it up and found a good article at this link. This is when the Developer and QA work together testing. This way they will hit things normally they would not hit separately plus things can be fixed right there on the spot or you can at least you can get the information the developer needs to fix the issue. This will also help with QA, Dev relationship as they will work closely and see how each other operates. This we be a good idea for doing 60-90 minutes a week.
Not only is this a good idea with a dev and a QA working together we have found within our own QA that 2 QA testing together you will hit issues neither of you have hit together. This is because you are both different testers and will test differently and so together you will try things neither have thought of. Also you will be able to track down more detail since you both have different ways to figure out the issue. You can also write a very thorough Test Plan if 2 of you work together to hit all the test cases. So do I feel Paired Testing is the wave of the future? I believe I think it is.
<div class="paragraph" style="text-align:left;">When working in QA you will receive tickets submitted by clients, support, or various other people. It is your job to go through them using these 5 steps.</div> <div class="paragraph" style="text-align:center;"><span style=""><strong>Receive (the ticket)</strong></span></div> <div><div class="wsite-image wsite-image-border-thin " style="padding-top:10px;padding-bottom:10px;margin-left:0;margin-right:0;text-align:center">
Verify (that it is a problem)
Justify the Problem (is it user error over defect)
Sometimes when testing software you will come across the dreaded white whale. In my case this was a crash that only I have reproduced, but our clients were hitting it quiet often. Every time I hit the crash it seemed something different caused it, and was only reproducible 12% of the time but I had clients hitting the crash left and right. So “Call me Ishmael” I had to hunt this white whale down. I spent 3 weeks hunting using all kinds of tools and using resources just like Moby Dick did to hunt down the white whale, to no avail. Then one day when hunting a run of the mill crash that I knew how to reproduce I hit the whale. It turns out I was running all these tools to catch the run of the mill crash that it slowed down the software enough to allow me to hit the white whale. After I finally got all the info in it was a line change of code and the white whale was no longer a bother.
As the picture states there are several white whales out there in the testing world. You will run into them and you will hunt them down. Just don’t get to frustrated when you can’t, because sometimes you will run across and figure it out without even realizing that is what you where doing. Remember to think outside the box, and use all the tricks you can. As testers get more tricky, so will the bugs.
What does it take to write a good bug? There is a lot of time where a bug comes in from a somewhere else and you have no idea what they are talking about or what was tried. How can you reproduce it if you are unsure about where the bug is?
Here are some must haves for a bug.
Short Description/Title – This is where people will look to see a summery of the bug, it is very important as people will look here to find the bug when searching.
Description – Now this is the brass tax. This is the meat. Here you will enter all you can, and give a lot of detail. The more detail the better. But keep in mind that developers will only pay attention to the first paragraph or so. So remember if you have multiple issues make multiple tickets instead of just one giant ticket. Also give any workarounds or steps you tried so that the engineer will know.
Expected Results – This is where you explain what you were expecting to happen vs what did.
Steps to Reproduce – How did you recreate this so we can? This tells us the steps you took to get to the bug, and is very important.
The rest are going to be drop downs based on the software. Make sure you select the appropriate one.
Attachments are a huge help. Even the worse bug can be trumped by a great video or screenshot. I have gotten bugs where I had no clue what they were talking about until the video played. Once you get good at putting in usability and functional bugs you can step using some awesome tools. We can go over that in a later blog.
Damn bugs was founded by Crowd Soured Testing as a online tracker for bugs. You start by creating a Organization and then you have projects. The projects can then have all types of bugs. This is mostly a web based one so there are web browser extensions to put in bugs for websites, which is really useful. This can also be used as a stand alone. There are different statuses for project management, different types to tell you if it is design or dev or any other kind of ticket. You can add other team members with different jobs. They are also really good about listening to feedback and getting back to you. There is sorting, searching, and color coding. Also the bug reporting works well and prompts good reporting to anyone who is not use to bug reporting. All in all, for where it is at, this is a great system, and I can’t wait to see what it has in store for the future. Also one of my suggestions I made were to add test plan integration. The response I got was they had bought Overlook and were working on it… yes many awesome things for this sites future.
Overlook was currently bought by Crowd Sourced Testing and is a test plan website that will be integrated with Damn Bugs. Overlook is about making a simple test plan. The test plans for this were also more designed around testing web based applications but again you can use it for other plans. First you will create a project within your account. You can than manage a team, and invite new testers and create new test plans for the project. From the first page you are also able to edit, see results, and start the test. When you create the test you are able to do it line by line with no formatting. You can pass, fail, and add notes to failures. When tied to a site you can run the test plan at the top while having the webpage below. It is very simple and could use a lot of growth. It has the same mechanism Damn Bugs has for feedback but currently that is not heavily watched. My guess is it will grow as Crowd Soured Testing takes a bigger look at it.
Trello helps managing projects, it is useful for several things that a tester can use. Trello you can make list, add cards to the list, and move them around like your very own work board. This is great for managing a backlog and assigning bugs. This shows a great workflow that works really well for agile teams. Our team uses it for managing projects, and for managing our back log. You can organize, prioritize, add labels, and filter by the labels. You can also build test plans where you can create checklist for each card. This site is very versatile and allows for many different uses. You can also create organizations and add people to them.
Workflowy is a list website that can be used for test plans and organization. In this site you make list and list within those list. Things can be marked as completed. You can hide the completed. You can add notes to each element in order to kept track of things. So a test plan can be created using the list, within the list mark them as completed as you go. Then add notes to the completed test. Again you could use this very simple tool for a lot of different things, just use your imagination.
Over much of my research I found that a lot of people, to this day use excel for their test plans. There are articles posted all over about how to best use excel. For example this link, talks about how to utilize excel when writing your test plan, and I have attached a template I found on that site that utilizes excel.
So why is excel so widely used? Well, it is simple and versatile. Each tester has their own ways of doing things and excel allows everyone to have a different set up. When I looked at other testing software such as Test link which is widely used and open sourced test plan software. In the end it took me over an hour to set everything up without a tutorial. It is overly complicated, that might be why it is not used as widely as excel. Another one, I am excited to see how it grows, is Overlook. However, the main page currently says it best “Really Simple Software Test Plan”. It is very basic, but the reason I am excited about it is because it was recently bought by Crowd Soured Testing, and they told me they would be working on integrating it with another new site of theirs, Damn Bugs. Plus they work with the people on their site to help make the best product. I will go over Overlook and Damn bugs at a later time though when I go over free sites that can be of uses to a QA professional.
The problem lies in the software for test plans, it is either not versatile enough, or it is too simple. Excel allows the test plan to be any type. What needs to be made, and you can find within software companies who make their own, is a system that has an integrated test plan, and bug tracking system. This allows for when something is failed, there is a bug, and it is tracked immediately. There are paid systems out there that are good, but again not everyone is going to fork that much money into QA. So in the end, excel just works where others fail.
Also remember that test plans aren’t everything and Exploratory Testing is needed in conjunction with your test plans.
