NicholasCollins at 02:25, 3 May 2019

2019-05-03T02:25:38Z

NicholasCollins at 02:23, 3 May 2019

2019-05-03T02:23:51Z

BobJonkman: /* Meeting Notes */ Add link to YouTube video

2019-04-28T03:22:16Z

Meeting Notes: Add link to YouTube video

BobJonkman: Add link to Software Testing Additional Notes

2019-04-28T03:20:07Z

Add link to Software Testing Additional Notes

BobJonkman: New Page

2019-04-28T03:10:46Z

New Page

New page

{{:Software Testing}}

__TOC__

==== Resources ====

* [https://www.techsoupcanada.ca/en/directory/334 TechSoup Canada catalogue: Developer Software]
* [https://obsproject.com/ OBS Project] - software that is good for capturing videos of tests including drop menus

One of James Bach's [https://www.youtube.com/watch?v=ILkT_HV9DVU talks on YouTube]
{{#evu:https://www.youtube.com/watch?v=ILkT_HV9DVU | dimensions=640 | alignment=center}}

==== Meeting Notes ====
* Introductions

* Nicholas Collins
** Software tester for a few years, knowledge of how ''his'' company works
** But two years isn't a long time compared to some software testers
** Nick has prepared notes, will be presenting slightly differently from other KWNPSA sessions
** SysAdmin in insurance industry; laid off (as are many of us); back to school to upgrade IT skills
** Uses Visual Studio, C#, other languages
** People he's met were developers, or business-specific skills; when software testers are needed these people are thrust into the role
** This might change as more universities offer software testing as a major
** There are very few courses or certificates in software testing, more prevalent in the US
*** but Fanshawe college in London has a certificate program
** Some institutions have a couple of courses in tech writing, project management, quality management; maybe a night course in software quality testing
** Without academic rigour, different people use different terminology, nomenclature
*** "Should I know what all these different terms mean?" But it's fairly common with other software testers Nick has spoken to
** At Microsoft, developers use their development skills to write tests. Needs more skills than just coding
*** Microsoft has internal courses to train testers how to test software
*** Get promoted to full developer once you've proven you can write tests
** people use Terminology like "Post-mortem" (although nobody dies), mix up "Milestone" and "Benchmark", &c.
** Software testing is the start for a developer's career, then to DevOps
*** Does this mean the most junior, inexperienced programmers are responsible for testing software?
** Nick: large companies use junior testers to run tests, senior testers to supervise
** During an upgrade Nick (a programmer at the time) did testing for the Database Analyst
*** But a junior intern was assigned to that role as well, just to gain experience.
*** Worked out details at a high level, then applied tests to get results

* Project Managers take different approaches

* '''You can always think of more tests'''
** It's a fine balance between staying on schedule and being thorough
** Walkthroughs and working in a team can be helpful

* Some testing instructors do not like teaching from texts
** eg. "Software Testing" by Yogesh Singh
** But Nicholas gets good ideas from texts, doesn't agree with those testing instructors
** THe problem is that the authors suffer from "Perfect Worldism"
*** A world where there is unlimited time and money, and the perfect tests can be developed
** Nicholas has experience with sticky problems, gets ideas from texts to adapt to his problem

* Party talk: Software tester does not lead to stimulating conversation
** YouTube presenter on software testing is not dull! (**********Need link!***********)

* Even the simplest test "is A < 70 ?" can have seven or eight tests
** Test results, but also overflows, boundaries, different data types
** Input validation can require many tests

* Working with other people, eg. technical writers
** For them to understand the software they'll play with the software, and may create unanticipated conditions
** Everyone can be a software tester to some degree: Project manager, developer, writer. Even sales?
** Sometimes testers find problems with usability as they're running tests; not part of the test suite

* How effective are some of these ad-hoc testers?
** Is there a bias? Do they have some incentive to pass tests even when there are problems?
*** Sometimes a QA will hold back tests that would have been better to give to the developer in the first place
** Accessibility testing is a new skill for QA, may become a testing requirement
** Business Analyst (BA), developer and tester make a good team
*** Sometimes the process of testing will identify the need for more testing

* Reporting bugs
** Requires consideration, tact

* Test plans may need to be developed quickly
** But near the end of a project when time is tight there may not be time to develop tests
** So quality of code may suffer near the end of the project
*** Breaking things during testing that no-one has time to fix

* Automated testing?
** Nick has experience with automated regression testing
** Automated regression testing reduces the introduction of new bugs

* Open Broadcaster Software
** Used to catch all activity during user testing
** Also use Virtual Box recorder uses host to capture all the output on the VM screen

* "Monkey Testing"
** Also "fuzz testing" or "fuzzing"
** Fill all fields, try to overflow, pound on the keyboard, click as fast as possible
*** But this this does not lead to reproducible errors (fine timing errors)
*** Although some testers claim they can reproduce

* Pride in finding bugs?
** Nick finds that the "high five" time should occur only after the entire team has identified, reported, documented, and fixed the error, and re-tested

* Load testing
** Hitting a system with a large number of transactions, &c.
** But a bogged down system may not be writing to logs, making analysis difficult
** A benefit in load testing is adding assertions, find issues with threads
*** Assertions and Singletons...
** Be sure to validate the output even when just testing for capacity

* Nick has written a test for XML testing
** But the code Nick wrote was not well tested at all! Oh, the irony!

* Q: Do you use debugger software like GDB to examine the flow of code?
* A: Not common, but becoming more prevalent.
** Certainly having a debugger to throw at the code is nice to have
** But much testing is done with the software under test as a black box, just examine the input and the expected output

* Nick speaks of the complexity of software testing.
** One thing works fine by itself, and other thing does too, but do they work together?
** Different software on different platform needs to interoperate, but sometimes differences in date formats causes problems
*** although each platform by itself passed all tests
** Dealing with currencies, eg. USD and CAD, and GBP
** Dealing with leap years and 29 February
** General rule: Anything date sensitive needs to test for leap years
*** and time zones! Anything dealing with calendars needs to worry about time zones

* What happens internationally when different countries need to interoperate?
** Companies have service contracts that define how the service is implemented
*** If the system is changed, the contract defines who is responsible for continued interoperation
*** If I make a change and it breaks your system, it's your fault for not defining the contract accurately
*** called "spring contracts"

* Nick gives an example from James ---- YouTube video ("nominal input voltage is 100VAC to 250VAC")
** "Test the nominal range" is an incomplete answer
** Also need to test outside the range
** The user manual may give advice not to go outside the nominal range, but users don't necessarily read the manual
** So, does the system fail gracefully outside the nominal range?
** This is the function of the software tester, to design the test to ensure that software or equipment is failsafe
*** eg. for medical equipment
*** How much money is available to fry the device under test? Some prototypes may be '''really''' expensive
*** Many examples of people damaging electronics with incorrect application of voltage!
** It's good for testers to think outside the parameters of the system

* Testing to ensure system has a consistent look and feel
** eg. fonts on some menus were different
*** Is that a software testers responsibility? Sometimes as an additional task
*** There are tools (overlays, templates) to find these issues
** Window resizing can make the application fail, but there need to be limits for those tests
** Testing for "greyed out" functions can be time consuming
*** When a function is available when it shouldn't be can result in errors
** These are general things for a tester to keep in mind

* Systems that have features which have little to do with each other
** Easy to test they're not contending for resources, &c.
** But still important to run these features simultaneous to shake loose bugs, eg. memory allocation, concurrent DB access
** Perhaps a simple monitor with limited functions: But what if something goes wrong, does the device report an error?

* Client-side data validation: All testing needs to be duplicated at the server to ensure malusers don't bypass client-side validation
** But that increases load on the server

* Logging
** Logs may indicate problems with the way the code executes, eg. repeated log entries indicate an invalid loop
** Circular reasoning: How can the logs from software under test be considered
*** Logs are only one step, begin the process of analysis
*** NewRelic will test user experience (surveillance software)

* Nick has found bugs because the test suites are well designed
** But at least half the time the bugs discovered were found in spite of the test, which was not designed to find that kind of bug

* Q&A
** Is the developer + tester model usable?
*** May be a bit scary for shops not set up for that collaborative arrangement
** Nick says to just forge ahead.
*** Having experience is good, but can also develop that experience in-house
** Worries about the coming requirements for accessibility for software
*** May take changes in coding practices (use POSH: Plain Ol' Semantic HTML instead of Javascripted forms)
*** Jurisdictional differences may be difficult to deal with

Back to: [[Software Testing]]

[[Category:KWNPSA Meeting Notes]]

← Older revision		Revision as of 03:22, 28 April 2019
Line 64:		Line 64:

	* Party talk: Software tester does not lead to stimulating conversation		* Party talk: Software tester does not lead to stimulating conversation
−	YouTube presenter on software testing is not dull! ~~(******Need link!*********)~~	+	** YouTube presenter on software testing is not dull! https://www.youtube.com/watch?v=ILkT_HV9DVU

← Older revision		Revision as of 03:20, 28 April 2019
Line 13:		Line 13:
	One of James Bach's [https://www.youtube.com/watch?v=ILkT_HV9DVU talks on YouTube]		One of James Bach's [https://www.youtube.com/watch?v=ILkT_HV9DVU talks on YouTube]
	{{#evu:https://www.youtube.com/watch?v=ILkT_HV9DVU \| dimensions=640 \| alignment=center}}		{{#evu:https://www.youtube.com/watch?v=ILkT_HV9DVU \| dimensions=640 \| alignment=center}}
		+
		+
		+	* [[Software Testing Additional Notes]] - Nicholas Collins' presentation notes

@@ Line 68: / Line 68: @@
 * Even the simplest test "is A < 70 ?" can have seven or eight tests
 ** Test results, but also overflows, boundaries, different data types
 ** Input validation can require many tests

@@ Line 153: / Line 153: @@
 * Nick gives an example from James ---- YouTube video ("nominal input voltage is 100VAC to 250VAC")
 ** "Test the nominal range" is an incomplete answer
 ** Also need to test outside the range

Software Testing/Meeting Notes 2019-04-08 - Revision history

NicholasCollins at 02:25, 3 May 2019

NicholasCollins at 02:23, 3 May 2019

BobJonkman: /* Meeting Notes */ Add link to YouTube video

BobJonkman: Add link to Software Testing Additional Notes

BobJonkman: New Page