Problems: So What's On All Those Sticky Notes?

Posted by Ben Simo

In my previous post about the Agile Alliance Functional Testing Tools Workshop , I wrote the following:

After reviewing existing tools used by agile teams: we identified software testing issues that have been solved (yellow), those that have been partially solved (orange), and those that have not been solved (pink). As I recollect, most of the solved issues were technical problems and most of the unsolved problems were people problems. Many of the partially solved problems were those for which I believe we have technical solutions but have not yet been integrated and presented in ways that best support people.

In case you are wondering what problems we wrote down on these notes, Frank Maurer kindly transcribed them for the workshop participants and I have posted them below.

Looking at this list reminds me of the traffic safety problem lists I made in the defensive driving classes I used to teach. As unsafe driving practices were brought up by students, I would add them to a list on the whiteboard. I then asked the students to identify whether each item on my list was primarily due to driver skill or driver attitude. The students usually blamed most of the problems on driver attitude.

Skill and attitude play important roles in software development and testing. Team members need both. Brian Marick addresses this in his guiding values of discipline, skill, ease, and joy. Instead of looking at the functional testing problems as skill or attitude problems, I looked at them as man or machine problems. I asked myself if each problem appeared to be mainly a human problem or a technical problem.

Software development and testing involves a mix of people and technical problems. Interfacing people with people and people with technology is often harder than interfacing technology with technology. Most of the identified problems have both technical and human aspects to the problem and possible solutions. Some are due to the nature of people or the nature of software and will likely never be completely solved.

I find that identifying whether a problem is primarily a people or technology problem helps me identify possible solutions. I quickly scanned this list and identified whether I thought things were primarily human or technical issues. My notes are included to the right of each item. I don't necessarily interpret each item as its author (a human communication problem), and I do not necessarily agree that each item is a problem or belongs in the specified group.

As you review this list, ask yourself if the problem and possible solutions are grounded in people or technology.

Unsolved	Define Test	Human
Unsolved	Organizing large sets of Tests/Expect. Actions/Examples for a large, complex system so you can wrap your head around the whole thing.	Human
Unsolved	Having tests survive handoffs. Project team -> op support -> proj team.	Human and technical
Unsolved	Getting people to care	Human
Unsolved	Transferability of ubiquitous language to other projects	Human
Unsolved	Write SW that is understood	Human
Unsolved	Reducing Uncertainty	Human
Unsolved	Limitations of natural language	Human
Unsolved	How would we act if we really believed code was an asset?	Mostly Human
Unsolved	Allowing Customers to articulate their expectations in a format/tool/way that is comfortable for them	Mostly Human
Unsolved	Multi-model specification text + table + graphic in one test.	Mostly Technical
Unsolved	Domain experts	Human
Unsolved	Fully Automated regr. That does not reduce dev.velocity.	Mostly Technical
Unsolved	Generate a domain model from tests.	Human and technical
Unsolved	Testing usability as part of acceptance testing in incremental development.	Human
Unsolved	Common language to express GUI based tests.	Mostly Human
Unsolved	Conveying "experts" perspective to majority of development team.	Human
Unsolved	Automated Software Development	Technical (Machines aren't creative)
Unsolved	Functional tests that can be easily re-used later in lifecycle.	Mostly Technical
Unsolved	Test business requirements independent of current interaction/Api design.	Mostly Human
Unsolved	Composing tests into useful larger tests	Technical and Human
Unsolved	Test first performance	Human
Unsolved	Getting BA to write the tests	Mostly Human
Unsolved	Having customer to be able to write test and enjoy it.	Mostly Human
Unsolved	Different test notations for different user groups.	Human problem, technical solution?
Unsolved	Acceptance/Functional Tests good for communication and automation.	Human problem, technical solution?
Unsolved	Model the time domain " and 3 months later an email.	Mostly Technical? (Don't want execution to take 3 months.)
Unsolved	Change touch-point dynamically.	?
Unsolved	Terminology (Test or not a Test?)	Human
Partially Solved	Understand what has not been tested.	Human and Technical
Partially Solved	Trace tests into project management tools	Human and Technical
Partially Solved	Getting buy-in for need to automate.(docs,tests,specs)	Human and Technical
Partially Solved	Accurately & completely communicating requirements.	Human problem, partial technical solutions
Partially Solved	Satisfying every role's need/desire to be at center, in control.	Human
Partially Solved	Having functional tests specify requirement specifications.	Human and Technical
Partially Solved	Sustain a productive conversation with all stake holders.	Human problem, partial technical solution
Partially Solved	How do we get across what the project would feel like if things were going well.	Human
Partially Solved	Write robust (U.I) tests that are not brittle.	Mostly Technical
Partially Solved	Fragile tests.	Mostly Technical
Partially Solved	Valuing individuals and interactions over processes and tools.	Human
Partially Solved	Describing customer intent.	Human
Partially Solved	Executable (as tests) Models (as specifications)	Human and technical
Partially Solved	Test partitioning	Human and technical
Partially Solved	Tests as support artifacts.	Human and technical
Partially Solved	Test Generation Automation.	Human and technical
Partially Solved	Running tests parallely ( Fast feedback)	Mostly technical
Partially Solved	Reconciling preferred style of abstraction.	Human and technical
Partially Solved	Dealing with size.	Human
Partially Solved	Finding the right words in which to write a test.	Human
Partially Solved	Express requirement in the domain language, graphical, word based , table based.	Mostly Technical
Partially Solved	Functional Test driven development..not just for agilists. How to sell to waterfallists?	Mostly Human
Partially Solved	How to Integrate tools?	Mostly Technical
Partially Solved	Common test case format.	Human and Technical
Partially Solved	Cooperation and collaboration between tool developers (tool stack, tool platform).	Human and Technical
Partially Solved	Change from one notation to another. (graphic -> tabular)	Mostly technical
Partially Solved	Test/Example -> model (generated model based tests)	Mostly technical
Partially Solved	IDE for testers and BAs	Human and technical
Partially Solved	Get all roles actively involved	Mostly Human
Partially Solved	View specs/examples/tests, differently for different roles.	Mostly Technical
Partially Solved	Different editors for different roles?	Mostly Technical
Partially Solved	Super-power IDE	Technical solution to human problems?
Partially Solved	Test Refactoring	Human and technical
Partially Solved	Refactoring tests	Human and technical
Partially Solved	Prioritize and execute tests to get faster feedback.	Mostly Human
Partially Solved	Describe a test at an appropriate level of abstraction	Mostly Technical
Partially Solved	Choosing what to automate (when you can't automate everything)	Mostly Technical
Partially Solved	Tools to support exploratory testing	Human and Technical
Partially Solved	Reusable test artifacts (poor modularity cohesion)	Mostly Technical
Partially Solved	Build community with BAs	Mostly Human
Partially Solved	Allow for refactoring from /to code <-> tests <-> req'ts	Mostly Technical
Partially Solved	Setup a wiki to discuss smaller problem solving.	People
Partially Solved	Capture war stories + testimonials + experiences.	Human, partial technical solution
Partially Solved	Test Maintenance	Human and Technical
Partially Solved	Book: Patterns of Self testing software	?
Partially Solved	Shared vocabulary around parts of a functional testing solution ("Fixture",etc)	Human
Partially Solved	Ensure adequate test coverage.	Mostly Technical
Partially Solved	Communication of what has been tested	Human and technical
Partially Solved	Communication using the ubiquitous language.	Human
Solved	Express automated/able tests in tables	Mostly Technical
Solved	Correctly implementing programmer intent	Mostly Technical
Solved	Deliver SW to test that doesn’t crash immediately	Mostly Technical
Solved	Provide traceability between Story or Requirement and accpetance/functional test	Mostly Technical
Solved	Gui Testing - functional testing is more than GUI testing	Technical
Solved	Data-Driven Testing	Technical
Solved	Driving Apps	Technical
Solved	Integrating test executors/drivers with build process	Technical
Solved	Edit Fit tests from eclipse	Technical
Solved	Report Results	Technical to report, human to be understood
Solved	Express expectations in code	Human and technical
Solved	Unit testing	Mostly Technical

I think we can solve the technical issues and use technical solutions to help people manage some of the people problems. Applying discipline and skill to solve the technical problems may help add to testers' ease and joy.

Where do you think the solutions lie? Have a solution? Please share it.