What is Performance?

Posted by Ben Simo

As we design and test for performance, let's look beyond speed. Let's look beyond basic stability. Let's look at the many facets of performance. Consider in which ways your system needs to perform. How fast does it need to be? How long does it need to keep running? Does it need to grow? Does it need to be available at all times? How much can we spend? Can we make it faster?

There are endless questions we could ask. Therefore, categorizing facets of performance and creating tests for each category can be helpful. However, let's not fail to look at the interaction between these categories.

OFAT (One Factor At a Time) testing (as exampled in the above performance testing checklist excerpt) often fails to provide information related to the interaction between the categories. Let's do some MFAT (Multiple Factors At a Time) testing and analysis. Let's look at the system as a whole. Let's mix it up. Let's consider how these facets interact. Let's create test scenarios that include multiple facets.

Time Machine: Devops, 45 Years Ago

Posted by Ben Simo

"Two major inputs are required 

to provide computer service: 

equipment and manpower. 

For development, 

employees such as 

programmers and systems analysts are needed;

for operation, 

the services of managers, operators, etc., are essential. 

In practice the two activities usually coexist. 

Some sort of computer installation 

is required 

to check out development efforts. 

And almost every installation 

engages in continuing development 

as old systems are modified and new ones begun."

- William F. Sharpe, Economics of Computers, 1969

The above suggests that Devops is not at all a new concept. Forty-five years ago, computer rental cost slightly more than the Devops staff required to use it. Non-personnel operations cost typically rounded out the final third of the total cost. Today, hardware is cheap and people are expensive.

So, what did a Devops organization look like forty-five years ago?  The table below presents some data gathered in a "nation-wide census of data processing personnel" in 1967.

William F. Sharpe, Economics of Computers, 1969 

How do these wages add up in today's dollars?

Check out

The Co$t of Living.

Compared to my experience over the past couple decades, this seems to be heavy on management -- both in cost and percent of personnel. The roles of computer operator and librarian are pretty much gone. Programming and analyst roles have since blurred. Software has replaced many things that people used to do.  As the user base has moved from specialist operators to the general public, new testing and user experience design roles have evolved.

What else has changed?

What do you think devops will look like forty-five years into the future?

The Right Stuff?

Posted by Ben Simo

Update: I haz new job. I am no longer seeking work. (12 June 2012)

You cannot be anything you want to be

-- but you can be a lot more of who you already are.

- Tom Rath, StrengthsFinder 2.0

Given that I resigned a couple weeks ago and am looking for new work, I've been asking myself a lot of questions about what I want to do next. Do I want to do detailed technical work? Do I want to lead people? Do I want to consult? Do I want to be someone's employee, or do I want to do short term contract work? Do I want to stay in Arizona, go back to Colorado, or go somewhere new? I've not yet decided. I am considering a number of options.


What I do know is that I want to find work that is a good match for my strengths and mindset. When looking for new work for myself, and when I've been involved in hiring, matching strengths and mindset has been more important to me than matching specific technical skills. People who are enabled to exercise their strengths and philosophy tend to have the intrinsic motivation needed to update their skills as each situation demands.

Rightshifting:

Improving the effectiveness 

of knowledge-work businesses.

I am intrigued by the Rightshifting ideas coming from Bob Marshall, the Flowchain Sensei. The basic idea of the Rightshifting model is that the curve shown below indicates that the majority of organizations (peak of the curve) are ineffective compared to the few effective organizations on the right side of the curve. While organizations to the right of Bob's model tend to better match my personal mindset, finding work with such organizations is difficult -- there aren't many of them. My experience suggests that organizations further to the left can better benefit from what I have to offer than organizations to the right. However, if such organizations have no desire to improve their effectiveness, I am likely to get frustrated. Therefore, one more question I am asking myself is whether I prefer to be more of a hero in the midst of analytical dysfunction or more of a team member in a synergistic organization. I suspect my place is somewhere in the middle -- with an organization that is shifting right.

Side Note: One way in which I think I may disagree with Bob Marshall's model is the assertion that the chaotic organizations on the left are less productive than the analytic organizations to their right. I suspect many of the majority (the analytical organizations) are just better at measuring (or pretending to measure) their productivity than the organizations to their left. What we label chaos is often order we don't yet understand well enough to describe. I'm also not convinced that organizations need to take a trip through the dehumanizing methods of many analytical organizations in order to become more productive.

I recently watched a video in which Bob Marshall recommended the Strengths Finder book and the associated online assessment as a useful tool in discovering your strengths. I am typically skeptical of such assessments. I fear they tend to provoke the Forer effect in which people tend to demonstrate a confirmation bias and attribute accuracy to generalized information about themselves when it is presented as being specifically tailored for them. However, I decided to give Strengths Finder a try. Despite my reservations and desire to critically evaluate the timed questionnaire as I completed it, I found the resulting top "themes" to be a surprisingly accurate description of my strengths. Perhaps this only demonstrates that even a skeptic like me can succumb to the Forer effect.

According to Strengths Finder, my top five strengths are:

1. Ideation
2. Command
3. Activator
4. Individualization
5. Relator

Whether this is based on a meaningful algorithm or is total BS, I found the results useful. (All models are wrong, but some are useful.)  These five strengths seem to example what I believe can make a good context-driven tester; but then, I'm biased. :)

So, back to what do I want in my next job or independent business venture: I want to exercise my strengths in an environment where I don't have to suffer cognitive dissonance in order to serve my employer or clients.

In thinking about this, I created a mind map of the strengths from Strengths Finder and things I like, as well as things I don't like. Following up on my desire to find a mindset match over simply matching skills, I've decided to ignore typical job hunting advice and share this.

Think I might be a match for your organization? Hire me. :)

Target Tested ✔

Posted by Ben Simo

Computers empower us to create and manipulate things in ways that don't produce quite what we intended. This requires testing for what we don't expect.

There is a problem with this advertisement from Target Australia. Can you spot it?