Tips for Making Multi-User QA More Efficient

As a project manager with a focus on custom development projects, I do a lot of quality assurance testing. While our developers use various automated testing approaches to ensure code is well-designed and working correctly (along with pull requests before merging code, to maintain code quality), I manually test all user-facing functionality before turning over a given feature to a client or to users.

Those custom dev projects come with a recurring special treat: multiple user roles to test! Sometimes lots of them! Sometimes with bonus complex workflows involving three or more users doing specific things in sequence! Whee!

Testing with multiple users and user roles (e.g. admin, editor, logged-out user) can quickly become a nightmare. Here are three strategies for making this kind of testing more manageable and more efficient.

Gmail Aliases + Simple Naming Convention

There are two main barriers to efficiency when testing with multiple users (assuming email address and name are required to create an account).

The first is logistical: How do you efficiently create a unique email address for potentially dozens of test accounts? I used to create new Yahoo or Hotmail accounts for each test user; filling out forms and repeatedly stepping through confirmation workflows slowed my testing considerably.

The second is organizational: How do you easily come up with and keep track of the local part of the email address and a name for each test user, without wasting time and causing confusion? Like: I just created my 17th test user but need to go back and test with the third user — wait, what email address did I use for User No. 3?

Now I use Gmail aliases and a simple naming convention to bust through both barriers. A quick explanation about Gmail aliases from the good folks at the Googs (what, you don't call them "the Googs?"):

You can receive messages sent to your.username+any.alias@gmail.com. For example, messages sent to jane.doe+notes@gmail.com are delivered to jane.doe@gmail.com

I use a simple incrementing pattern to create multiple users with multiple gmail aliases: josh.korr+1@viget.com, josh.korr+2@viget.com, etc. If I'm testing with multiple roles and multiple users within each role, I'll do something like josh.korr+admin1, josh.korr+admin2, josh.korr+editor1, josh.korr+editor2, etc. (Or josh.korr+ad1 and josh.korr+ed1 to keep things even shorter.)

That takes care of the email account time-suck and half of the naming problem. To solve the rest of the naming problem, I use a simple naming pattern like John1 Tester, John2 Tester, Admin1 Tester, Admin2 Tester.

Combine these names with the email address pattern and it's super-easy to create and keep track of multiple test users.

Testing Permissions

If an app has multiple user roles, the app by definition has a "permissions layer." From an external QA perspective, this is logic that determines:

1. Which user roles can view a given page (we think of these as "view permissions")
2. Which content or interactions should be hidden/shown for certain user roles on a given view ("inline permissions")

Keeping track of the permissions logic becomes difficult if there are more than, say, two user roles. To help track the permissions and test efficiently, I create a spreadsheet for view permissions logic and one for inline permissions logic. I list all pages in the app and all user roles, and detail each role's permissions for each page. My craziest permissions spreadsheet: nine user roles to test across about 50 pages. 

I still test discrete permissions functionality throughout a project when a given feature is completed. But spreadsheets like this make it much easier to retest the entire permissions layer once it's fully implemented.

Testing Complex Workflows

Permissions aren't the only tricky thing to test in apps with multiple user roles. I've been extra-lucky to face multiple complex, twisty workflows that require testing with multiple users in specific sequences and with multiple branching logic paths. As with permissions, there's a point at which it's impossible to keep that straight in my head and still feel confident I'm testing rigorously. In these cases, I whip up another spreadsheet to write a QA script beforehand.

In one section, I outline all the user roles I'll need for the particular test. Then I note the actual test accounts that I'll use for each role, creating new ones if needed.

In a separate section, I outline the granular steps I'll need to take for the entire test. (The linked spreadsheet template is artificially truncated; in reality there could be eight steps to "As josh.korr+role1, complete and submit assessment.") I like to color code the user role cells in the left section, and color code the corresponding cells in the script section to denote when I need to switch to a different test user.

This is definitely overkill for routine QA. But I find that for these complex flows, it's helpful to plan out the test beforehand. This ensures that I cover every possible piece of the flow, and makes it easy to pinpoint a specific failing step and repeat the test later if needed (both key for writing good bug reports).

Now let's hear your tips for multi-user QA!