Sometimes it is important to challenge questions, and explicitly or implicitly point out that the question being asked is wrong.
Once upon a time, on a project far far away....
Measuring Situation Awareness (SA) and Mental Workload (MW) are important ways of gaining insight into how an integrated system (in this sense people using technology to deliver a capability) will perform. Understanding the troughs and peaks of SA and MW is important in evaluating and designing a system in terms of automation and decision support, task and role design, manning levels, and so on. If at a critical point in a task MW is high and SA is low, then clearly this suggests that there may be problem with the capability delivery, and this is a point in the taskflow that is a strong candidate to have some form of intervention. Similarly identifying areas of low MW and high SA might be a candidate of rebalancing workload.
When measuring SA and MW, you want to have representative users, carrying out representative tasks, in a representative environment. The closer you can get to the true system operation, the closer your results will be to the real system operation.
On the aforementioned entirely mythical project one of my colleagues pointed out that it was standard practice to measure the SA and MW of typical/representative end users. The question then asked was to provide a reference to what standard this was stated in.
To ask this question is to misunderstand what the point of measuring SA and MW is. It is a wrong question. If one were evaluating a new Space Shuttle, then one would be interested in understanding the SA and MW of the likely pilots flying the new Space Shuttle, with all their relevant skills, training, experience, and so forth. The SA of Bob in the accounts department is not going to give you insight into how your target user population will actually perform when flying the Space Shuttle mk2.
This is somewhat like being really keen to make sure that people take driving tests, but not being concerned whether the person you are giving a licence to is one of the people who has taken the test.
Sometimes it is better to give people the information they need, rather than the information they want.
Once upon a time, on a project far far away....
Measuring Situation Awareness (SA) and Mental Workload (MW) are important ways of gaining insight into how an integrated system (in this sense people using technology to deliver a capability) will perform. Understanding the troughs and peaks of SA and MW is important in evaluating and designing a system in terms of automation and decision support, task and role design, manning levels, and so on. If at a critical point in a task MW is high and SA is low, then clearly this suggests that there may be problem with the capability delivery, and this is a point in the taskflow that is a strong candidate to have some form of intervention. Similarly identifying areas of low MW and high SA might be a candidate of rebalancing workload.
When measuring SA and MW, you want to have representative users, carrying out representative tasks, in a representative environment. The closer you can get to the true system operation, the closer your results will be to the real system operation.
On the aforementioned entirely mythical project one of my colleagues pointed out that it was standard practice to measure the SA and MW of typical/representative end users. The question then asked was to provide a reference to what standard this was stated in.
To ask this question is to misunderstand what the point of measuring SA and MW is. It is a wrong question. If one were evaluating a new Space Shuttle, then one would be interested in understanding the SA and MW of the likely pilots flying the new Space Shuttle, with all their relevant skills, training, experience, and so forth. The SA of Bob in the accounts department is not going to give you insight into how your target user population will actually perform when flying the Space Shuttle mk2.
This is somewhat like being really keen to make sure that people take driving tests, but not being concerned whether the person you are giving a licence to is one of the people who has taken the test.
Sometimes it is better to give people the information they need, rather than the information they want.
No comments:
Post a Comment