Of Drones, ethics, and remote working

A recent topic of ethical debate relates to the use of military drones. As well as the current issues we should consider what the future might have in store for us, and I have outlined below a couple of potential ethical issues relating to the remote operation of drones.

Blurring Training and Operations
We are nearing a point where the experience of a training simulation will be more or less indistinguishable to an actual operation. Because operation of a drone is mediated via a software interface it should soon be possible to have the fidelity of the training software to match that of an effective operational human-drone interface. This could be achieved by increases in the fidelity of the training simulation to match the operational experience, or via lowering the fidelity of the operational experience to match the training experience. 'Real life' fidelity and information feeds are not required to effectively pilot a drone, and indeed abstracted interface elements may even be beneficial. At this point, due to the remote nature of operating drones, it may be possible to insert real operations into a series of training runs without the operator even knowing. As far as the operator is concerned they could be practicing a scenario, and not realise they have been swapped over to the real thing.

Outsourced Warriors
Once the scenario outlined above becomes possible, we could potentially see outsourced warfighting to gamers. I mean this in two senses. The first that suitable games available to the general public could be used to train relevant warfighting skills into to the general populace. In the second sense actual operations could be outsourced into gaming networks, such that players are usually playing a simulated game but every so often (knowingly or unknowingly) they are performing actual missions.

Redundancy
By the time we get to the point where the scenarios outlined above could be reality it may be a moot point anyway; the machines might be better off executing missions without us.

Situation Awareness, Workload, and asking the right question

Sometimes it is important to challenge questions, and explicitly or implicitly point out that the question being asked is wrong.

Once upon a time, on a project far far away....

Measuring Situation Awareness (SA) and Mental Workload (MW) are important ways of gaining insight into how an integrated system (in this sense people using technology to deliver a capability) will perform. Understanding the troughs and peaks of SA and MW is important in evaluating and designing a system in terms of automation and decision support, task and role design, manning levels, and so on. If at a critical point in a task MW is high and SA is low, then clearly this suggests that there may be problem with the capability delivery, and this is a point in the taskflow that is a strong candidate to have some form of intervention. Similarly identifying areas of low MW and high SA might be a candidate of rebalancing workload.

When measuring SA and MW, you want to have representative users, carrying out representative tasks, in a representative environment. The closer you can get to the true system operation, the closer your results will be to the real system operation.

On the aforementioned entirely mythical project one of my colleagues pointed out that it was standard practice to measure the SA and MW of typical/representative end users. The question then asked was to provide a reference to what standard this was stated in.

To ask this question is to misunderstand what the point of measuring SA and MW is. It is a wrong question. If one were evaluating a new Space Shuttle, then one would be interested in understanding the SA and MW of the likely pilots flying the new Space Shuttle, with all their relevant skills, training, experience, and so forth. The SA of Bob in the accounts department is not going to give you insight into how your target user population will actually perform when flying the Space Shuttle mk2.

This is somewhat like being really keen to make sure that people take driving tests, but not being concerned whether the person you are giving a licence to is one of the people who has taken the test.

Sometimes it is better to give people the information they need, rather than the information they want.

The Subject and Purpose of Education

We should be mindful of policies and debates around our education system. For schools in particular we should assess each change and ask whether it serves the purpose of nurturing future citizens or is producing workers.

Our society needs productive citizens, but they should be that: citizens that are productive, not educated to be part of industry.

Lifespans and Divorce

Via Gray's Cyborg Citizen p144:

Stephanie Coontz, in The Way We Never Were, notes that increased longevity created the potential for marriage to last longer and therefore more marriages to end in divorces. In the nineteenth century the average length of a marriage was 10 years, mainly due to the high mortality rate. Today divorce rates compensate for life expectancies that produce, on the average, 40-year marriages. In essence, divorce has replaced death in terminating many unions.

Changes to humans and the overall techno-social spaces (what Gray would refer to as cyborgian) have consequences that we may not anticipate. They will shake our unconsidered faith in age old foundationals, and open up new horizons, both good and bad. 

The Dangers of Choice

Choice is generally accepted as a good thing. We value our free will and autonomy, and being able to make decisions and influence the course of our lives is important to us. Choice isn't always positive however. Choices can be misleading, they can distract from worthwhile options, and they can be a shield or be a distraction from what matters. Sometimes it is better to omit choice, and provide a good approach or value as the default. This needs to be balanced against providing flexibility, utility, and personalisation.

In this list below, which is by no means exhaustive, I have outlined some of the problems with choices and some strategies relating to using choice in design. These issues and approaches should apply across many contexts where choice is an issue, from user interfaces, to business design, and even how we build the basic institutions of our society.

Choices can be intimidating

Choices can be intimidating. A user interface that has too many options can deter people from using the product or service, especially if they need or want something quick and straight forward. You should not want to scare away your users or customers. A search interface that explicitly provides every possible option is overwhelming. It will take time for users to grapple with the interface and figure out what needs to be done.

Provide guidance and a safe environment. Signal and signpost the option that will be the most suitable for the majority of cases. Keep things simple and easy to understand; more advanced options can be provided, but tuck them away.

In one sense this is a curse of a graphical interface. The command line is unlikely to overwhelm users with options, but it can be just as intimidating in its blinking mystery and lack of guidance. A good example of an interface with progressive choice would be a search engine that allows modifiers and additional commands to be entered into the search box. These options are available to users, and the fact of their existence could be signposted, but they are not visually apparent on the initial screen so do not overwhelm a user. Yet advanced users can make use of them immediately.

Choices can give too much choice

Choices, like design, need constraints. Constraints provide context and direction, as well as narrowing the field of potential options. Too much choice can be daunting, and it may be impractical or too difficult to meaningfully compare or understand them all.

You should design with focus. Provide meaningful choices that support the user. Provide 'one step wizards' that will handle a task for a user, and also provide small steps and functions that are easy to understand and can be chained together to perform larger actions and tasks. The available choices can be designed and presented such that they impart meaning to the application or service, and help the user understand what is available to them.

Choices can be unhelpful

You are in charge of a nuclear power station. There is an emergency, and the reactor needs to be shut down. As a user you are only interested in safely shutting down the reactor. You want clear, identifiable and easily understood routes to shutting down the reactor. You do not want to be presented with superfluous options, nor do you want the option of omitting things such as raising an alarm.

Likewise rather than being presented with the choice of a range of differently performing schools, it is far better for the users of the service that the standards of schools are raised. In this example choice is masking the failure to provide an adequate level of service.

Within the concept of freedom there is a distinction between formal freedoms and real freedoms. Although formally a freedom may be accorded to you (you are free to do X), in practice you may be subject to constraints that limit or remove your ability to freely make a choice. Give people real choices, don't give the impression of choice where in practice there is no choice.

Choices should be meaningful, relevant, and supportive. Don't provide the choice to do things badly, or to offer a poorer level of service, or if you do provide the best choice by default; require the users to actively choose a poorer (e.g. less safe or less desirable state).

Choices can be presented poorly

The more differences there are between choices the harder they are to evaluate. In the market place there is a proliferation of choice (or at least the proliferation of the perception of choice). On the one hand this provides a greater diversity of products and services, but it also adds unnecessary choices and makes selection more difficult. As choice proliferates the effort and knowledge required to make an effective choice expands, ultimately harming the end user.

Support users in making choices. Provide meaningful comparisons. Indicate the costs and consequences of choices. Keep the field of choice small but useful.

Choice can be bad for your health

Choices are not always good. Who wants to choose between good healthcare or poor healthcare? A fulfilling life or a miserable life? Choice is generally considered to be a positive thing, but you should be cautious and carefully consider whether the choices being offered are beneficial to the other party.

A useful example comes via Brian Barry's Why Social Justice Matters. Imagine a system for health care, much like the American one, where your employer supplies health insurance as part of your reward package. Ideally for those who are the beneficiaries, this should provide for all of their health care needs. If this isn't possible then it should provide for the most likely and the most pressing health care needs. The example that Barry gives is that some companies start to provide a choice of health insurance packages. On the face of it, this seems positive because choice is considered to be good and empowering. In practice it becomes difficult for individuals to pick between the choices, and requires an investment of time and effort to fully understand all of the options and their ramifications. Additional issues include that: not everyone will be as well equipped to make the choice and so the less educated and those with less time will make poorer decisions; human beings are notoriously bad at assessing risk and so this is something better worked out objectively by experts; the choices may not be as good as the original comprehensive option and worse still employees may be asked to sacrifice more to select certain choices, and; rather than have a small number of people examining the available options the result is that now everyone has to carry out the effort. Pushing choice out to individuals, rather than having experts make choices, multiplies the overall work and effort required to make the choice. Once such a system is in place it has further ramifications: the stratified choices will create an administrative burden that has to be met by someone; it breaks people into groups which allows for treating them differently, and; it shifts responsibility, or the perception of responsibility, onto the employee. Where before there may have been a single fit-for-purpose system, there is now a series of options that are difficult to choose from, are not as comprehensive, cost more to make use of, take more individual and overall effort to asses, and if something goes wrong the perception is that the employee chose the wrong package rather than that the employer failed to provide adequate cover.

When considering and implementing choice we should always consider who benefits, and if we would be better off without a choice. Choosing between a poor transcoding algorithm and a good one is not a worthwhile choice, and nor is choosing between a poor reward package and a good one.

Provide a good level of basic service. Choice should be made available to personalise or adapt a system, not to diminish the experience of users. Clearly signpost risky or irreversible changes. Provide safety nets and recovery routes  (e.g. undo and/or a recycle bin).

On Engineering Organisations and Teams

Via Brian Barry's Why Social Justice Matters:

"Any dictatorship takes a psychological toll on its subjects. If you are treated as an untrustworthy person — a potential slacker, drug addict, or thief — you may begin to feel less trustworthy yourself. If you are constantly reminded of your lowly position in the social hierarchy, whether by individual managers or be a plethora of impersonal rules, you being to accept that unfortunate status." - Barbara Ehrenreich (2001) Nickel and Dimed: On (not) Getting By in America

and:

"The re-engineering of corporations may sound progressive, especially to shareholders, but the apparent price workers pay is an undercurrent of anxiety and diminished loyalty and commitment, their morale eroded by a chaotic and dysfunctional work environment in which individuals are discounted or devalued altogether... A recent study showed that workers who kept their jobs during a major downsizing were twice as likely to die from cardiovascular disease, perhaps triggered by work stress" - Cited in Stefan Fern, Let's Give Change a Rest, Guardian 21 May 2004.

If the latter is true, then providing suitable fit-for-purpose working environments that treats people as autonomous agents worthy of respect may be just as effective as some health programmes and interventions.

Flexible working

When designing systems you need to be aware of the unintended consequences. As an example flexible working arrangements (e.g. flexitime) allow people to access social capital more readily. If parents have a choice of schools and consider the more distant school to be better, they will more readily be able to send their child to it if they have flexible working arrangements. Similarly, flexible working arrangements allow people to more easily access medical care and attend doctor's appointments both in terms of actually being able to attend, and not having a large impact on their finances. Of course this is a greater boon for those who are less able to afford to take unpaid time off of work.

The Importance of Getting Specialist Advice Early

In the development of a system (or product, process, service..) getting relevant specialist advice early on in a project will help to get a better system, and avoid potentially expensive—in terms of money and time—rework.

The image below is a rough visualisation of the “hill climbing problem”. Imagine the wiggly line as being a range of hills on the horizon. The horizon represents the problem space for your system. Each hill is a potential solution; a different way of meeting the requirement or need. Different hills have different properties and advantages, but generally you want to be as high as you can be. Or at least as high as your requirement and contract demands.

Systems can be complex beasts and require input from a variety of disciplines. Without support from appropriate specialists the project team may be picking the wrong hill to climb. Your new software application, command and control system, service, gadget or gizmo will likely benefit from, or need, input from a variety of experts. Does the system need to be safe? Does it need to conform to particular legislation? Will the human component of the system have an impact on its performance? Does it need a long battery life? Will it need to be maintained? What is the potential logistical impact? Without appropriate input the team may not know that other hills even exist, let alone that they need to be aiming for a different one (i.e. their product may not be designed with certain safety legislation in mind).

Design decisions represent selecting a hill in the hill climbing metaphor. Once a hill has been selected effort is expended to scale the slope of the hill; meetings, decisions, analysis, design work, prototyping, and so on. Money, time, reputation, and careers can be invested in the chosen design.

Unfortunately, the team might be scaling the wrong hill, or at least not the best hill. If a team isn’t aware of all of the factors—all of the requirements and considerations—that will have an influence on the design they are making, then they are making uninformed decisions. If relevant experts are involved early on, in the design or ideally the concept phase, then the team will have a better understanding of what hills they should be aiming for; they will be making more informed decisions. This reduces risk and will contribute to a better design, more or less for free compared to the costs of having to rework a design.

The alternative, and is a situation often faced in the case of Human Factors, looks like the diagram below. When a team discovers they have a problem (i.e. problems with legislation, discovering that the workload is too high for the number of operators, etc), or there is a need for improvement it may be too late to implement transformational change. In this situation the team has been making great progress towards a local maximum. They’re doing great things, but they’re not going to go get to the top of a really big hill because the hill they’re climbing (the solution they’re building) isn’t the best hill, or might not get them as high as they need to be.

Faced with this situation a Human Factors practitioner (or Safety Specialist, etc) has two options in providing support to the project; helping the team get to the top of their local maximum, or shifting them to a better hill. In many cases it isn’t realistic to get a change of hill. Time, cost, effort, careers, and reputations may have been heavily invested in the current approach. Aside from the effort to get the system to its current state there will be a wealth of documentation and design decisions behind it, hard fought negotiations and compromises, and a great deal of work to establish a common understanding of the current design amongst the team members and stakeholders. On top of these issues there are reputational and career issues to consider. For example, if there is a lot invested in a project it may prove unlikely that senior figures (or the organisation as a whole) will admit that mistakes have been made and there needs to be a change of direction. In an ideal world this will not be the case, but realistically while transformational change may be desirable and achievable in terms of resources it may not be a pragmatic option to pursue. The appetite and capacity for change may simply not be present.

Getting specialist advice early on in a project, before design decisions have been made, can help to direct a project and result in a better solution whilst avoiding the need for expensive rework or adaptation. It also makes better use of specialist staff; they’ll be influencing the design based on fundamental considerations that improve the system, rather than coming up with patches and fixes.