Levels of Automation (LOA)

Levels of automation (LOA, also called levels of autonomy or degrees of automation) are a system of scales that describe how to divide actions or decisions between machines and human operators. Different organizations use different variants of LOA scales, but they usually range from fully manual to fully automatic through various task assignments. LOA scales are useful as a design guide for product managers to evaluate the appropriate level of automation for proposed features, as a shared vocabulary for product management and engineering teams to visualize trade-offs between control and observability, and as a way to describe the expected level of human involvement in automated processes. Such systems are also often used as maturity models or improvement roadmaps, with mixed results and controversial justifications.

Systematic LOA scales emerged in late 1970s and early 1980s, with the introduction of computer control systems in submarines and airplanes in the US military. They now shape the implementation of various technologies such as self-driving cars, robots and AI agents. The systematization of scales, particularly the implementation caveats and problems, and today and are particularly relevant for designing autonomous or semi-autonomous agentic software factories, to avoid repeating the mistakes or re-learning the lessons that other industries discovered decades ago.

This article explains the popular scale formats, the origins of LOA scales, applicability and common problems of applying such scales in product development.

Parasuraman, Sheridan and Wickens’s 10-level scale

The most popular LOA scale today comes from A Model for Types and Levels of Human Interaction with Automation, by Raja Parasuraman, Thomas Sheridan and Christopher Wickens. It has 10 levels:

Parasuraman, Sheridan and Wickens's scale of levels of automation of decision and action selection (2000), listed from high to low as in the original
LevelThe computer…
10 (HIGH)…decides everything, acts autonomously, ignoring the human.
9…informs the human only if it, the computer, decides to
8…informs the human only if asked, or
7…executes automatically, then necessarily informs the human, and
6…allows the human a restricted time to veto before automatic execution, or
5…executes that suggestion if the human approves, or
4…suggests one alternative
3…narrows the selection down to a few, or
2…offers a complete set of decision/action alternatives, or
1 (LOW)…computer offers no assistance: human must take all decisions and actions.

In practice, there are 3 important groups of levels. From levels 1 to 4, the role of a machine is to offer advice to assist with decision making. On levels 5 and 6, a computer takes actions allowed (or not prevented) by the operator. On levels 7 to 10, a computer decides and acts, potentially informing the human operator.

Parasuraman, Sheridan and Wickens’s numeric ranking suggests that the scale is single-dimensional, but it is composed from several dimensions of task allocations (proposing appropriate actions, selecting one to take, approving the action, executing it and announcing what action was taken). So the scale is actually multi-dimensional, compressed by selecting 10 of the many possible combinations. The authors did not validate their choices of dimensions or the levels in any specific way, but they propose them more as a design tool. Because the selection is a matter of judgement, many other variants of scales emerged, with other authors selecting different dimensions or compressing them differently, leading to a considerable debate around applicability and limitations.

Choosing the level of automation for a function

The LOA scales primarily are useful as a design tool to agree on the appropriate level of automation for a feature or a function. Parasuraman, Sheridan and Wickens propose using scales to select the upper bounds for automation depending on the risk and cost of errors. Here are some of the key ideas from current practice and literature for using the scales in practice:

Describe automation levels for individual functions, not for a whole system. Sheridan and Verplank framed their original scale for a single decision step, not as an overall rating of a system. Christopher Miller argues that a single level for a whole system is “something like an average or modal” level of its subtasks, which may be misleading as it does not necessarily match any single function. He provides an example of a junk e-mail filter that has five subtasks automated at level 9 or above and two closer to level 2, with an average rating of 8. The overall number is highly misleading. Describing each function separately shows operators what parts they actually control, and which functions are handled by the automation.

Evaluate sub-activities to decide on the appropriate level for a function: Parasuraman, Sheridan and Wickens suggest evaluating the options from the perspective of four classes of activities: information acquisition, information analysis, decision and action selection and action implementation (note that this is very similar to John Boyd’s OODA Loop). Each of these subtasks may require a different maximum level of automation. For example, a car collision warning system can be highly automated for information acquisition and analysis, leaving the decision and the corrective action entirely to the driver.

Levels of automation across four stages
Parasuraman, Sheridan and Wickens suggest analysing automation from the perspective of acquisition, analysis, decision and implementation, each with its own level

Set upper bounds by risk: Warning that “automation can have both beneficial and negative effects on human performance”, Parasuraman, Sheridan and Wickens suggest considering the levels for each function class through two filters. The “primary evaluative criteria” is related to potential impact on humans. Parasuraman and colleagues list potential impacts on mental workload, situation awareness, complacency and skill degradation. The “secondary evaluative criteria” applies to the reduced set of options produced by the primary filter, and deals with automation reliability and cost of wrong actions.

Consider failure cases when planning: Like any complex system, automation can fail in unexpected ways. When introducing automated control, designers should also consider what happens when the automation fails, and how difficult it is for a human operator to act accordingly. In a 1975 survey paper for an engineering congress, K. S. Bibby, F. Margulies, J. E. Rijnsdorp and R. M. J. Withers warned that a high degree of automation “tends to reduce supervisory work to 99% boredom and 1% sudden terror”, citing a 1974 workshop contribution by H. Van Droffelaar on stress among operators of a highly automated process (we were not able to trace the source fully, and Bibby and colleagues do not provide any data source references, so this is most likely just a memorable aphorism). One of the key design considerations should be a trade-off between routine task performance and unexpected failure. Lisanne Bainbridge argues that an operator may need to take over control from automation when something is already going wrong, so the person stepping in needs to be “more rather than less skilled”, and not overloaded with other tasks. Designing for failure should include testing that handover explicitly, with realistic warning times, and not assuming that people can easily catch mistakes made by a machine. In critical systems such as flight control, Warren VanderBurgh advised that operators should take control first and work out the cause afterwards.

Design for multiple levels, and switching the level of automation when required: A product that offers only one level of automation for a function may leave operators with the difficult choice of completely taking over or reprogramming the automation under stress in unexpected situations. VanderBurgh argues that automation “lacks the ability to create flexible responses” to unanticipated changes, so when the situation changes quickly, switching to a lower level of automation reduces the workload instead of raising it, and allows operators to handle the exceptions more easily. Earl Wiener found in his 1986–87 survey of B-757 pilots that many switched the automation off as their workload rose (without providing any specific data). An interesting aspect to consider is to split controlling and monitoring, as proposed by Wiener and Renwick Curry in 1980. This allows an alternative route to switching automation levels: transfer the implementation to the operator, and keep the monitoring automated, potentially providing the operator with valuable information collection and analysis that they may or may not choose to trust. Wiener argued in 1985 that the long-standing advice that “humans should serve as monitors” of automatic devices must be reconsidered, and that the human should be “brought back into a more active role in the control loop”, aided by decision support (note that this is an essay built on press reports of incidents, so it is a design direction, not a tested one).

Allow fully manual execution for exceptions and training: Bainbridge argued that “it can be important to maintain manual skills”, and suggested hands-on control for a short period in each shift, or simulator practice where that is impractical. The US Federal Aviation Administration 2017 Safety Alert for Operators recommends that “air carrier’s line operations policy should permit and encourage manual flight operations” when conditions permit, while warning against quotas for manual flying that are not based on evidence.

Using the LOA Scales as a maturity model

Levels of Automation Scales and similar systems are often used as maturity models, to evaluate teams or to compare products, with controversial results and disputable justifications.

Sheridan warned that the published level descriptions had been “taken more seriously than were expected”, because the many dimensions of automation “might be better left as a multi-dimensional space”. In 2021, he wrote that the scale was suggested “as something to think about” and not as a rule or guideline, and that no behaviour can be reduced to a single dimension.

Jeffrey Bradshaw and colleagues list the belief that “the conceptualization of ‘levels of autonomy’ is a useful scientific grounding for the development of autonomous system roadmaps” among the myths of autonomous systems, arguing that autonomy cannot be measured on “a single ordered scale of increasing levels”.

J. E. Rijnsdorp, R. N. Pikaar and T. M. J. Lenior argued that a very high degree of automation reduces the operator’s job mainly to monitoring, which is stressful, unsatisfying and poor preparation for unexpected events, so “it is not always advisable to aim for the highest degree of automation which is technically possible”. VanderBurgh instructed pilots to break the habit of trying to “operate at the highest levels at all times”.

Generally, there are two issues with treating LOA scales as a maturity model:

  1. A higher level is not necessarily better. Onnasch and colleagues pooled 18 laboratory and simulation experiments that compared different degrees of automation, and found “no clear optimum of automation support”: routine performance improved as automation rose, but performance after a failure got worse (see the Routine/Failure trade-off section). Essentially, a higher level signals a different allocation of work, not a better product.
  2. Different situations may call for different levels of automation even in the same environment. Discussing flight control automation, Wiener reports that many B-757 pilots switched the automation off as their workload rose, and VanderBurgh explains that pilots should adjust the automation level based on the task at hand, including moving down the scales.

A 2012 US Defense Science Board task force recommended that the Department of Defense abandon the use of “levels of autonomy”, after finding that debating definitions had wasted “both time and money”. SAE J3016 industry standard document makes the point that level assignments are nominal rather than ordinal: its levels “do not specify or imply hierarchy in terms of” merit, technical sophistication or order of deployment, and level 4 is not “better” than level 3, it is just different. The numbers on levels should not be understood as ranks.

Instead of treating the scales as a maturity model, it might be better to consider the scales as a performance path map, providing capabilities and training for human operators to work on different levels of automation depending on the context.

Origins of LOA scales

Thomas Sheridan is the main authority behind the evolution of model LOA scales, from his work on submarine supervisory control in the 1970s to position papers published in early 2020s. Early work on automation scales was funded by the US Office of Naval Research, with later iterations including influences from military and civilian flight control, and factory automation.

Bright’s 17 levels of mechanization

The earliest scale of automation we have found is from the article “How to Evaluate Automation” by James R. Bright’s, included in the July–August 1955 issue of the Harvard Business Review. (We were not able to fully trace it, so the scale below is from various second-hand papers.) Paul G. Adams 1956 thesis at George Washington University in 1956 lists Bright’s “17 levels of mechanization”:

Bright's levels, as quoted by Adams
  1. Hand
  2. Hand tool
  3. Powered hand tool
  4. Power tool, hand control
  5. Power tool, fixed cycle (single function)
  6. Power tool, program control (sequential functioning)
  7. Power tool, remote control
  8. Actuation by introduction of work piece or material
  9. Measurement of a characteristic
  10. Signalling the measurement value
  11. Recording performance
  12. Changing speed, position or direction according to measurement
  13. Segregating or rejecting according to measurement
  14. Identifying and selecting appropriate action
  15. Correcting performance after operating
  16. Correcting performance while operating
  17. Anticipating required performance and adjusting accordingly

Adams reports that automatic control “begins somewhere between 9 and 12” on this ladder, and that Bright primarily used the scale to judge whether replacing factory labour with automation was worth the cost.

Sheridan and Verplank’s table

Sheridan’s 1974 textbook with William Ferrell describes the role of a person as a supervisor of “semiautomatic, limited-intelligence machines”, but does not set out a specific scale for automation levels. Sheridan’s 1978 report Human and Computer Control of Undersea Teleoperators for the US Office of Naval Research, co-authored with William Verplank, includes a table titled “Levels of Automation in Man-Computer Decision-Making” that can be applied “for a single elemental decisive step”.

Sheridan and Verplank's levels of automation in man-computer decision-making (1978, Table 8.2), descriptions only
LevelDescription of interaction
1human does the whole job up to the point of turning it over to the computer to implement
2computer helps by determining the options
3computer helps determine options and suggests one, which human need not follow
4computer selects action and human may or may not do it
5computer selects action and implements it if human approves
6computer selects action, informs human in plenty of time to stop it
7computer does whole job and necessarily tells human what it did
8computer does whole job and tells human what it did only if human explicitly asks
9computer does whole job and tells human what it did and it, the computer, decides he should be told
10computer does whole job if it decides it should be done, and if so tells human, if it decides he should be told

The original table differs from the 2000 version in several aspects.

The first is that Sheridan and Verplank explicitly structure the levels through the lens of six types of tasks, selecting whether the human operator or the machine are responsible for those types of tasks: requests, gets, selects, approves, starts and tells. The 2000 version implies multiple dimensions used to select levels, but does not list them explicitly.

The second major difference is that the 1978 version starts and ends on different levels from the 2000 version. The initial point of the 1978 scale is for humans to make all the decisions but machines implementing them, where then 2000 version starts from a human performing the actions as well. The 1978 version ends with the computer informing the operator if the computer decides that is important, where the 2000 version ends with a computer performing actions without informing the operator. As the 2000 scale is broader than the 1978 scale, the intermediate levels are then merged.

The wording evolved gradually, through Sheridan’s work with other co-authors. In a 1983 paper on adapting automation to people and cultures, Sheridan, Tibor Vámos and Shuhei Aida offer an ordered scale of “degrees of assistance”, which starts from “The computer offers no assistance”, adds a level that “narrows the selection to a restricted set”, and merges the 1978 levels 3 and 4. A 1997 National Research Council report on air traffic control includes the scale with several levels renamed, particularly mentioning the “restricted time to veto” and “acts autonomously, ignoring the human”. The report references Sheridan’s chapter “Supervisory control” in Gavriel Salvendy’s Handbook of Human Factors (we were not able to fully trace the original edition to verify the exact wording). The follow-up 1998 National Research Council report suggested separately applying the scales for information acquisition and action implementation. Parasuraman, Sheridan and Wickens all served on that NRC panel, publishing the current version a few years later.

Variants of the levels of automation scale

Because the scales and the underlying dimensions are largely driven by the author’s experience and preferences, many different authors proposed their own variants. Vagia, Transeth and Fjerdingen catalogue 13 variants of LOA scales between 1978 and 2007. Most variants are slight modifications of Sheridan and Verplank’s original work, so they are not worth specifically analyzing. A few are worth studying separately for further research:

Increased use of AI coding tools and assistants in recent years led to a renewed interest in classifying levels of automation, and several such scales emerged with companies trying to define the role of AI and changing roles of humans in software development. Meredith Ringel Morris and colleagues at Google DeepMind proposed a scale with six levels of autonomy for human–AI interaction, from “No AI”, through “AI As a Tool”, “AI As a Consultant”, “AI as a Collaborator”, “AI as an Expert” to “AI as an Agent”, modelled on the SAE levels and positioned against “prior taxonomies of computer automation”. K. J. Kevin Feng, David McDonald and Amy Zhang define five levels of autonomy for AI agents by the user’s role: operator, collaborator, consultant, approver and observer, without any prior influences.

The routine/failure trade-off and the lumberjack effect

The main trade-off implied by LOA scales is between performance for routine tasks and difficulty of recovering when things fail. Generally, more automation improves performance for routine tasks, but can make exceptions more difficult to handle. Exceptions can be unforeseen scenarios or non-routine tasks, which are not handled well by the current automation configuration, or they can be caused by the failure of the automation mechanisms requiring recovery or reconfiguration. Wiener suggested that automation “tunes out small errors” and creates opportunities for large ones. Higher degrees of automation remove flexibility and can cause human operators to lose situational awareness or skills to perform tasks manually.

Bibby and colleagues suggest that many control engineers dream of “complete automation, in which man’s role is only that of a consumer”, yet even highly automated systems need people to supervise, adjust, maintain and improve them, so “automated systems still are man/machine systems”. Bainbridge reflects on that paper suggesting that it is ironic that “the more advanced a control system is, so the more crucial may be the contribution of the human operator”, but an operator that is only used to monitoring the system loses the skills to take over the execution if a situation demands it.

Wiener and Curry analyse the levels of automation from the perspective of who controls operations, and who monitors them, and suggest that when both tasks are assigned to the automation, operators risk boredom, complacency and erosion of competence, while when both are manual, they face high workload and fatigue.

Automation of control and monitoring
Automating control and monitoring separately: when both are manual, operators face high workload and fatigue; when both are automated, they risk boredom, complacency and erosion of competence (after Wiener and Curry, 1980)

The newest AI-agent scales run into the same problem. Feng and colleagues raise the risk of users “meaningless rubber stamping” the agent’s requests at the higher levels of their scale. Making people approve the work of automated agents puts them in the role Bainbridge singled out in 1983 as the ironic one, where the operator is being asked to monitor a system installed because it does the job better.

Neither Wiener nor Bainbridge provide any supporting data for their claims, and generally most of the literature on the levels of automation or risks is based on practitioner experience and expert assumptions, not on operational research. There are only a few studies of underlying assumptions, with mixed results.

Linda Onnasch and colleagues evaluated the trade-off between routine task performance and failure recovery with laboratory and simulation experiments, finding that routine performance improved as the degree of automation rose, and that performance after an automation failure got worse. They named it the lumberjack effect: “the higher they are, the farther they fall”. Onnasch did not report how many people took part in total, the reported workload benefits are not statistically significant, and the research was in a laboratory setting. Christopher Wickens, Huiyang Li, Amy Santamaria, Angelia Sebok and Nadine B. Sarter published a conference paper reviewing roughly the same laboratory studies, finding that 12 of 14 routine performance slopes were positive, but that there was no link between routine gains and failure losses, so “the ‘lumberjack analogy’ is not well supported”. Greg Jamieson and Gyrd Skraaning point out that only 14 of the 18 studies in the 2014 meta-analysis involved university students, not professionals, so the results may not be relevant for industry work.

Jamieson and Skraaning ran tests with 24 licensed nuclear power plant operators in 8 crews, in a full-scope simulator with four levels of procedure automation. Their conclusion is that situation awareness increases with automation (on an almost linear trendline), which was the opposite of the prediction, and there was no measurable effect on performance or workload after a failure. This study also has a small sample, and did not measure routine performance, so it’s best to consider it as a potential reason to doubt the trade-off in complex work, not as proof against it.

Wickens, while defending the levels of automation models, agrees that its trends are “quite muted or absent” in naturalistic studies.

Applicability and limitations

One of the most comprehensive reviews of the applicability and limitations of LOA scales is David Kaber’s article for a 2018 special issue of the Journal of Cognitive Engineering and Decision Making. Kaber reviews the main objections to levels of automation and provides answers to the objections, defending the scales overall. The objections he collects from other researchers are:

Kaber adds two problems of his own. The scale models are “presumptive in their concepts” of how operators behave, built from designers’ assumptions rather than observation. The underlying dimensions or evaluation criteria, such as risk, complacency, situation awareness and trust, are mostly loosely defined. Kaber suggests running experiments and then using the results of those experiments to refine the scales, instead of abandoning them, because abandoning a framework with some design utility “makes little sense” without a sound alternative.

Miller agrees that better models are needed, but argues that the problem is deeper. Any system of levels compresses a continuous, multidimensional space into a few categories, and that compression discards detail, such as how two subtasks at the same level differ in what users can see or change. In his view better models must come before any scheme of levels, so that the details that are lost by selecting only a few levels do not matter.

The most recent systematic review of levels of automation systems, by Luke Richardson, Justin Fidock and Indra Gunawan, finds the confusion between automation and autonomy “commonly unaddressed and persistent”. The authors still see a “compelling argument for retaining” levels of automation taxonomies, as tools for disciplined thinking rather than for prediction.

Matthew Johnson, Jeffrey Bradshaw and Paul Feltovich argue that a levels scale can describe a design after the fact rather than helping to shape one, suggesting that the middle levels would be better designed by analysing how operators and automation mechanisms depend on each other. Neelam Naikar argues more broadly that work systems “are necessarily self-organizing”, so any fixed allocation between operators and machines constrains them. Emilie Roth and colleagues suggest that LOA scales group “tasks at too gross a level of classification”, but they propose a more comprehensive allocation process rather than abandoning the scales.

Get notified about new articles & major updates
Share