“Keep your values and your facts separate.” Daniel Kahneman- Noise: A Flaw in Human Judgment
Introduction
What is noise in this context? It’s the range of difference in decision making generated by a number of people using subjective responses to a particular question. This includes what are claimed to be ‘objective’ processes of assessment.
Psychologist Jonathan Haidt observes that we make decisions using unconscious processes and our conscious processes are often no more than PR agents employing rationale to defend our decisions.
What we call reasons are often no more than a rational defence of an irrational decision. For example, researchers in the USA have found judges make harsher decisions just before meal breaks and less harsh decisions after they have eaten.
Other research involving multiple judges assessing a hypothetical case shows a huge range in sentences proposed. Reasons for decision would be well reasoned argument – but such argument, as objective as it appears, essentially papers over real unconscious reasons for the decision.
Across public sectors discontent with grievance handling is consistent. The normal response is to assume there is something wrong with the process, rather than how process is employed by people making decisions about grievances. Improving process won’t solve the problem if it’s about how decisions are arrived at in the decision-makers minds.
Keep Your Values and Your Facts Separate
This is far easier said than done. In fact, the Behavioral Economist, Iris Bohner argues that we need far better guidance on making fact-based decisions. Our values may not be the values of our employer. Values constructed at organizational, and even sector level, are not intimately linked to complex psychological processes that make us unique decision makers. An effective decision-making guide can deliver fairer decisions.
Daniel Kahneman and his colleagues demonstrate that wide disparity in judging a situation across any group indicates that while the ‘facts’ of a matter are plain to all, how those ‘facts’ are processed and evaluated varies sometimes wildly.
This leads to a staggering proposition. You can’t evaluate a decision on the basis that it was reasonably argued. Beautifully argued cases can still be an extreme (and irrational) expression of a range of possibly fair judgements. If you agree with a decision, or not, that may just be your bias kicking in.
We like to believe we are smart and fair minded – and that may be true – up to a point. But we are, according to research, more likely to over-estimate our own virtues and strengths than be modest or realistic.
There is a big problem that Kahneman and others have identified. We usually don’t make decisions that are optimally smart or fair. Consider the plaintiff guilty of an offence and facing a judge for sentencing. Let’s assume the guilty verdict is okay. His sentence will depend on who is the judge – and the judge’s mood, whether the decision is make before or after a meal, whether the plaintiff triggers a strong reaction in the judge, as well as how the judge has decided prior recent cases. The fact that justice might be blind does not impede influence of unconscious responses – biases essentially.
But Grievances are Not Court Cases
This is true, but consider the following:
In response to the statement “I have confidence in the ways my organisation resolves grievances” on the NSW Public Service Commission’s People Matter Employee Survey (PMES) the following results are very interesting:
- 2016 page 58 – 43% Agree, 10% Strongly Disagree, 15% Disagree.
- 2017 page 63 – 36% Agree, 11% Strongly Disagree, 17% Disagree.
- 2018 Page 48 – 40% Agree, 11% Strongly Disagree, 15% Disagree.
- 2019 Page 28 – 41% Agree, 10% Strongly Disagree, 15% Disagree.
- 2020 page 30 – 45% Agree, 22% Strongly Disagree, 33% Disagree.
- 2021 page 18 – 46% Agree, 22%, Strongly Disagree, 32% Disagree.
In the 6 years (2016-2021) agreement rates have shifted 3%. But strongly disagree and disagree have gone from static 10%-11% and 15%-17% to more than double for 2020 and 2021. There is little doubt this is an impact of COVID. But even so having 25%-28% of staff not agreeing grievances are well-handled should not be acceptable, let alone around 55% of staff. Here I am not suggesting that COVID is an excuse for higher levels of discontent. I am acknowledging that it is an abnormal factor that has adversely impacted the level of discontent. If anything, it reflects poorly on organisations in managing issues precipitated by the pandemic.
I will allow that not all discontent about how a grievance has been handled is because of how it was handled. Some folk will expect that the only just and proper decision is in their favour – and that’s not always a fair stance to take.
In the context of staff with disability the rate of failure to obtain a decent outcome without intervention and advocacy is high. We know that in recent times in the NSW public sector, around 24% of staff with disability say they have been bullied. On page 21 of the 2018 Sector PMES report we can read:
Of those who reported bullying, 20% made a formal complaint to their agency (compared to 22% last year). Only 21% of those who formally complained felt their case had been resolved satisfactorily, and 50% indicated that it had not.
The low level of reporting is a cause for concern. But the fact that half of those who did report were not satisfied with the outcome is even more worrying. It takes some strong need to make a formal report – a genuine sense of grievance. It is, in my experience, unlikely that people with disability who make formal reports are exaggerating or inventing their cause for grievance.
Staff with disability tend to be more reluctant than most to submit a formal grievance. They feel more vulnerable to reprisals – and finding an alternate role can be far harder for them. As DEN Chair I was asked to intervene in complaints and grievances. Only one was not justified in the terms of the complainant’s perception, but the intervention turned what could have been catastrophic for the person’s psychological health into a positive outcome for all parties.
It is clear that there is a problem in how grievance are responded to. I have written recently on the problem of bias. Noise is another factor that makes a mockery of rational and objective assessments. If there is not one assured fair standard what can be done to assure fairness?
The Need for Really Fair Measures and Processes
Processes that may be administratively rational may still not account for the people factor – which introduces bias and noise. In fact, administratively rational processes may entrench injustice because it is mistakenly assumed that by following a rational process the outcome must be rational and fair.
In fact, an administratively rational process essentially provides a defensible outcome – so if there is a complaint, a decision can be plausibly defended – because defending decisions is essential for an administrative process. Whether the decision is objectively right or just is less important – because determining that is, as demonstrated here, an entirely problematic affair.
Kahneman argues that machine learning algorithms are better at decisions than humans. He uses the example of sentencing guidelines being a better guide for making fair decisions than allowing judges to arrive at sentencing decision on their own. Iris Bohner argues that designed systems give better results than the kind of guesswork we have now. Bohner has a particular interest in recruitment processes that can remove bias.
The argument for an evidence-based system for assessing complaints and grievances raised by staff with disability that takes decision-making away from managers is compelling. In a document titled Disability Management Across the UK Civil Service, Microlink argued, in relation to workplace adjustments, for significant changes to the way things are done. To be clear, Microlink is a private sector company advocating for the adoption of its own services. But if we allow that its claim to know what works is valid, we can explore the key ideas. Here are some key ones:
- Enable self-referral to expert case advisers.
- Appoint a named business manager with the authority and accountability to ensure end-to-end service quality.
- Provide a service that uses experts and not managers to diagnose what is needed and to order the adjustment, drive the process and minimise unnecessary assessments.
- Make sure that the first point of contact is an expert who can provide triage.
- An effective case management system is required to track cases and collate relevant data.
- Publish measurements to increase visibility, awareness and accountability of the process.
This is a rational process that suits workplace adjustments. Would it work for grievances and complaints? The immediate objection would be cost, and that it would complicate a process that an agency is perfectly well capable of handling – despite compelling evidence to the contrary. What it does is take control out of the hands of those who benefit from keeping that control.
Obviously, the noise and bias risks are just being transferred to a different source. Would an independent expert be less noisy and biased than managers with no expertise and a possible interest in controlling the process and the outcome? I think the answer is a clear “Yes.”
External experts are engaged to assess ergonomic needs of injured staff. Injured staff are sent to medical experts to assess fitness to remain at work. Legal branch is engaged when there is a legal question to be answered. ‘Independent’ investigators are called on, at an agency’s discretion, to assess a complaint allegation. The extent to which such investigators are ‘independent’ is debateable. An agency defines the task and pays for the service. The investigator is in service of who pays, not the truth – at least those who want to get further business understand this to be true.
The presumption of the need for independent expertise to intervene confirms that such a need is recognised – albeit selectively.
Conclusion
I am sharing my struggle to make sense of why good people do not act to address injustice swiftly and effectively. When I set up the DEN’s Guidance and Action Team (GAT) in September 2018, I encountered an astonishing number of colleagues with disability who had endured years of discrimination and bullying. In February 2019 half a dozen of them told their stories to our department’s executive board of management. That was a transformative experience that fundamentally changed the department’s response to Disability Inclusion.
But revelations of conduct that constituted serious misconduct did not result in holding those responsible to account. Why was this? At the time I was so grateful for the positive responsive from our executive leaders, and the subsequent positive impact, I didn’t ask. It has taken the distance of over 24 months for that to become a question in focus. This is important. I have no doubt that the executives who were present would agree that the lack of action is a cause for concern. I have retained contact with the 6 DEN members, who share that sentiment.
How a senior executive team squares away the lack of progress is another question. Don’t get me wrong. We made a huge amount of progress in a lot of areas. The impact of that meeting reverberated powerfully. But it stopped short of holding identifiable perpetrators of bullying and abuse to account. Why was that?
This has nothing to do with conscious or intentional choices. None of the people I esteem as Disability Champions are fully conscious of how their championship is failing the cause. Tremendous progress has been made over the past couple of years. That is everywhere evident – but so are the gaps.
The important thing is that what is causing those gaps is an attribute of the ‘system’ which is being revealed through a lot of very good research. That research is accessible through excellent publications. I have put links about the ones I am aware of below. I use Amazon, not out of any affection – it simply provides the best range of available publication formats.
I don’t know what the solution is in any practical and immediate sense. I do know that the solution must be doing other than what is done now. My purpose here is to stimulate reflection and conversation – toward an agreed understanding of the challenges, risks and possible solutions.
A major consideration is the distinction between rules and standards when making decisions about the kind of behaviour that is acceptable/required. Standards are noisy, whereas rules are not. There are questions about optimal wording, and interpretation of either – and the consequences for non-compliance. Consideration of these things is where effective decision-making is critical.
One of those possible solutions might be implementing “Decision Hygiene”. Here are a few links where you can download brief discussions:
They may get the conversation moving.
Books
Noise: A Flaw in Human Judgement Daniel Kahneman et al (also see YouTube interviews)
What Works: Gender Equality by Design, Iris Bohnet
The Fearless Organisation: Creating Psychological Safety in the Workplace for Learning, Innovation, and Growth, Amy Edmondson
The Righteous Mind: Why Good People are Divided by Politics and Religion, Jonathan Haidt