As in any scientific discipline, studies of nonviolent communication (NVC) have their share of shortcomings, biases and errors. This short article aims to explore these imperfections through three main axes: experimental design, sampling and interpretation of results. By examining these aspects, we will highlight the methodological challenges that future studies will have to face, and suggest ways of reinforcing scientific rigor in this field.

Experimental design

Experimental design is a rigorous scientific approach to the study of cause-and-effect relationships between variables. It involves organizing an experiment in such a way as to eliminate external influences that could distort the results. The aim is to ensure that observations reflect reality and that conclusions are sound.

To achieve this, the researcher must carefully plan several steps: selecting the factors to be studied, dividing the participants or samples into different groups (e.g., a test group and a control group), and controlling the experimental conditions. Next, the data collected are analyzed using statistical methods to determine whether the differences observed are significant or simply due to chance.

Confirmation bias and self-reported tests

Confirmation bias is a cognitive mechanism whereby a person tends to favor information that confirms their preconceptions or assumptions, while giving less weight to information and assumptions that contradict them. For example, take the CNVC page summarizing scientific studies on Nonviolent Communication (NVC). The proposed summaries cite only the passages where NVC has positive effects, without mentioning the limitations or nuances brought to bear by the researchers. This is a typical case of confirmation bias: we select only what supports our ideas, as if we were sorting through information to keep only that which suits us. As a reminder, the scientific approach is the exact opposite: to look for reasons to doubt rather than reasons to believe.

Self-reported, or explicit, tests are tests that directly measure participants’ responses. So participants understand exactly what is being measured and can easily guess what the studies are trying to prove. For example, in Vazhappilly‘s 2017 study, subjects are tested with explicit questionnaires about their experiences as a couple while they undergo training in couple communication. This obvious link runs the risk of distorting participants’ answers, prompted by a bias of complacency (conscious or otherwise) to validate the experiment. This could artificially improve the results of post-training tests. Yet these self-reported tests still largely dominate scientific publications on NVC.

However, implicit alternatives exist to circumvent these biases:

• Implicit questionnaires, these tools mask the real purpose of the evaluation, thus reducing complacency bias. For example:
  • Perspective-taking test: Measures the ability to adopt the point of view of others, without the link with training being explicit.
  • Causal attribution test: Evaluates how subjects explain social situations, without direct reference to the skills being worked on.
• Tasks with reaction time measurement, these methods rely on automatic responses, difficult to control voluntarily. For example:
  • Implicit Association Test (IAT): Detects spontaneous mental associations (e.g. between Gender and Science) via the speed of responses.
• Physiological measures (more complex but objective): capture involuntary bodily responses, independent of declarative bias. For example:
  • Heart rate/breathing: Indicators of emotional activation.
  • Electroencephalogram (EEG): Measures brain activity related to a specific task, such as emotion recognition.

The learning effect and the absence of a control group

The learning effect arises when, by reproposing a test to a subject within a short interval, the subject has learned from the first test and will obtain better results the second time around. For example, in a pre-post experiment, a test is administered before and after NVC training, with the aim of quantifying the impact of this training. If there is no control group, as is unfortunately the case in the studies by Marlow (2012) or Museux (2016), it is impossible to distinguish the impact of the training from the learning effect of the test. The control group thus makes it possible to dissociate the real impact of training from methodological artifacts (learning effect) and contextual factors.

No intervention in control group

Several studies evaluating NVC have indeed included a control group, but this does not participate in any activity comparable in duration or format to the NVC training (e.g., Wacker, 2018; Kang, 2019). For example, the control group does not attend group workshops or structured exercises over an equivalent period. Consequently, the differences observed between the two groups may not reflect the specific effectiveness of NVC, but simply the effect of methodological biases: time spent outside the professional setting, social interactions, or simple participation in a new activity. Without rigorous control of these confounding variables, the impact attributed to NVC remains difficult to isolate.

To demonstrate the real effectiveness of NVC training, the control group must participate in an activity similar in duration, format and intensity, but without NVC content. This comparability is crucial to rule out non-specific effects (group dynamics, change of context, etc.).

Long-term effects

The duration of the effects of an intervention is an essential criterion for measuring its effectiveness. When studying the results of training at the very end (Jung 2023), these very short-term effects do not tell us anything about the long-term dynamics and impact on daily life. It may be difficult to find subjects more than 6 months after an intervention, but they are valuable data.

Sampling

Sampling methods not representative of the general population

A sampling method that is not representative of the population necessarily introduces bias into the results. These biases limit, or even render impossible, the generalization of conclusions to the entire population. For example, Marlow’s 2012 study recruited its participants (men on parole) exclusively from a residential center. This specific population may differ significantly from other people on parole. Sampling bias can also occur when participants are recruited on a voluntary basis (Yang 2021).

To avoid this bias as far as possible, sampling methods must be totally random within the population of interest.

Sample size

A rule of thumb in statistics suggests at least 30 subjects per group. However, certain studies, such as those involving monkeys, rare diseases or the recording of specific neurons, cannot always reach this number due to their complexity. In cognitive science, reaching this threshold is often more feasible, even if some studies on NVC are published with very small numbers, such as Museux 2016, where there were only 9 participants. For a study to be valid, its statistical power (ability to detect a real effect) must be appropriate. This means:

• Define in advance the minimum effect size considered significant (for example, a 20% improvement in NVC skills).
• Adjust the sample size according to this target effect size

A small sample can only detect very large effects, while a large sample can identify more subtle effects. Statistical power analyses, although classic, are still too rarely used to determine the right sample size for the desired effect.

Homogeneity of groups prior to intervention

To make a statistical comparison between a control group and an intervention group, it is crucial that these groups are homogeneous. This means they must be similar in terms of demographic and contextual characteristics. For example, in Wacker’s 2018 study, where participation in NVC training was voluntary, the two groups were not balanced in terms of educational level. On average, the NVC training group had three years more education than the control group. Furthermore, the Kim 2022 study compares students trained in NVC with a control group of students from a different geographical and institutional context. However, the supposed socio-cultural homogeneity between the two groups is not verified by the study’s authors. It is therefore impossible to determine whether the differences observed between the two groups are due to the CNV program or to differences in context.

Lost to view

The loss of participants during the course of a study can introduce a significant bias into the interpretation of results. When participants drop out, the initial groups may become unrepresentative of the study population, which can distort conclusions. For example, if participants who drop out share specific characteristics (such as different levels of motivation or health), the final results may no longer accurately reflect the effect of the intervention studied. For example, in Marlow’s 2012 study, 26% of participants were excluded during the course of the research due to relapse (substance abuse) or arrest. These profiles – probably more fragile – could have significantly altered the conclusions if they had remained in the final sample.

Interpreting results

Statistical significance and effect size

Statistical significance is a concept that helps determine whether the results of a study are due to chance or reflect a true trend. For example, if we’re testing the effect of a NVC module, we want to know whether the positive results are real or merely coincidental. Statistical significance is often measured by a “p-value”. If this value is below a threshold (often 0.05), we consider the results to be statistically significant. This arbitrary threshold is increasingly criticized, and it is possible to overcome it with other statistical approaches, such as the Bayesian approach. This is still rarely the case in cognitive science studies.

Effect size is used to assess the practical – and not just statistical – magnitude of a result. For example, NVC training could reduce the duration of anxiety attacks by 10%, a statistically significant but clinically marginal effect. Conversely, a 50% reduction would represent both a statistical and clinically relevant improvement. Effect size thus contextualizes results: it distinguishes theoretically detectable changes from truly significant progress.

In short, statistical significance tells us whether an effect is likely to be real, while effect size tells us how large the effect is. In most scientific articles, results are reduced to a positive effect, yet it’s essential to report results by specifying and discussing effect size and statistical significance. This new requirement is still in its infancy in science in general, not to mention NVC studies in particular…

Multiply statistical tests

When several independent statistical tests are carried out, as in Kim 2022‘s article where the authors evaluated six different questionnaires (primary anger, secondary anger, interpersonal relationships, empathy, self-esteem and communication effectiveness), the probability of finding a difference, due only to chance, increases considerably. In fact, each additional test increases the probability of false-positive results. To alleviate this problem, a commonly used method is the Bonferroni correction, which adjusts the significance threshold. For example, if the initial p-value threshold is 0.05, it would be divided by the number of tests (in this case 6), giving a new threshold of 0.008. According to the results presented (in Table 3 of the article), after this correction, only communication efficiency would remain significantly different between groups.
Kang’s 2019 study presents a significant risk of false positives. By simultaneously testing the effects of cognitive training on resignation intentions in several subgroups, the authors multiply statistical comparisons. This approach artificially increases the probability that certain observed differences are due to chance rather than to a real effect of the intervention. To counter this bias, they apply statistical corrections (e.g. Bonferroni correction), which enable them to obtain more reliable conclusions.

Conclusion

Although studies on nonviolent communication (NVC) are essential for understanding human interactions, most are not yet free of bias and methodological errors. By analyzing experimental design, sampling and statistical tests, we have highlighted several biases that can compromise the validity of results. To overcome these obstacles, more rigorous approaches and appropriate statistical corrections are crucial. By strengthening the research methodology, we can hope to obtain increasingly reliable conclusions, thus contributing to the dissemination of NVC with greater robustness and credibility. This research would also help to identify the most relevant components of NVC.