DESCRIPTIVE STATISTICS

Sample Overview

Data Scope
Metric Count
Number of Articles 72
Number of Studies 172
Number of Effect Sizes 440
Number of Unique Study Subjects 76142
Discipline Count Percentage
Information Systems 150 34.1
Marketing 116 26.4
Management 76 17.3
Psychology 43 9.8
Medicine 16 3.6
General Interest 10 2.3
Transportation 6 1.4
Applied Natural Sciences 5 1.1
Law 4 0.9
Public Sector 4 0.9
NA 10 2.3

AI Label

Label Count Percentage
AI Algorithms 215 48.9
AI Systems 102 23.2
AI Assistants 98 22.3
AI Robots 13 3.0
NA 12 2.7

Consumer Response

Consumer Response
Response Count Percentage
Affective 161 36.6
Behavioral 131 29.8
Cognitive 121 27.5
NA 27 6.1

AI Characteristics

Autonomy
Level Count Percentage
Extensive 351 79.8
Limited 86 19.5
NA 3 0.7
Performance
Level Count Percentage
Equivalent 102 23.2
Superior 35 8.0
Inferior 28 6.4
NA 275 62.5
Anthropomorphism
Level Count Percentage
Absent 391 88.9
Present 49 11.1
Uniqueness Recognition
Level Count Percentage
Absent 321 73.0
Present 116 26.4
NA 3 0.7

Application Domain

Domain Count Percentage
Operations & Management 167 38.0
Entertainment & Lifestyle 81 18.4
Investing & Finance 60 13.6
Healthcare 55 12.5
Legal & Public Safety 28 6.4
Other 20 4.5
Social Welfare 15 3.4
Transportation 7 1.6
NA 7 1.6

Temporal AI Advancement

Range of Publication Years
2002 — 2023
AI Development Phases
Development Phase Count Percentage
2020-2025 300 68.2
2015-2019 97 22.0
2010-2014 29 6.6
before 2010 14 3.2

Task Characteristics

Task Objectivity
Level Count Percentage
Objective 350 79.5
Subjective 69 15.7
NA 21 4.8
Task Consequentiality
Level Count Percentage
Low 278 63.2
High 162 36.8

Study Characteristics

Human Benchmark
Level Count Percentage
Expert 355 80.7
Non-Expert 83 18.9
NA 2 0.5
AI Explanation
Level Count Percentage
Absent 427 97
Present 13 3
Field Setting
Level Count Percentage
No 419 95.2
Yes 21 4.8
Stimulus Presentation
Level Count Percentage
Text Only 397 90.2
Perceptually Rich 43 9.8
Incentive Compatibility
Level Count Percentage
No 388 88.2
Yes 52 11.8
Behavioral DV
Level Count Percentage
No 373 84.8
Yes 67 15.2
Within-Subject Design
Level Count Percentage
No 393 89.3
Yes 47 10.7
Online Sample
Level Count Percentage
Yes 256 58.2
No 184 41.8

Sample Characteristics

Age and Gender
Availability Count Percentage
Age Data Available 373 84.8
Age Data Missing 67 15.2
Gender Data Available 414 94.1
Gender Data Missing 26 5.9
Variable Percentage
Mean Age 33.9
Mean % Female 51.7
Geographic Distribution of Samples
Region Count Percentage
America 167 38.0
Europe 53 12.0
Asia 34 7.7
NA 186 42.3

PRIMARY META-ANALYSIS

Overall Effect Size Estimate

## 
## Multivariate Meta-Analysis Model (k = 440; method: REML)
## 
##    logLik   Deviance        AIC        BIC       AICc   
## -281.0839   562.1679   568.1679   580.4214   568.2230   
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed            factor 
## sigma^2.1  0.1097  0.3312     72     no        article_id 
## sigma^2.2  0.1396  0.3736    440     no  article_id/es_id 
## 
## Test for Heterogeneity:
## Q(df = 439) = 10072.2559, p-val < .0001
## 
## Number of estimates:   440
## Number of clusters:    172
## Estimates per cluster: 1-24 (mean: 2.56, median: 2)
## 
## Model Results:
## 
## estimate      se¹     tval¹   df¹    pval¹    ci.lb¹    ci.ub¹      
##  -0.2107  0.0382   -5.5099   171   <.0001   -0.2862   -0.1352   *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 1) results based on cluster-robust inference (var-cov estimator: CR1,
##    approx t-test and confidence interval, df: residual method)

Prediction Interval

## 
##     pred     se   ci.lb   ci.ub   pi.lb  pi.ub 
##  -0.2107 0.0382 -0.2862 -0.1352 -1.1991 0.7776

Forest Plot

Sensitivity Analysis

We set eval = FALSE for this part of the analyses to keep the document concise. The full analyses can be found in the source script.

Publication Bias

We set eval = FALSE for this part of the analyses to keep the document concise. The full analyses can be found in the source script.

Heterogeneity

## Range of Effect Sizes (Cohen's d): [-2.43, 1.66]
## Total Heterogeneity (τ²): 0.25
## Ratio of True to Total Variance (I²): 96.15%

META-REGRESSIONS

AI Labels & Consumer Responses

Variation in Effect Sizes across AI Labels & Consumer Responses

Effect Size Estimates for Each AI Label

Parameter Estimates
Predictor Estimate SE t-value p-value
AI Algorithms -0.24 0.06 -3.95 < .001 ***
AI Assistants -0.22 0.09 -2.51 0.013 *
AI Robots -0.83 0.18 -4.65 < .001 ***
AI Systems -0.17 0.07 -2.44 0.016 *
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across AI Label Categories (Reference Category: AI Robots)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.83 0.16 -5.18 < .001 ***
AI Algorithms (vs. AI Robots) 0.59 0.17 3.46 < .001 ***
AI Assistants (vs. AI Robots) 0.61 0.17 3.67 < .001 ***
AI Systems (vs. AI Robots) 0.65 0.17 3.96 < .001 ***
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Consumer Response

Parameter Estimates
Predictor Estimate SE t-value p-value
Affective -0.31 0.05 -5.70 < .001 ***
Behavioral -0.22 0.05 -4.40 < .001 ***
Cognitive -0.11 0.05 -2.03 0.044 *
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Consumer Responses (Reference Category: Affective)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.31 0.05 -5.70 < .001 ***
Behavioral (vs. Affective) 0.09 0.06 1.60 0.11
Cognitive (vs. Affective) 0.20 0.06 3.12 0.002 **
Significance levels: *** p < .001; ** p < .01; * p < .05

AI Characteristics

Variation in Effect Sizes across AI Characteristics

Effect Size Estimates for Each Level of Autonomy

Parameter Estimates
Predictor Estimate SE t-value p-value
Extensive -0.28 0.04 -6.34 < .001 ***
Limited -0.05 0.06 -0.82 0.41
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Autonomy (Reference Category: Extensive)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.28 0.04 -6.34 < .001 ***
Limited (vs. Extensive) 0.23 0.06 3.74 < .001 ***
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Performance

Parameter Estimates
Predictor Estimate SE t-value p-value
Equivalent -0.23 0.06 -3.63 < .001 ***
Inferior -0.53 0.11 -4.63 < .001 ***
Superior -0.11 0.10 -1.12 0.27
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Performance (Reference Category: Inferior)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.53 0.11 -4.63 < .001 ***
Equivalent (vs. Inferior) 0.30 0.13 2.31 0.025 *
Superior (vs. Inferior) 0.42 0.16 2.62 0.011 *
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Anthropomorphism

Parameter Estimates
Predictor Estimate SE t-value p-value
Absent -0.23 0.04 -5.83 < .001 ***
Present -0.07 0.15 -0.44 0.66
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Anthropomorphism (Reference Category: Present)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.07 0.15 -0.44 0.66
Absent (vs. Present) -0.16 0.16 -1.01 0.32
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Uniqueness Recognition

Parameter Estimates
Predictor Estimate SE t-value p-value
Absent -0.24 0.05 -5.33 < .001 ***
Present -0.13 0.07 -1.87 0.06
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Uniqueness Recognition (Reference Category: Present)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.13 0.07 -1.87 0.06
Absent (vs. Present) -0.11 0.08 -1.33 0.19
Significance levels: *** p < .001; ** p < .01; * p < .05

Application Domain

Variation in Effect Sizes across AI Application Domains

Effect Size Estimates for Each AI Application Domain

Parameter Estimates
Predictor Estimate SE t-value p-value
Entertainment & Lifestyle -0.18 0.06 -2.89 0.004 **
Healthcare -0.19 0.10 -1.88 0.06
Investing & Finance -0.26 0.11 -2.37 0.019 *
Legal & Public Safety -0.45 0.13 -3.37 < .001 ***
Operations & Management -0.12 0.06 -1.89 0.06
Other -0.19 0.17 -1.13 0.26
Social Welfare -0.40 0.05 -8.18 < .001 ***
Transportation -0.80 0.36 -2.22 0.028 *
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across AI Application Domains (Reference Category: Operations & Management)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.12 0.06 -1.89 0.06
Entertainment & Lifestyle (vs. Operations & Management) -0.06 0.08 -0.69 0.49
Healthcare (vs. Operations & Management) -0.07 0.12 -0.55 0.58
Investing & Finance (vs. Operations & Management) -0.14 0.12 -1.12 0.26
Legal & Public Safety (vs. Operations & Management) -0.33 0.15 -2.14 0.034 *
Other (vs. Operations & Management) -0.07 0.18 -0.40 0.69
Social Welfare (vs. Operations & Management) -0.28 0.08 -3.49 < .001 ***
Transportation (vs. Operations & Management) -0.68 0.37 -1.84 0.07
Significance levels: *** p < .001; ** p < .01; * p < .05

Temporal AI Advancement

Variation in Effect Sizes Over Time

Effect Size Estimate for Temporal Evolution

Parameter Estimates
Predictor Estimate SE t-value p-value
Overall 0.09 0.03 3.35 0.001 **
Significance levels: *** p < .001; ** p < .01; * p < .05

Variation in Effect Sizes Across Consumer Responses Over Time

Comparison of Effect Sizes Across AI Development Phases

Parameter Estimates
Predictor Estimate SE t-value p-value
Before 2010 -0.33 0.10 -3.36 < .001 ***
2010-2014 -0.47 0.13 -3.72 < .001 ***
2015-2019 -0.42 0.08 -5.11 < .001 ***
2020-2025 -0.12 0.04 -2.67 0.008 **
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Consumer Responses and AI Development Phases

Cognitive Responses

Parameter Estimates
Predictor Estimate SE t-value p-value
Before 2010 -0.51 0.10 -5.30 < .001 ***
2010-2014 -0.40 0.10 -4.04 < .001 ***
2015-2019 -0.31 0.09 -3.38 0.001 **
2020-2025 0.02 0.07 0.32 0.75
Significance levels: *** p < .001; ** p < .01; * p < .05

Affective Responses

Parameter Estimates
Predictor Estimate SE t-value p-value
Before 2010 -0.09 0.17 -0.52 0.61
2010-2014 -0.72 0.19 -3.82 < .001 ***
2015-2019 -0.60 0.10 -6.08 < .001 ***
2020-2025 -0.17 0.08 -2.21 0.030 *
Significance levels: *** p < .001; ** p < .01; * p < .05

Behavioral Responses

Parameter Estimates
Predictor Estimate SE t-value p-value
Before 2010 -0.71 0.07 -9.58 < .001 ***
2015-2019 -0.32 0.16 -2.04 0.045 *
2020-2025 -0.19 0.05 -3.58 < .001 ***
Significance levels: *** p < .001; ** p < .01; * p < .05

Task and Study Characteristics

Variation in Effect Sizes across Task & Study Characteristics

Effect Size Estimates for Each Level of Task Objectivity

Parameter Estimates
Predictor Estimate SE t-value p-value
Objective -0.19 0.04 -4.37 < .001 ***
Subjective -0.26 0.07 -3.86 < .001 ***
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Task Objectivity (Reference Category: Objective)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.19 0.04 -4.37 < .001 ***
Subjective (vs. Objective) -0.07 0.08 -0.90 0.37
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Task Consequentiality

Parameter Estimates
Predictor Estimate SE t-value p-value
High -0.28 0.06 -4.84 < .001 ***
Low -0.17 0.05 -3.61 < .001 ***
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Task Consequentiality (Reference Category: Low)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.17 0.05 -3.61 < .001 ***
High (vs. Low) -0.11 0.07 -1.59 0.11
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Human Benchmark

Parameter Estimates
Predictor Estimate SE t-value p-value
Expert -0.27 0.05 -5.74 < .001 ***
Non-Expert 0.01 0.09 0.09 0.93
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Human Benchmark (Reference Category: Non-Expert)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept 0.01 0.09 0.09 0.93
Expert (vs. Non-Expert) -0.27 0.11 -2.54 0.012 *
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of AI Explanation

Parameter Estimates
Predictor Estimate SE t-value p-value
Absent -0.21 0.04 -5.39 < .001 ***
Present -0.22 0.12 -1.80 0.07
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of AI Explanation (Reference Category: Absent)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.21 0.04 -5.39 < .001 ***
Present (vs. Absent) -0.01 0.12 -0.04 0.97
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Field Setting

Parameter Estimates
Predictor Estimate SE t-value p-value
No -0.21 0.04 -5.55 < .001 ***
Yes -0.25 0.15 -1.68 0.10
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Field Setting (Reference Category: No)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.21 0.04 -5.55 < .001 ***
Yes (vs. No) -0.04 0.14 -0.26 0.80
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Stimulus Presentation

Parameter Estimates
Predictor Estimate SE t-value p-value
Perceptually Rich -0.26 0.15 -1.82 0.07
Text Only -0.20 0.04 -5.26 < .001 ***
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Stimulus Presentation (Reference Category: Text Only)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.20 0.04 -5.26 < .001 ***
Perceptually Rich (vs. Text Only) -0.06 0.15 -0.41 0.68
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Incentive Compatibility

Parameter Estimates
Predictor Estimate SE t-value p-value
No -0.22 0.04 -5.44 < .001 ***
Yes -0.16 0.10 -1.60 0.11
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Incentive Compatibility (Reference Category: No)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.22 0.04 -5.44 < .001 ***
Yes (vs. No) 0.07 0.11 0.62 0.54
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Behavioral DV

Parameter Estimates
Predictor Estimate SE t-value p-value
No -0.23 0.04 -5.73 < .001 ***
Yes -0.14 0.08 -1.69 0.09
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Behavioral DV (Reference Category: No)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.23 0.04 -5.73 < .001 ***
Yes (vs. No) 0.09 0.08 1.07 0.29
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Within-Subject Design

Parameter Estimates
Predictor Estimate SE t-value p-value
No -0.19 0.04 -4.66 < .001 ***
Yes -0.38 0.09 -4.35 < .001 ***
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Within-Subject Design (Reference Category: No)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.19 0.04 -4.66 < .001 ***
Yes (vs. No) -0.19 0.09 -2.07 0.040 *
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Level of Online Sample

Parameter Estimates
Predictor Estimate SE t-value p-value
No -0.16 0.05 -3.34 0.001 **
Yes -0.26 0.06 -4.64 < .001 ***
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Levels of Online Sample (Reference Category: No)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.16 0.05 -3.34 0.001 **
Yes (vs. No) -0.09 0.07 -1.34 0.18
Significance levels: *** p < .001; ** p < .01; * p < .05

Sample Characteristics

Variation in Effect Sizes across Sample Characteristics

Effect Size Estimate for Age

Parameter Estimates
Predictor Estimate SE t-value p-value
Overall 0.02 0.07 0.24 0.81
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimate for Share of Female Participants

Parameter Estimates
Predictor Estimate SE t-value p-value
Overall 0.04 0.05 0.88 0.38
Significance levels: *** p < .001; ** p < .01; * p < .05

Effect Size Estimates for Each Geographical Region

Parameter Estimates
Predictor Estimate SE t-value p-value
No -0.16 0.05 -3.34 0.001 **
Yes -0.26 0.06 -4.64 < .001 ***
Significance levels: *** p < .001; ** p < .01; * p < .05

Comparison of Effect Sizes Across Geographical Regions (Reference Category: Europe)

Parameter Estimates
Predictor Estimate SE t-value p-value
Intercept -0.16 0.09 -1.77 0.08
America (vs. Europe) -0.08 0.11 -0.77 0.44
Asia (vs. Europe) -0.11 0.19 -0.58 0.57
Significance levels: *** p < .001; ** p < .01; * p < .05

AI LABELS X APPLICATION DOMAINS

SESSION INFO

## R version 4.3.2 (2023-10-31)
## Platform: aarch64-apple-darwin20 (64-bit)
## Running under: macOS Sonoma 14.6.1
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.0.dylib 
## LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## time zone: Europe/Zurich
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] broom_1.0.5         kableExtra_1.4.0    patchwork_1.2.0    
##  [4] cowplot_1.1.3       forcats_1.0.0       readr_2.1.5        
##  [7] metafor_4.6-0       numDeriv_2016.8-1.1 metadat_1.2-0      
## [10] Matrix_1.6-5        ggplot2_3.5.0       dplyr_1.1.4        
## [13] stringr_1.5.1       knitr_1.46         
## 
## loaded via a namespace (and not attached):
##  [1] tidyr_1.3.1       sass_0.4.9        utf8_1.2.4        generics_0.1.3   
##  [5] xml2_1.3.6        stringi_1.8.3     lattice_0.22-6    hms_1.1.3        
##  [9] digest_0.6.35     magrittr_2.0.3    evaluate_0.23     grid_4.3.2       
## [13] fastmap_1.1.1     jsonlite_1.8.8    backports_1.4.1   mgcv_1.9-1       
## [17] groundhog_3.2.1   purrr_1.0.2       fansi_1.0.6       viridisLite_0.4.2
## [21] scales_1.3.0      jquerylib_0.1.4   cli_3.6.2         rlang_1.1.3      
## [25] splines_4.3.2     munsell_0.5.1     withr_3.0.0       cachem_1.0.8     
## [29] yaml_2.3.8        tools_4.3.2       parallel_4.3.2    tzdb_0.4.0       
## [33] colorspace_2.1-0  mathjaxr_1.6-0    vctrs_0.6.5       R6_2.5.1         
## [37] lifecycle_1.0.4   pkgconfig_2.0.3   pillar_1.9.0      bslib_0.7.0      
## [41] gtable_0.3.5      glue_1.7.0        systemfonts_1.0.6 highr_0.10       
## [45] xfun_0.43         tibble_3.2.1      tidyselect_1.2.1  rstudioapi_0.16.0
## [49] farver_2.1.1      htmltools_0.5.8.1 nlme_3.1-164      labeling_0.4.3   
## [53] svglite_2.1.3     rmarkdown_2.26    compiler_4.3.2