RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity

Author

Phillip Alday, Douglas Bates, and Reinhold Kliegl

Published

2025-12-17

1 Background

Kliegl et al. (2015) is a follow-up to Kliegl et al. (2011) (see also script kwdyz11.qmd) from an experiment looking at a variety of effects of visual cueing under four different cue-target relations (CTRs). In this experiment two rectangles are displayed (1) in horizontal orientation , (2) in vertical orientation, (3) in left diagonal orientation, or in (4) right diagonal orientation relative to a central fixation point. Subjects react to the onset of a small or a large visual target occurring at one of the four ends of the two rectangles. The target is cued validly on 70% of trials by a brief flash of the corner of the rectangle at which it appears; it is cued invalidly at the three other locations 10% of the trials each. This implies a latent imbalance in design that is not visible in the repeated-measures ANOVA, but we will show its effect in the random-effect structure and conditional modes.

There are a couple of differences between the first and this follow-up experiment, rendering it more a conceptual than a direct replication. First, the original experiment was carried out at Peking University and this follow-up at Potsdam University. Second, diagonal orientations of rectangles and large target sizes were not part of the design of Kliegl et al. (2011).

We specify three contrasts for the four-level factor CTR that are derived from spatial, object-based, and attractor-like features of attention. They map onto sequential differences between appropriately ordered factor levels. Replicating Kliegl et al. (2011), the attraction effect was not significant as a fixed effect, but yielded a highly reliable variance component (VC; i.e., reliable individual differences in positive and negative attraction effects cancel the fixed effect). Moreover, these individual differences in the attraction effect were negatively correlated with those in the spatial effect.

This comparison is of interest because a few years after the publication of Kliegl et al. (2011), the theoretically critical correlation parameter (CP) between the spatial effect and the attraction effect was determined as the source of a non-singular LMM in that paper. The present study served the purpose to estimate this parameter with a larger sample and a wider variety of experimental conditions.

Here we also include two additional experimental manipulations of target size and orientation of cue rectangle. A similar analysis was reported in the parsimonious mixed-model paper (Bates et al., 2015); it was also used in a paper of GAMEMs (Baayen et al., 2017). Data and R scripts of those analyses are also available in R-package RePsychLing.

The analysis is based on reaction times rt to maintain compatibility with Kliegl et al. (2011).

In this vignette we focus on the reduction of model complexity. And we start with a quote:

“Neither the [maximal] nor the [minimal] linear mixed models are appropriate for most repeated measures analysis. Using the [maximal] model is generally wasteful and costly in terms of statiscal power for testing hypotheses. On the other hand, the [minimal] model fails to account for nontrivial correlation among repeated measurements. This results in inflated [T]ype I error rates when non-negligible correlation does in fact exist. We can usually find middle ground, a covariance model that adequately accounts for correlation but is more parsimonious than the [maximal] model. Doing so allows us full control over [T]ype I error rates without needlessly sacrificing power.”

Stroup, W. W. (2012, p. 185). Generalized linear mixed models: Modern concepts, methods and applica?ons. CRC Press, Boca Raton.

2 Packages

Code

using AlgebraOfGraphics
using BoxCox
using CairoMakie
using CategoricalArrays
using Chain
using DataFrameMacros
using DataFrames
using MixedModels
using MixedModelsMakie
using Random
using StatsBase

using AlgebraOfGraphics: density
using SMLP2026: dataset

progress = isinteractive()

3 Read data, compute and plot means

dat = DataFrame(dataset(:kkl15))
describe(dat)

5×7 DataFrame

Row	variable	mean	min	median	max	nmissing	eltype
	Symbol	Union…	Any	Union…	Any	Int64	DataType
1	Subj		S001		S147	0	String
2	CTR		dod		val	0	String
3	rt	293.147	150.22	276.594	749.481	0	Float32
4	cardinal		cardinal		diagonal	0	String
5	size		big		small	0	String

dat_subj = combine(
  groupby(dat, [:Subj, :CTR]),
  nrow => :n,
  :rt => mean => :rt_m,
  :rt => (c -> mean(log, c)) => :lrt_m,
)
dat_subj.CTR = categorical(dat_subj.CTR, levels=levels(dat.CTR))
describe(dat_subj)

5×7 DataFrame

Row	variable	mean	min	median	max	nmissing	eltype
	Symbol	Union…	Any	Union…	Any	Int64	DataType
1	Subj		S001		S147	0	String
2	CTR		val		dod	0	CategoricalValue{String, UInt32}
3	n	156.294	49	64.0	448	0	Int64
4	rt_m	308.223	208.194	304.862	584.71	0	Float32
5	lrt_m	5.6908	5.33226	5.69848	6.36141	0	Float32

Code

boxplot(
  dat_subj.CTR.refs,
  dat_subj.lrt_m;
  orientation=:horizontal,
  show_notch=true,
  axis=(;
    yticks=(
      1:4,
      [
        "valid cue",
        "same obj/diff pos",
        "diff obj/same pos",
        "diff obj/diff pos",
      ]
    )
  ),
  figure=(; resolution=(800, 300)),
)

┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/FUAHr/src/scenes.jl:238

Figure 1: Comparative boxplots of mean response time by subject under different conditions

Mean of reaction times for four cue-target relations. Targets appeared at (a) the cued position (valid) in a rectangle, (b) in the same rectangle cue, but at its other end, (c) on the second rectangle, but at a corresponding horizontal/vertical physical distance, or (d) at the other end of the second rectangle, that is \(\sqrt{2}\) of horizontal/vertical distance diagonally across from the cue, that is also at larger physical distance compared to (c).

4 Contrasts

contrasts = Dict(
  :CTR => SeqDiffCoding(; levels=["val", "sod", "dos", "dod"]),
  :cardinal => EffectsCoding(; levels=["cardinal", "diagonal"]),
  :size => EffectsCoding(; levels=["big", "small"])
)

Dict{Symbol, StatsModels.AbstractContrasts} with 3 entries:
  :CTR      => SeqDiffCoding(["val", "sod", "dos", "dod"])
  :size     => EffectsCoding(nothing, ["big", "small"])
  :cardinal => EffectsCoding(nothing, ["cardinal", "diagonal"])

5 Maximum LMM

This is the maximum LMM for the design; size is a between-subject factor, ignoring other information such as trial number, age and gender of subjects.

m_max = let
  form = @formula rt ~ 1 + CTR * cardinal * size +
                           (1 + CTR * cardinal | Subj)
  fit(MixedModel, form, dat; contrasts)
end;

Minimizing 2    Time: 0:00:00 (79.43 ms/it)
   objective: 600847.1219998157


Minimizing 1363    Time: 0:00:00 ( 0.39 ms/it)

issingular(m_max)

true

only(MixedModels.PCA(m_max))


Principal components based on correlation matrix
 (Intercept)                     1.0   …    .      .      .      .      .
 CTR: sod                        0.6        .      .      .      .      .
 CTR: dos                        0.44       .      .      .      .      .
 CTR: dod                        0.37      1.0     .      .      .      .
 cardinal: diagonal             -0.03      0.15   1.0     .      .      .
 CTR: sod & cardinal: diagonal   0.32  …   0.02   0.26   1.0     .      .
 CTR: dos & cardinal: diagonal   0.41     -0.06   0.23   0.45   1.0     .
 CTR: dod & cardinal: diagonal   0.15      0.89   0.11   0.27   0.05   1.0

Normalized cumulative variances:
[0.3581, 0.5916, 0.7445, 0.8637, 0.9443, 1.0, 1.0, 1.0]

Component loadings
                                  PC1  …    PC4    PC5    PC6    PC7    PC8
 (Intercept)                    -0.43     -0.16  -0.31   0.46  -0.19   0.46
 CTR: sod                       -0.36     -0.42  -0.0   -0.22  -0.12  -0.6
 CTR: dos                       -0.38      0.35  -0.11  -0.75  -0.06   0.17
 CTR: dod                       -0.45      0.22  -0.19   0.11   0.7    0.04
 cardinal: diagonal             -0.12  …  -0.56  -0.47  -0.18  -0.13   0.1
 CTR: sod & cardinal: diagonal  -0.32     -0.4    0.73   0.0    0.26   0.16
 CTR: dos & cardinal: diagonal  -0.22      0.22  -0.17   0.33   0.05  -0.6
 CTR: dod & cardinal: diagonal  -0.41      0.32   0.29   0.17  -0.61   0.0

VarCorr(m_max)

	Column	Variance	Std.Dev	Corr.
Subj	(Intercept)	2922.19580	54.05734
	CTR: sod	454.31268	21.31461	+0.60
	CTR: dos	57.69581	7.59578	+0.44	+0.18
	CTR: dod	192.67411	13.88071	+0.37	+0.54	+0.29
	cardinal: diagonal	262.32348	16.19640	-0.03	-0.01	+0.00	+0.15
	CTR: sod & cardinal: diagonal	30.08951	5.48539	+0.32	+0.21	+0.34	+0.02	+0.26
	CTR: dos & cardinal: diagonal	1.59272	1.26203	+0.41	-0.38	+0.55	-0.06	+0.23	+0.45
	CTR: dod & cardinal: diagonal	13.78487	3.71280	+0.15	+0.29	+0.25	+0.89	+0.11	+0.27	+0.05
Residual		3972.12250	63.02478

The LMM m_max is overparameterized but it is not immediately apparent why.

6 Reduction strategy 1

6.1 Zero-correlation parameter LMM (1)

Force CPs to zero. Reduction strategy 1 is more suited for reducing model w/o theoretical expectations about CPs. The better reduction strategy for the present experiment with an a priori interest in CPs is described as Reduction strategy 2.

m_zcp1 = let
  form = @formula rt ~ 1 + CTR * cardinal * size +
                   zerocorr(1 + CTR * cardinal | Subj)
  fit(MixedModel, form, dat; contrasts)
end;

issingular(m_zcp1)

true

only(MixedModels.PCA(m_zcp1))


Principal components based on correlation matrix
 (Intercept)                    1.0  .    .    .    .    .    .    .
 CTR: sod                       0.0  1.0  .    .    .    .    .    .
 CTR: dos                       0.0  0.0  1.0  .    .    .    .    .
 CTR: dod                       0.0  0.0  0.0  1.0  .    .    .    .
 cardinal: diagonal             0.0  0.0  0.0  0.0  1.0  .    .    .
 CTR: sod & cardinal: diagonal  0.0  0.0  0.0  0.0  0.0  1.0  .    .
 CTR: dos & cardinal: diagonal  0.0  0.0  0.0  0.0  0.0  0.0  0.0  .
 CTR: dod & cardinal: diagonal  0.0  0.0  0.0  0.0  0.0  0.0  0.0  1.0

Normalized cumulative variances:
[0.1429, 0.2857, 0.4286, 0.5714, 0.7143, 0.8571, 1.0, 1.0]

Component loadings
                                 PC1  …   PC3   PC4   PC5   PC6   PC7   PC8
 (Intercept)                    1.0      0.0   0.0   0.0   0.0   0.0   0.0
 CTR: sod                       0.0      0.0   0.0   0.0   0.0   0.0   0.0
 CTR: dos                       0.0      1.0   0.0   0.0   0.0   0.0   0.0
 CTR: dod                       0.0      0.0   0.0   0.0   0.0   1.0   0.0
 cardinal: diagonal             0.0   …  0.0   1.0   0.0   0.0   0.0   0.0
 CTR: sod & cardinal: diagonal  0.0      0.0   0.0   1.0   0.0   0.0   0.0
 CTR: dos & cardinal: diagonal  0.0      0.0   0.0   0.0   0.0   0.0   1.0
 CTR: dod & cardinal: diagonal  0.0      0.0   0.0   0.0   1.0   0.0   0.0

VarCorr(m_zcp1)

	Column	Variance	Std.Dev	Corr.
Subj	(Intercept)	2875.36834	53.62246
	CTR: sod	482.88548	21.97466	.
	CTR: dos	79.81159	8.93373	.	.
	CTR: dod	217.17364	14.73681	.	.	.
	cardinal: diagonal	250.29104	15.82059	.	.	.	.
	CTR: sod & cardinal: diagonal	35.96505	5.99709	.	.	.	.	.
	CTR: dos & cardinal: diagonal	0.00000	0.00000	.	.	.	.	.	.
	CTR: dod & cardinal: diagonal	6.85703	2.61859	.	.	.	.	.	.	.
Residual		3972.38091	63.02683

The LMM m_zcp1 is also overparameterized, but now there is clear evidence for absence of evidence for the VC of one of the interactions and the other two interaction-based VCs are also very small.

6.2 Reduced zcp LMM

Take out VCs for interactions.

m_zcp1_rdc = let
  form = @formula rt ~ 1 + CTR * cardinal * size +
                   zerocorr(1 + CTR + cardinal | Subj)
  fit(MixedModel, form, dat; contrasts)
end;

issingular(m_zcp1_rdc)

false

only(MixedModels.PCA(m_zcp1_rdc))


Principal components based on correlation matrix
 (Intercept)         1.0  .    .    .    .
 CTR: sod            0.0  1.0  .    .    .
 CTR: dos            0.0  0.0  1.0  .    .
 CTR: dod            0.0  0.0  0.0  1.0  .
 cardinal: diagonal  0.0  0.0  0.0  0.0  1.0

Normalized cumulative variances:
[0.2, 0.4, 0.6, 0.8, 1.0]

Component loadings
                      PC1   PC2   PC3   PC4   PC5
 (Intercept)         0.0   0.0   0.0   1.0   0.0
 CTR: sod            1.0   0.0   0.0   0.0   0.0
 CTR: dos            0.0   1.0   0.0   0.0   0.0
 CTR: dod            0.0   0.0   1.0   0.0   0.0
 cardinal: diagonal  0.0   0.0   0.0   0.0   1.0

VarCorr(m_zcp1_rdc)

	Column	Variance	Std.Dev	Corr.
Subj	(Intercept)	2880.90084	53.67402
	CTR: sod	484.39077	22.00888	.
	CTR: dos	79.50606	8.91662	.	.
	CTR: dod	216.56629	14.71619	.	.	.
	cardinal: diagonal	244.93221	15.65031	.	.	.	.
Residual		3980.88042	63.09422

LMM m_zcp_rdc is ok . We add in CPs.

6.3 Parsimonious LMM (1)

Extend zcp-reduced LMM with CPs

m_prm1 = let
  form = @formula rt ~ 1 + CTR * cardinal * size +
                           (1 + CTR + cardinal | Subj)
  fit(MixedModel, form, dat; contrasts)
end;

issingular(m_prm1)

false

only(MixedModels.PCA(m_prm1))


Principal components based on correlation matrix
 (Intercept)          1.0     .      .      .      .
 CTR: sod             0.6    1.0     .      .      .
 CTR: dos             0.45   0.19   1.0     .      .
 CTR: dod             0.37   0.54   0.3    1.0     .
 cardinal: diagonal  -0.1   -0.05  -0.09   0.13   1.0

Normalized cumulative variances:
[0.4489, 0.667, 0.8308, 0.9439, 1.0]

Component loadings
                       PC1    PC2    PC3    PC4    PC5
 (Intercept)         -0.55  -0.16   0.0    0.58  -0.57
 CTR: sod            -0.54   0.09  -0.49   0.16   0.66
 CTR: dos            -0.4   -0.25   0.8   -0.12   0.34
 CTR: dod            -0.49   0.36  -0.09  -0.71  -0.35
 cardinal: diagonal   0.05   0.88   0.32   0.34   0.07

LMM m_zcp_rdc is ok . We add in CPs.

VarCorr(m_prm1)

	Column	Variance	Std.Dev	Corr.
Subj	(Intercept)	2923.31335	54.06767
	CTR: sod	454.67288	21.32306	+0.60
	CTR: dos	56.63260	7.52546	+0.45	+0.19
	CTR: dod	188.56029	13.73173	+0.37	+0.54	+0.30
	cardinal: diagonal	245.42177	15.66594	-0.10	-0.05	-0.09	+0.13
Residual		3981.72067	63.10088

We note that the critical correlation parameter between spatial (sod) and attraction (dod) is now estimated at .54 – not that close to the 1.0 boundary that caused singularity in Kliegl et al. (2011).

6.4 Model comparison 1

gof_summary = let
  nms = [:m_zcp1_rdc, :m_prm1, :m_max]
  mods = eval.(nms)
  lrt = MixedModels.likelihoodratiotest(m_zcp1_rdc, m_prm1, m_max)
  DataFrame(;
    name = nms,
    dof=dof.(mods),
    deviance=round.(deviance.(mods), digits=0),
    AIC=round.(aic.(mods),digits=0),
    AICc=round.(aicc.(mods),digits=0),
    BIC=round.(bic.(mods),digits=0),
    χ²=vcat(:., round.(Int, diff(collect(lrt.lrt.deviance)))),
    χ²_dof=vcat(:., diff(collect(lrt.lrt.dof))),
    # StatsBase.PValue includes some pretty-printing methods
    pvalue=vcat(:., StatsBase.PValue.(collect(lrt.lrt.pval[2:end])))
  )
end

3×9 DataFrame

Row	name	dof	deviance	AIC	AICc	BIC	χ²	χ²_dof	pvalue
	Symbol	Int64	Float64	Float64	Float64	Float64	Any	Any	Any
1	m_zcp1_rdc	22	599486.0	599530.0	599530.0	599725.0	.	.	.
2	m_prm1	32	599418.0	599482.0	599482.0	599766.0	-68	10	<1e-09
3	m_max	53	599359.0	599465.0	599465.0	599936.0	-59	21	<1e-04

AIC prefers LMM m_prm1 over m_zcp1_rdc; BIC LMM m_zcp1_rdc. As the CPs were one reason for conducting this experiment, AIC is the criterion of choice.

7 Reduction strategy 2

7.1 Complex LMM

Relative to LMM m_max, first we take out interaction VCs and associated CPs, because these VCs are very small. This is the same as LMM m_prm1 above.

m_cpx = let
  form = @formula rt ~ 1 + CTR * cardinal * size +
                      (1 + CTR + cardinal | Subj)
  fit(MixedModel, form, dat; contrasts)
end;

7.2 Zero-correlation parameter LMM (2)

Now we check the significance of ensemble of CPs.

m_zcp2 = let
  form = @formula rt ~ 1 + CTR * cardinal * size  +
              zerocorr(1 + CTR + cardinal | Subj)
  fit(MixedModel, form, dat; contrasts)
end;

VarCorr(m_zcp2)

	Column	Variance	Std.Dev	Corr.
Subj	(Intercept)	2880.90084	53.67402
	CTR: sod	484.39077	22.00888	.
	CTR: dos	79.50606	8.91662	.	.
	CTR: dod	216.56629	14.71619	.	.	.
	cardinal: diagonal	244.93221	15.65031	.	.	.	.
Residual		3980.88042	63.09422

7.3 Parsimonious LMM (2)

The cardinal-related CPs are quite small. Do we need them?

m_prm2 = let
  form = @formula(rt ~ 1 + CTR * cardinal * size  +
                      (1 + CTR | Subj) + (0 + cardinal | Subj))
  fit(MixedModel, form, dat; contrasts)
end;

VarCorr(m_prm2)

	Column	Variance	Std.Dev	Corr.
Subj	(Intercept)	2924.00272	54.07405
	CTR: sod	454.44827	21.31779	+0.60
	CTR: dos	56.45775	7.51384	+0.45	+0.19
	CTR: dod	187.90094	13.70770	+0.37	+0.54	+0.30
	cardinal: diagonal	245.17053	15.65792	.	.	.	.
Residual		3981.74101	63.10104

7.4 Model comparison 2

gof_summary = let
  nms = [:m_zcp2, :m_prm2, :m_cpx, :m_max]
  mods = eval.(nms)
  lrt = MixedModels.likelihoodratiotest(m_zcp2, m_prm2, m_cpx, m_max)
  DataFrame(;
    name = nms,
    dof=dof.(mods),
    deviance=round.(deviance.(mods), digits=0),
    AIC=round.(aic.(mods),digits=0),
    AICc=round.(aicc.(mods),digits=0),
    BIC=round.(bic.(mods),digits=0),
    χ²=vcat(:., round.(Int, diff(collect(lrt.lrt.deviance)))),
    χ²_dof=vcat(:., diff(collect(lrt.lrt.dof))),
    # StatsBase.PValue includes some pretty-printing methods
    pvalue=vcat(:., StatsBase.PValue.(collect(lrt.lrt.pval[2:end])))
  )
end

4×9 DataFrame

Row	name	dof	deviance	AIC	AICc	BIC	χ²	χ²_dof	pvalue
	Symbol	Int64	Float64	Float64	Float64	Float64	Any	Any	Any
1	m_zcp2	22	599486.0	599530.0	599530.0	599725.0	.	.	.
2	m_prm2	28	599420.0	599476.0	599476.0	599725.0	-65	6	<1e-11
3	m_cpx	32	599418.0	599482.0	599482.0	599766.0	-2	4	0.6521
4	m_max	53	599359.0	599465.0	599465.0	599936.0	-59	21	<1e-04

The cardinal-related CPs could be removed w/o loss of goodness of fit. However, there is no harm in keeping them in the LMM. The data support both LMM m_prm2 and m_cpx (same as: m_prm1). We keep the slightly more complex LMM m_cpx (m_prm1).

8 Diagnostic plots of LMM residuals

Do model residuals meet LMM assumptions? Classic plots are

Residual over fitted
Quantiles of model residuals over theoretical quantiles of normal distribution

8.1 Residual-over-fitted plot

The slant in residuals show a lower and upper boundary of reaction times, that is we have have too few short and too few long residuals. Not ideal, but at least width of the residual band looks similar across the fitted values, that is there is no evidence for heteroskedasticity.

Code

CairoMakie.activate!(; type="png")
scatter(fitted(m_prm1), residuals(m_prm1); alpha=0.3)

Figure 2: Residuals versus fitted values for model m1

With many observations the scatterplot is not that informative. Contour plots or heatmaps may be an alternative.

Code

set_aog_theme!()
draw(
  data((; f=fitted(m_prm1), r=residuals(m_prm1))) *
  mapping(
    :f => "Fitted values from m1", :r => "Residuals from m1"
  ) *
  density();
)

Figure 3: Heatmap of residuals versus fitted values for model m1

8.2 Q-Q plot

The plot of quantiles of model residuals over corresponding quantiles of the normal distribution should yield a straight line along the main diagonal.

Code

CairoMakie.activate!(; type="png")
qqnorm(
  residuals(m_prm1);
  qqline=:none,
  axis=(;
    xlabel="Standard normal quantiles",
    ylabel="Quantiles of the residuals from model m1",
  ),
)

Figure 4: Quantile-quantile plot of the residuals for model m1 versus a standard normal

9 Conditional modes

9.1 Caterpillar plot

Code

cm1 = only(ranefinfo(m_prm1))
caterpillar!(Figure(; resolution=(800, 1200)), cm1; orderby=2)

┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/FUAHr/src/scenes.jl:238

Figure 5: Prediction intervals of the subject random effects in model m1

9.2 Shrinkage plot

Code

shrinkageplot!(Figure(; resolution=(1000, 1200)), m_prm1)

┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/FUAHr/src/scenes.jl:238

Figure 6: Shrinkage plots of the subject random effects in model m1L

10 Parametric bootstrap

Here we

generate a bootstrap sample
compute shortest covergage intervals for the LMM parameters
plot densities of bootstrapped parameter estimates for residual, fixed effects, variance components, and correlation parameters

10.1 Generate a bootstrap sample

We generate 2500 samples for the 15 model parameters (4 fixed effect, 7 VCs, 15 CPs, and 1 residual).

samp = parametricbootstrap(MersenneTwister(1234321), 2500, m_prm1;
                           optsum_overrides=(; ftol_rel=1e-8));

tbl = samp.tbl

Table with 48 columns and 2500 rows:
      obj        β01      β02      β03      β04       β05      β06      ⋯
    ┌────────────────────────────────────────────────────────────────────
 1  │ 5.99121e5  318.141  26.293   13.9172  6.74045   5.9774   27.2177  ⋯
 2  │ 5.99216e5  315.16   21.7191  13.9016  1.39841   4.28718  26.2128  ⋯
 3  │ 5.99389e5  304.789  22.8434  14.451   1.30953   5.97823  28.3132  ⋯
 4  │ 598759.0   304.984  25.7323  14.2535  4.46265   7.36731  25.5256  ⋯
 5  │ 5.99201e5  305.661  20.3514  11.7943  1.94445   7.55772  22.6509  ⋯
 6  │ 5.99825e5  303.206  20.6355  11.9631  3.00487   3.64828  14.0575  ⋯
 7  │ 5.98849e5  308.688  26.3973  14.9334  4.31493   9.43924  28.0372  ⋯
 8  │ 5.99044e5  308.728  22.0588  14.0064  2.05125   2.35669  21.4596  ⋯
 9  │ 5.99525e5  300.779  22.281   10.5455  4.47888   4.40489  21.3051  ⋯
 10 │ 5.99094e5  314.715  27.6056  12.8147  1.95067   5.65753  24.3927  ⋯
 11 │ 599566.0   313.361  25.4106  11.587   4.60696   9.92416  28.778   ⋯
 12 │ 5.99321e5  312.308  22.9888  15.053   0.255781  8.8578   25.8749  ⋯
 13 │ 5.99502e5  310.985  23.8325  14.7281  1.70401   9.73295  25.5336  ⋯
 14 │ 5.997e5    317.853  25.1802  13.9593  4.47966   8.08207  36.8839  ⋯
 15 │ 6.00081e5  309.606  24.4858  13.2237  2.80406   3.88078  24.2754  ⋯
 16 │ 6.00049e5  308.469  24.8433  10.4011  3.39769   6.34919  28.8349  ⋯
 17 │ 598988.0   313.964  24.6612  12.7279  3.89632   7.20685  27.1067  ⋯
 ⋮  │     ⋮         ⋮        ⋮        ⋮        ⋮         ⋮        ⋮     ⋱

10.2 Shortest coverage interval

confint(samp)

DictTable with 2 columns and 32 rows:
 par   lower      upper
 ────┬────────────────────
 β01 │ 297.133    319.552
 β02 │ 18.783     28.3183
 β03 │ 10.2093    15.9687
 β04 │ -1.01263   6.55024
 β05 │ 3.25452    9.79495
 β06 │ 14.0575    37.0171
 β07 │ 1.8633     5.39096
 β08 │ -0.992529  3.75391
 β09 │ -2.88891   1.78051
 β10 │ 3.90978    13.66
 β11 │ -3.31052   2.46331
 β12 │ 3.92582    11.1416
 β13 │ -1.06987   5.61839
 β14 │ -2.50356   1.05728
 β15 │ -2.49599   2.2805
 β16 │ 1.77487    6.49086
 ρ01 │ 0.409532   0.757823
  ⋮  │     ⋮         ⋮

We can also visualize the shortest coverage intervals for fixed effects with the ridgeplot() command:

Code

ridgeplot(samp; show_intercept=false)

Figure 7: Ridge plot of fixed-effects bootstrap samples from model m1L

10.3 Comparative density plots of bootstrapped parameter estimates

10.3.1 Residual

Code

draw(
  data(tbl) *
  mapping(:σ => "Residual") *
  density();
  figure=(; resolution=(800, 400)),
)

┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/FUAHr/src/scenes.jl:238

Figure 8: Kernel density estimate from bootstrap samples of the residual standard deviation for model m_prm1

10.3.2 Fixed effects and associated variance components (w/o GM)

The shortest coverage interval for the GM ranges from x to x ms and the associate variance component from .x to .x. To keep the plot range small we do not include their densities here.

Code

rn = renamer([
  "(Intercept)" => "GM",
  "CTR: sod" => "spatial effect",
  "CTR: dos" => "object effect",
  "CTR: dod" => "attraction effect",
  "(Intercept), CTR: sod" => "GM, spatial",
  "(Intercept), CTR: dos" => "GM, object",
  "CTR: sod, CTR: dos" => "spatial, object",
  "(Intercept), CTR: dod" => "GM, attraction",
  "CTR: sod, CTR: dod" => "spatial, attraction",
  "CTR: dos, CTR: dod" => "object, attraction",
])
draw(
  data(tbl) *
  mapping(
    [:β02, :β03, :β04] .=> "Experimental effect size [ms]";
    color=dims(1) =>
    renamer(["spatial effect", "object effect", "attraction effect"]) =>
    "Experimental effects",
  ) *
  density();
  figure=(; resolution=(800, 350)),
)

┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/FUAHr/src/scenes.jl:238

Figure 9: Kernel density estimate from bootstrap samples of the fixed effects for model m_prm1

The densitiies correspond nicely with the shortest coverage intervals.

Code

draw(
  data(tbl) *
  mapping(
    [:σ2, :σ3, :σ4] .=> "Standard deviations [ms]";
    color=dims(1) =>
    renamer(["spatial effect", "object effect", "attraction effect"]) =>
    "Variance components",
  ) *
  density();
  figure=(; resolution=(800, 350)),
)

┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/FUAHr/src/scenes.jl:238

Figure 10: Kernel density estimate from bootstrap samples of the standard deviations for model m1L (excluding Grand Mean)

The VC are all very nicely defined.

10.3.3 Correlation parameters (CPs)

Code

draw(
  data(tbl) *
  mapping(
    [:ρ01, :ρ02, :ρ03, :ρ04, :ρ05, :ρ06] .=> "Correlation";
    color=dims(1) =>
    renamer(["GM, spatial", "GM, object", "spatial, object",
    "GM, attraction", "spatial, attraction", "object, attraction"]) =>
    "Correlation parameters",
  ) *
  density();
  figure=(; resolution=(800, 350)),
)

┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
└ @ Makie ~/.julia/packages/Makie/FUAHr/src/scenes.jl:238

Figure 11: Kernel density estimate from bootstrap samples of the standard deviations for model m1L

Three CPs stand out positively, the correlation between GM and the spatial effect, GM and attraction effect, and the correlation between spatial and attraction effects. The second CP was positive, but not significant in the first study. The third CP replicates a CP that was judged questionable in script kwdyz11.jl. The three remaining CPs are not well defined for reaction times.

11 References

Baayen, H., Vasishth, S., Kliegl, R., & Bates, D. (2017). The cave of shadows: Addressing the human factor with generalized additive mixed models. Journal of Memory and Language, 94, 206–234. https://doi.org/10.1016/j.jml.2016.11.006

Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015). Parsimonious mixed models. arXiv. https://doi.org/10.48550/ARXIV.1506.04967

Kliegl, R., Kushela, J., & Laubrock, J. (2015). Object orientation and target size modulate the speed of visual attention. Department of Psychology, University of Potsdam.

Kliegl, R., Wei, P., Dambacher, M., Yan, M., & Zhou, X. (2011). Experimental effects and individual differences in linear mixed models: Estimating the relationship between spatial, object, and attraction effects in visual attention. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2010.00238

This page was rendered from git revision 60d6e27 using Quarto 1.7.33.