Package {matchednull}

Title:	Matched-Null Tests for Cluster-Count Claims
Version:	0.2.2
Description:	Builds matched nulls for cluster-count claims: synthetic twins of a dataset that preserve every marginal distribution exactly and the correlation matrix approximately, while containing no cluster structure by construction. A reported number of clusters or "types" can then be tested against what the data's own margins and covariance already produce, using any clustering pipeline. A t-copula option is a sensitivity check that adds tail dependence to the reference, so an apparent excess of clusters can be rerun against a heavier-tailed reference; neither outcome of that rerun establishes that types exist. Implements the matched-null procedure of Meng (2026) "Types Without Taxa" <doi:10.17605/OSF.IO/2EKCG>.
License:	MIT + file LICENSE
Encoding:	UTF-8
Imports:	stats
Suggests:	mclust, testthat (≥ 3.0.0), knitr, rmarkdown, future.apply, future
VignetteBuilder:	knitr
Config/testthat/edition:	3
URL:	https://haomeng797-ship-it.github.io/matchednull/, https://github.com/haomeng797-ship-it/matchednull
BugReports:	https://github.com/haomeng797-ship-it/matchednull/issues
Config/roxygen2/version:	8.0.0
NeedsCompilation:	no
Packaged:	2026-11-02 00:32:34 UTC; menghao
Author:	Miura Meng [aut, cre]
Maintainer:	Miura Meng <haomeng797@gmail.com>
Repository:	CRAN
Date/Publication:	2026-11-02 01:00:02 UTC

Draw one matched null ("null twin") of a dataset

Description

Builds a synthetic twin of x that preserves every marginal distribution exactly and reproduces the correlation matrix approximately, while containing no cluster structure by construction. Each column of the twin is a permutation of the corresponding column of x, so no marginal test can tell them apart. The correlations are only approximately reproduced: the observed Pearson correlation matrix serves as the latent correlation of the copula, and once the non-normal empirical margins are restored by rank reordering the output Pearson correlations are not guaranteed to equal it. The gap is not sampling noise alone: it carries a random component from the finite draw and, for skewed or heavy-tailed margins, a systematic component as well. With copula = "gaussian" (the default) all dependence in the twin is Gaussian. Clustering found in real data but not in its twins indicates structure this reference does not carry; clustering found equally in both is no detected excess under this reference, which is not evidence that clusters are absent.

Usage

copula_null(x, copula = c("gaussian", "t"), df = 8, ridge = 1e-06)

Arguments

Details

The construction is a rank reordering in the tradition of Iman and Conover (1982): draw a Gaussian sample with the data's correlation matrix as its latent correlation, then replace each column with the sorted real values laid down in the rank order of the Gaussian column. Every real value is reused exactly once per column, which is why the margins match exactly. When the observed column has no ties the sorted values are laid down in a strictly increasing map of the Gaussian column, so the twin's rank correlations are those of the Gaussian sample. Tied observed values are laid down together, the twin carries those ties, and the rank pattern then depends on how the ties are broken, so the equality no longer holds. The twin's Pearson correlations are close to, but not in general equal to, the observed ones.

With copula = "t" the same construction is driven by a multivariate t sample instead: margins are preserved exactly and correlations approximately as before, but the twin also carries tail dependence, so extreme values across variables arrive together, the more strongly the smaller df. A t twin is still a single population with no clusters. It is a sensitivity variant of the reference model with heavier-tailed dependence: a departure from the Gaussian twins that the t twins reproduce is consistent with tail dependence, and one they do not reproduce is harder to attribute to tails. Neither outcome by itself establishes or rules out clusters.

Value

A numeric matrix of the same dimensions as x: one matched-null draw. Each column is a permutation of the corresponding column of x, so the margins are exact; the correlations are approximate.

References

Iman, R. L., & Conover, W. J. (1982). A distribution-free approach to inducing rank correlation among input variables. Communications in Statistics - Simulation and Computation, 11(3), 311-334.

Examples

set.seed(1)
x <- matrix(rnorm(200 * 3), 200, 3) %*% chol(matrix(c(1, .5, .3,
                                                       .5, 1, .4,
                                                       .3, .4, 1), 3, 3))
twin <- copula_null(x)
# margins identical:
all(sort(twin[, 1]) == sort(x[, 1]))
# correlations: close, not exact
round(cor(x) - cor(twin), 2)

# a heavier-tailed twin, as a sensitivity variant:
stress <- copula_null(x, copula = "t", df = 3)
all(sort(stress[, 1]) == sort(x[, 1]))

Test a cluster count against matched nulls

Description

Runs the user's own clustering pipeline on the real data and on R matched-null twins of it (see copula_null()), and asks whether the real result exceeds what the twins produce. The twins are a reference model fitted to the data: every twin column is a permutation of the observed values, so the margins are preserved exactly; the correlations are reproduced approximately (see copula_null() for why); and the twins contain no cluster structure by construction. The pipeline is supplied as a function, so any procedure that returns a number, a BIC-selected mixture, a k-means heuristic, a published typology's own workflow, can be tested unchanged.

Usage

matched_null_test(
  x,
  cluster_fn,
  R = 200,
  probs = c(0.025, 0.975),
  copula = c("gaussian", "t"),
  df = 8,
  ridge = 1e-06,
  parallel = FALSE
)

Arguments

Details

cluster_fn must return one finite number on every call. If it returns NA, NaN, Inf, a non-numeric value, or a vector of the wrong length, on the real data or on any twin, the test stops with an error naming the offending call. Invalid twins are never dropped and R is never quietly reduced, because an exceedance computed from a subset of the twins would not be the one that was asked for. Errors raised inside cluster_fn propagate unchanged.

p_exceed is a plug-in Monte Carlo exceedance quantity: one plus the number of twins whose statistic reaches the real value, divided by R + 1, under a reference fitted to the same data. It is not an exact finite-sample p-value. The reference's correlations are estimated from x and reproduced only approximately, so p_exceed is a comparison against that fitted reference, not a test with guaranteed size.

Reading the outcome. A real statistic above the twins' interval means the data depart from this reference model, one that carries their margins, approximately their correlations, and no cluster structure. It does not by itself establish that clusters exist: structure the reference does not carry, heavy-tailed dependence included, can produce the same departure, and whether a particular departure registers depends on the statistic, the pipeline and the sample size. A real statistic inside the interval means no detected excess under this reference; it is not evidence that clusters are absent. A statistic below the interval is reported as a diagnostic (the pipeline finds less on the data than on its twins), not as a verdict.

copula = "t" is a sensitivity variant. It gives the twins heavier-tailed dependence at a df fixed by the caller (8, then 3, is a reasonable pair), so a rerun shows whether a departure from the Gaussian twins persists once tail co-movement is built into the reference. A departure that persists is harder to attribute to tails; one that disappears is consistent with tail dependence. Neither outcome by itself establishes or rules out clusters. The Gaussian run is the primary analysis and the t run is reported alongside it, not in its place.

Value

An object of class "matched_null_test": a list with the real statistic (real), the R twin statistics (null), the reference interval (interval, the probs quantiles of null), the plug-in Monte Carlo exceedance quantity (p_exceed, computed as (1 + sum(null >= real)) / (R + 1); not an exact finite-sample p-value, see Details), whether the real statistic lies inside the interval (within), the requested R, and the reference family used (copula, with df when it is "t"). Reproducibility is the caller's: set a seed before calling.

Examples

if (requireNamespace("mclust", quietly = TRUE)) {
  suppressPackageStartupMessages(library(mclust))
  # a pipeline: how many components does BIC select?
  pick_k <- function(d) Mclust(d, G = 1:5, verbose = FALSE)$G
  set.seed(42)
  x <- matrix(rnorm(500 * 4), 500, 4)  # typeless data
  matched_null_test(x, pick_k, R = 30)

  # sensitivity variant with heavier-tailed dependence in the reference:
  # matched_null_test(x, pick_k, R = 30, copula = "t", df = 8)
}