GRASP Technical Review: Replacing Redundant LLM Layers with Adaptive Singular Parameters

Paper: GRASP: Replace Redundant Layers with Adaptive Singular Parameters for Efficient Model Compression
Acronym: GRASP = Gradient-based Retention of Adaptive Singular Parameters
Authors: Kainan Liu, Yong Zhang, Ning Cheng, Zhitao Li, Shaojun Wang, Jing Xiao
Affiliations: Ping An Technology (Shenzhen); The Hong Kong University of Science and Technology (Guangzhou)
arXiv: 2501.00339v4, dated 22 Feb 2026
Code: https://github.com/LyoAI/GRASP

1. One-sentence summary

GRASP is a structured LLM compression method that first finds transformer layers whose input and output hidden states are very similar, then replaces those redundant layers with a small number of singular-vector components chosen by gradient-based sensitivity rather than by singular-value magnitude alone.

The important correction to keep in mind is that GRASP is not just “keep the largest SVD values.” The paper’s full name says adaptive singular parameters because a retained component is a whole singular group $\Phi_k = \{u_k, \sigma_k, v_k^\top\}$ , and the chosen groups are selected using gradients on a calibration set.

2. Why this paper exists

Large language models are expensive because every generated token has to pass through many transformer blocks. A 7B or 13B model may look small compared with frontier models, but it is still heavy for a single GPU, edge deployment, high-throughput serving, or low-latency applications.

Many compression methods attack this cost from different angles:

Quantization reduces numerical precision.
Unstructured pruning removes individual weights, often requiring sparse kernels to show real speedups.
Structured pruning removes larger structures such as layers, neurons, channels, or dimensions.
Low-rank compression replaces large matrices with products of smaller matrices.
Distillation trains a smaller model to mimic a larger model.

GRASP sits at the intersection of layer pruning and low-rank compression. It starts from the observation used by layer-pruning methods: some transformer layers are nearly redundant. But instead of deleting a redundant layer completely, GRASP asks a more careful question:

If a layer is mostly redundant, can we keep only the few internal directions that still matter?

That question is the whole paper.

3. The core problem: direct layer pruning is too blunt

A transformer layer is not always equally useful. Some layers strongly change the hidden representation; others perform a smaller adjustment. Prior work such as ShortGPT, LaCo, and LLM-Streamline exploits this by identifying layers that appear less important.

The blunt option is:

Layer i input ──> [full transformer layer] ──> Layer i output

If the layer looks redundant:
Layer i input ──────────────────────────────> Layer i output
                   skip / delete layer

This reduces latency, but it can break the model’s internal representation. A layer may be redundant in the sense that its input and output are close, while still preserving some small but important transformations. Removing it entirely can delete those transformations too.

LLM-Streamline-style methods try to avoid this by replacing pruned layers with lightweight trainable modules. That is less destructive, but those modules are often randomly initialized and therefore need extra training to recover performance.

GRASP proposes a middle path:

1
2
3

Do not keep the full redundant layer.
Do not replace it with a random new module.
Keep a small, data-informed low-rank part of the original layer.

4. Background for readers

4.1 Transformer layers and hidden states

A decoder-only LLM is a stack of transformer layers. At a high level:

input tokens
   ↓
embedding
   ↓
Layer 1
   ↓
Layer 2
   ↓
...
   ↓
Layer N
   ↓
language-model head
   ↓
next-token probabilities

Each layer receives a hidden state and outputs a new hidden state. If $H_i$ denotes the input hidden state of layer $i$ , and $H_{i+1}$ denotes the output after that layer, then the layer is doing something like:

$H_{i+1} = \text{Layer}_i(H_i).$

A layer is “transformative” if $H_{i+1}$ is meaningfully different from $H_i$ . It is “redundant” if $H_{i+1}$ is very similar to $H_i$ on representative data.

This is not the same as saying the layer is useless. It means the layer’s overall transformation is small, so the layer is a good candidate for compression.

4.2 Structured pruning versus unstructured pruning

It helps to distinguish two styles of pruning:

Type	What is removed?	Main benefit	Main caveat
Unstructured pruning	Individual weights	Can reach high sparsity	Speedups require sparse hardware/kernels
Structured pruning	Whole layers/modules/dimensions	Easier to realize latency gains	More likely to disrupt representations

GRASP is a structured method because it targets whole redundant layers, but its replacement is low-rank rather than simply deleted.

4.3 Singular Value Decomposition

For a weight matrix $W \in \mathbb{R}^{m \times n}$ , singular value decomposition writes:

$W = U \Sigma V^\top.$

Equivalently, this can be written as a sum of rank-one components:

$W = \sum_{k=1}^{l} u_k \sigma_k v_k^\top, \quad l = \min(m,n).$

Here:

$u_k$ is the $k$ -th left singular vector,
$v_k$ is the $k$ -th right singular vector,
$\sigma_k$ is the $k$ -th singular value,
$u_k \sigma_k v_k^\top$ is one rank-one direction in the matrix.

Most basic SVD compression keeps the largest singular values and discards the rest:

$\widetilde{W}_{\text{magnitude}} = \sum_{k=1}^{r} u_k \sigma_k v_k^\top.$

That is optimal for minimizing matrix reconstruction error under some norms, but the LLM does not care directly about matrix reconstruction error. It cares about language-model loss and downstream task performance.

This distinction is central to GRASP.

4.4 Gradient-based sensitivity

Suppose a parameter $\theta$ is changed or zeroed out. A first-order Taylor approximation says the loss change is roughly related to:

$\theta^\top \nabla_{\theta} L.$

A larger value suggests the parameter is more important for the loss, at least locally around the current model.

The full second-order form used as motivation in the paper is:

$T(\theta) = \theta^\top \nabla_\theta L + \frac{1}{2}\theta^\top H\theta + O(\|\theta\|^3),$

where $H$ is the Hessian. GRASP drops the second-order term for efficiency and uses a first-order gradient attribution score.

5. What GRASP does

The GRASP pipeline has two major stages.

            
            flowchart TD
    A[Dense LLM] --> B[Small calibration set]
    B --> C[Run forward pass]
    C --> D[Compute hidden-state cosine similarity per layer]
    D --> E[Select most redundant layers]
    E --> F[For each selected layer and each attention/MLP weight matrix]
    F --> G[SVD: W = U Sigma V^T]
    G --> H[Run gradient attribution on singular groups]
    H --> I[Keep top-r percent singular groups]
    I --> J[Reconstruct low-rank replacement]
    J --> K[Compressed GRASP model]
    K --> L{Optional?}
    L -->|No| M[Training-free compressed model]
    L -->|Yes| N[Light post-training compensation]

5.1 Stage 1: choose redundant layers by hidden-state similarity

GRASP follows prior layer-pruning work and measures how much each layer changes the hidden state. For a transformer layer with input hidden state $H_i$ and output hidden state $H_{i+1}$ , the paper uses cosine similarity:

$\cos(H_i, H_{i+1}) = \frac{H_i^\top H_{i+1}}{\|H_i\|_2 \|H_{i+1}\|_2}.$

Interpretation:

High cosine similarity means the layer output points in almost the same direction as the input.
That suggests the layer performs a relatively small transformation.
Layers with the highest similarity are selected as redundant candidates.

This stage is simple and cheap: it needs a calibration set and forward activations, not full retraining.

A useful mental model:

Low similarity layer:
H_i  ---- layer ---->  H_{i+1}
(direction changes a lot)
Do not prune first.

High similarity layer:
H_i  ---- layer ---->  H_{i+1}
(direction barely changes)
Candidate for GRASP replacement.

5.2 Stage 2: replace selected layers with adaptive singular parameters

For each selected redundant layer, GRASP looks at its weight matrices in attention and MLP modules. For a matrix $W$ , it computes:

$W = U\Sigma V^\top = \sum_{k=1}^{l} u_k \sigma_k v_k^\top.$

The paper defines a singular group:

$\Phi_k = \{u_k, \sigma_k, v_k^\top\}.$

Instead of retaining singular groups purely by the size of $\sigma_k$ , GRASP assigns each group an importance score based on gradient attribution.

The paper first writes the group score as:

$I(\Phi_k) = T(\sigma_k) + \sum_{i=1}^{m} T(u_{k,i}) + \sum_{j=1}^{n} T(v_{k,j}).$

Then, using the first-order approximation, it becomes:

$I(\Phi_k) = \sigma_k \frac{\partial L}{\partial \sigma_k} + \sum_{i=1}^{m} u_{k,i}\frac{\partial L}{\partial u_{k,i}} + \sum_{j=1}^{n} v_{k,j}\frac{\partial L}{\partial v_{k,j}}.$

The loss is the usual language-modeling objective:

$L = -\sum_t \log P(y_t \mid x_{\leq t}).$

After scoring all singular groups, GRASP keeps the top $r\%$ groups and reconstructs the matrix from them.

5.3 Why this is different from ordinary low-rank SVD

Ordinary truncated SVD would keep singular groups in this order:

1 2	Keep by magnitude: σ1, σ2, σ3, σ4, ...

GRASP can keep a non-contiguous set:

1 2	Keep by loss sensitivity: Φ2, Φ5, Φ9, Φ11, ...

That matters because the paper’s Figure 2 shows that larger singular values are not always the most important for downstream performance. A smaller singular direction can carry task-relevant information.

Conceptually:

Magnitude view:
"Big singular value = important."

GRASP view:
"A singular group is important if removing it changes the language-model loss."

5.4 Why the replacement remains efficient

A full matrix uses all singular groups. GRASP uses only a small retained subset inside selected redundant layers. If the layer is truly redundant and low-rank, the retained directions should preserve most of the layer’s useful effect while discarding a large amount of parameter mass.

The paper’s ablation finds that increasing the retain ratio from 0% to 10% gives a sharp accuracy improvement, while gains largely saturate after 10%. This is why the paper typically uses a 10% retain ratio.

A key nuance: retain ratio and overall compression ratio are not the same number. The retain ratio is the fraction of singular groups kept inside selected matrices. The overall model compression ratio also depends on how many layers are selected. In Table 6, when replacing 8 redundant layers of LLaMA2-7B, retain ratios of 0%, 5%, 10%, 15%, and 20% correspond to overall compression ratios of 24.0%, 22.8%, 21.6%, 20.4%, and 19.2%.

So when the paper says “20% compression ratio,” read it as roughly a 20% structured compression setting, not “only 20% of the original model remains.”

6. Algorithm in plain language

The paper’s algorithm can be restated as:

Choose a small calibration dataset, usually WikiText-2 in the experiments.
Run the model and collect layer input/output hidden states.
Compute cosine similarity for every layer.
Select the top- $L$ most similar layers as redundant.
Process selected redundant layers in reverse order, from later redundant layers backward.
For each selected layer:
- for each attention and MLP weight matrix,
- decompose $W$ with SVD,
- compute gradient-based importance for every singular group,
- keep the top $r\%$ groups,
- reconstruct the replacement matrix.
Return the compressed model.
Optionally run lightweight post-training compensation.

The reverse-order detail matters because later layers are closer to the output distribution. The paper also studies one-shot versus iterative pruning and finds similar accuracy, with one-shot being faster.

            
            flowchart LR
    A[Layer similarity ranking] --> B[Top-L redundant layers]
    B --> C[Reverse-order processing]
    C --> D[SVD per selected matrix]
    D --> E[Gradient score per singular group]
    E --> F[Keep top-r percent]
    F --> G[Low-rank layer replacement]

7. Experimental setup

The paper evaluates GRASP across several settings. The most important setup details are:

7.1 Models

The evaluated model families are LLaMA and Mistral. The paper reports experiments on:

LLaMA-7B
LLaMA2-7B
LLaMA2-13B
LLaMA3.1-8B-Instruct
Mistral-7B

7.2 Baselines

The baselines include three groups:

Layer-pruning baselines

ShortGPT
LaCo
LLM-Streamline

Module/structured pruning baselines

LLMPruner
SliceGPT

Low-rank pruning baselines

FWSVD
ASVD
SVD-LLM

The appendix also compares with unstructured 2:4 sparsity methods SparseGPT and Wanda.

7.3 Calibration and hardware

The main implementation details reported in the paper:

512 randomly sampled WikiText-2 examples are used as the calibration set in the main comparisons.
Experiments are run on NVIDIA A100-SXM4 80GB GPUs.
Zero-shot commonsense evaluations use the LM-Evaluation-Harness framework.
The post-training compensation setup uses Alpaca for one epoch, batch size 32, AdamW, learning rate $3\times 10^{-4}$ , maximum length 256, and mixed precision on a single A100.

The appendix lists LoRA-style hyperparameters for compensation: rank 256, alpha 16, dropout 0.05, and target modules including q_proj, k_proj, v_proj, o_proj, up_proj, down_proj, and gate_proj.

8. Main result 1: training-free structured compression

The first main comparison evaluates LLaMA3.1-8B-Instruct under a 20% compression setting without post-training compensation.

Benchmarks:

OpenBookQA
ARC-easy
WinoGrande
HellaSwag
ARC-challenge
PIQA
MathQA

The paper reports average zero-shot accuracy:

Method	Average	Retained dense performance
Dense	0.60	100.0%
LaCo	0.42	70.9%
ShortGPT	0.44	74.1%
SliceGPT	0.35	57.7%
GRASP	0.47	78.6%

Important reading:

GRASP is best among the listed structured-pruning baselines in this no-compensation table.
It does not preserve 90% of dense performance in this specific table; it preserves 78.6% of dense average accuracy.
The stronger “around 90%” claim is better supported in the low-rank comparison with post-training compensation at 20%, where GRASP* reaches 91.3% on LLaMA-7B in Table 3.

This distinction matters because paper abstracts often summarize across experiments, while a technical review should keep each table separate.

9. Main result 2: post-training compensation

The next comparison compresses LLaMA2-7B under a 25% compression ratio and allows post-training compensation.

The evaluation uses OpenCompass in PPL mode and reports accuracy on:

C3
CMNLI
CHID
BoolQ
WSC
CoQA
HellaSwag
PIQA
Race-M
Race-H
MMLU
CMMLU

Average results:

Method	Average	Retained dense performance
Dense	49.2	100.0%
LLMPruner*	38.4	78.1%
SliceGPT*	38.4	78.0%
LaCo*	40.3	82.0%
ShortGPT*	44.0	89.4%
LLM-Streamline-FFN*	45.8	93.1%
LLM-Streamline-Layer*	45.2	92.0%
GRASP	46.7	94.9%

The asterisked baseline numbers are reported from Chen et al. (2024), according to the paper.

The result supports the paper’s central claim: retaining original singular components gives the compressed model a better starting point than replacing layers with newly initialized modules.

10. Main result 3: low-rank pruning comparison across compression ratios

The paper also compares GRASP with low-rank pruning baselines on LLaMA-7B across 20% to 50% compression ratios. Benchmarks include seven commonsense reasoning datasets plus GSM8K.

Condensed average results:

Compression ratio	Method	Average	Retained dense performance
20%	FWSVD	0.28	60.8%
20%	ASVD	0.38	82.6%
20%	SVD-LLM	0.39	84.7%
20%	GRASP	0.39	84.7%
20%	GRASP*	0.42	91.3%
30%	SVD-LLM	0.35	76.1%
30%	GRASP	0.35	76.1%
30%	GRASP*	0.40	87.0%
40%	SVD-LLM	0.33	71.7%
40%	GRASP	0.32	69.6%
40%	GRASP*	0.38	82.6%
50%	SVD-LLM	0.29	63.0%
50%	GRASP	0.28	60.9%
50%	GRASP*	0.32	69.6%

Here GRASP* denotes the version with post-training compensation.

This table gives a more nuanced picture:

Training-free GRASP is very competitive with SVD-LLM.
With lightweight compensation, GRASP becomes clearly stronger in these reported averages.
At aggressive 50% compression, performance drops substantially even with compensation, which is an important boundary condition.

11. Efficiency and compression time

The paper measures generation throughput for LLaMA2-7B and its GRASP-compressed version under a 25% compression ratio on a single A100 GPU. The figure reports that GRASP improves throughput and is comparable to direct layer removal.

This is plausible because the retained singular components are very low-rank and only exist in selected redundant layers. The paper emphasizes that the extra retained parameters add negligible inference overhead relative to keeping the full layers.

Compression-time table for LLaMA2-7B at 25% compression on one A100:

Method	Pruning time (h)	Compensation time (h)
LaCo	0.05	1.20
SliceGPT	0.60	0.76
LLM-Streamline	0.03	0.70
GRASP	0.16	none
GRASP*	0.16	0.66

The fair interpretation is:

GRASP is not the fastest pruning step by wall-clock time.
It is still practical at this scale.
In its training-free form, it avoids compensation time entirely.
With compensation, its total cost is comparable to other post-training methods while achieving strong accuracy.

12. Ablation studies

12.1 Calibration data choice

GRASP uses a calibration set for hidden-state similarity and gradient attribution. The paper tests whether the method is sensitive to this choice.

For LLaMA3.1-8B-Instruct under 20% compression:

Calibration data	WikiText-2 PPL ↓	PTB PPL ↓	Average accuracy ↑
WikiText-2, 512 samples	37.86	63.97	47.12
C4, 512 samples	40.54	71.42	46.17

The average downstream accuracy changes by less than one point in this table, which supports the robustness claim.

12.2 Calibration data amount

The paper also varies the number of WikiText-2 samples:

Calibration samples	Average accuracy ↑
64	47.06
128	46.93
256	46.67
512	47.12

This is one of the more deployment-friendly results. It suggests that GRASP does not need a huge calibration set to rank singular groups reasonably well.

12.3 Retain ratio

For LLaMA2-7B, the paper replaces 8 redundant layers and varies the retain ratio:

Retain ratio	Overall compression ratio	Average accuracy
0%	24.0%	41.1
5%	22.8%	44.8
10%	21.6%	45.5
15%	20.4%	45.5
20%	19.2%	45.8

The key point: 10% is a practical inflection point. Keeping no singular parameters is essentially layer removal; keeping 10% recovers much of the damage; beyond that, returns are smaller.

12.4 One-shot versus iterative pruning

For LLaMA3.1-8B-Instruct under 20% compression:

Strategy	WikiText-2 PPL ↓	PTB PPL ↓	Average accuracy ↑	Compression time (h)
One-shot	37.86	63.97	47.12	0.16
Iterative	38.39	72.18	47.13	0.22

Accuracy is almost identical, while one-shot is faster. This supports using the simpler one-shot procedure in practice.

13. LongBench and unstructured pruning comparisons

The appendix includes two useful extra comparisons.

13.1 LongBench

For LLaMA3.1-8B-Instruct under 20% compression, the LongBench average is:

Model	LongBench average
Dense LLaMA3.1-8B-Instruct	26.09
GRASP	26.26

Individual LongBench tasks vary: GRASP improves on some subtasks and drops on others. The average being similar is encouraging, but it should not be over-read as universal long-context superiority. It shows that the compression did not obviously collapse long-form evaluation in the reported setting.

13.2 SparseGPT and Wanda

The paper compares 2:4 unstructured sparsity methods with GRASP:

Method	Compression/sparsity setting	Average
Dense	none	53.38
SparseGPT	2:4 sparsity	46.28
Wanda	2:4 sparsity	45.16
GRASP	20% structured compression	45.27
GRASP*	20% structured compression + compensation	50.01

This comparison is useful but not perfectly apples-to-apples. Unstructured sparsity and structured low-rank layer replacement stress the hardware and runtime system differently. The practical winner depends on whether the deployment stack has efficient sparse kernels.

14. Why GRASP works, intuitively

The paper’s explanation has two parts.

14.1 Redundant layers are often low-rank in function

If a layer barely changes its hidden state, its effective transformation may live in a smaller subspace. A full dense layer is overkill for such a transformation.

The method therefore tries to preserve the subspace directions that matter most.

Full redundant layer:
[ many directions: useful + nearly useless + noisy/redundant ]

GRASP replacement:
[ a few gradient-selected useful directions ]

14.2 Gradient attribution aligns compression with task loss

Magnitude-based SVD answers:

Which components best reconstruct the weight matrix?

GRASP answers:

Which components most affect the language-model loss on calibration data?

Those are related but not identical questions. For model compression, the second question is usually closer to what we care about.

15. Limitations and boundary conditions

The paper lists several limitations, and there are a few practical caveats worth making explicit.

15.1 Cosine similarity is a heuristic

GRASP assumes that if $H_i$ and $H_{i+1}$ are close, the layer is redundant. This is useful but incomplete. A layer could make a small directional change that matters for rare tokens, long-context behavior, multilingual tasks, or safety behavior.

The cosine score is a good candidate-selection heuristic, not a proof that the layer is dispensable.

15.2 GRASP needs internal access

The method needs:

hidden states for layer similarity,
gradients for singular-group attribution,
access to model weights for SVD and replacement.

So it is not suitable for strict black-box API-only models.

15.3 Calibration data still matters

The ablation suggests GRASP is not very sensitive to WikiText-2 versus C4 or to sample counts between 64 and 512. But a calibration set is still a proxy for deployment traffic. If your target domain is legal, medical, code, multilingual chat, or tool-use traces, generic WikiText-2 may miss important directions.

A practical deployment should calibrate on data close to the target workload when possible.

15.4 The paper’s largest models are modest by 2026 standards

The reported main model sizes go up to 13B, with LLaMA3.1-8B-Instruct and Mistral-7B included. That is useful, but it does not automatically prove the same behavior on 70B-scale, MoE, very long-context, or heavily instruction-tuned production models.

15.5 Mostly English-centric evaluation

The paper states that experiments are primarily English-language tasks. Multilingual robustness remains future work.

15.6 Compression ratio should be read carefully

Because GRASP selects layers and then chooses a retain ratio within selected matrices, “20% compression” is not simply one universal architecture-independent setting. It depends on:

how many layers are selected,
which matrices in those layers are replaced,
the singular-group retain ratio,
implementation details for storing and executing low-rank modules.

16. Reproducibility notes

If I were reproducing this paper, I would track the following details carefully.

16.1 Use the exact calibration protocol

The main comparison uses 512 WikiText-2 samples. The ablation also tests C4 and smaller sample counts. Because layer selection and gradient attribution both depend on calibration data, the sample choice should be logged.

16.2 Separate three procedures

Do not mix these up:

Layer selection: forward pass, cosine similarity of hidden states.
Singular-group attribution: SVD plus backward pass for gradient scores.
Post-training compensation: optional training after compression.

GRASP can be discussed as training-free only when step 3 is skipped. Computing gradients for attribution is not post-training optimization.

16.3 Report both average and per-task metrics

Averages can hide large task variation. For example, a method might preserve PIQA but damage MathQA or GSM8K. The paper reports per-task numbers, and a faithful reproduction should too.

16.4 Record real runtime behavior

The method is designed for structured speedups, but actual throughput depends on the implementation of low-rank replacement modules. A reproduction should report:

model size on disk,
GPU memory usage,
prefill throughput,
decode throughput,
latency at realistic batch sizes,
whether low-rank modules are fused or executed as separate kernels.

16.5 Compare with quantization combinations

The paper focuses on pruning/low-rank compression. In real deployment, GRASP would likely be combined with quantization. That combined setting is important but not the main focus of the reported experiments.

17. Practical takeaway for engineers

GRASP is most attractive when:

you can access model weights and gradients,
you want structured compression rather than sparse weights,
you can afford a small calibration pass,
you want a training-free compressed model or only lightweight recovery training,
target latency benefits from removing or replacing whole layers.

It is less attractive when:

the model is only available through an API,
the deployment stack cannot efficiently run the low-rank replacements,
target behavior is far from the calibration distribution,
you need strong guarantees on multilingual, safety, or rare-domain behavior.

18. My assessment

GRASP is a sensible and technically clean idea: it improves layer pruning by refusing to treat “redundant” as “worthless.” The strongest part of the paper is the combination of two signals:

hidden-state similarity for finding where compression is plausible, and
gradient attribution for deciding what low-rank structure to keep.

That combination is more convincing than either direct layer deletion or magnitude-only SVD.

The main caveat is that the method still depends on heuristics and calibration data. It is a practical compression recipe, not a theoretical guarantee that the selected layers are universally redundant. The experiments are broad enough to be interesting, especially across LLaMA and Mistral models, but I would still want deployment-domain calibration and real serving benchmarks before trusting it in production.

19. Final takeaways

GRASP means Gradient-based Retention of Adaptive Singular Parameters.
It compresses LLMs by replacing redundant layers, not merely deleting them.
Redundant layers are selected using hidden-state cosine similarity.
Each selected weight matrix is decomposed by SVD.
Singular groups are retained by gradient-based loss sensitivity, not by magnitude alone.
A 10% singular-group retain ratio is the paper’s practical default because returns saturate after that point.
Training-free GRASP is competitive; GRASP with light compensation is stronger.
The best reported numbers show strong retention of dense performance, but the exact percentage depends on the table, model, task set, and whether compensation is used.
Limitations include heuristic layer selection, need for internal gradients, calibration dependence, model-scale boundaries, and mostly English evaluation.