Computation and Performance Studies · Vol. 1 No. 1 (2026)
Original publication: October 28, 2026
Cameron Peterson — France; Andrew King — United States; Evan Anderson — Australia
An actual payload and reconstruction benchmark for twenty-four public language-model matrices on one CPU session.
Four-bit packing reduced selected weight payloads from 2,359,296 to 626,688 bytes, including one float32 scale per 128 weights. In the same local session, reconstructing packed weights to float32 took modestly longer than converting source float16 payloads: the geometric mean ratio of matrix-median times was 1.064368. Both paths were measured after warmup with balanced execution order. Exact checks confirmed that unpacking reproduced the declared rounded values, rather than the original source weights. The result concerns a transparent memory-resident NumPy prototype. It measures neither a whole-model compression ratio nor inference acceleration through optimized low-bit kernels, and its timing cannot be assumed invariant across machines.
A small numeric weight payload can reduce stored input bytes while requiring extra work before a computation consumes the values. If the next stage expects float32, a packed representation must be unpacked and scaled, while a float16 source must be converted. Those operations depend on implementation and hardware. A nominal bit width alone cannot establish which path is faster or how the tradeoff affects an application's full inference pipeline.
We ask how a transparent four-bit packing prototype compares with source-float16 conversion in payload size and memory-resident reconstruction time. The storage quantity is directly countable. The time quantity is measured for a specified implementation and session. We also verify whether the decoder returns the exact values defined by the rounding rule. These are complementary checks: a small payload, a fast call, and a correct reconstruction of its intended representation are different claims.
Compression work represented by Wang et al., 2026 motivates preserving those distinctions when discussing efficiency. We do not implement that framework or evaluate its speed. The present code packs selected public matrices and reconstructs them with ordinary NumPy array operations. It supplies a small inspectable resource experiment, without claiming a competitive inference system or treating a prototype decoder as representative of every optimized quantization implementation.
The weights come from the final standard-Pile checkpoint associated with Biderman et al., 2023. We directly parse its numeric safetensors header and float16 byte ranges. The selected set contains four attention or multilayer-perceptron projection matrices in each of six layers, giving twenty-four matrices and 1,179,648 weights. No image, text, or activation data are needed to form the payloads. Embeddings, biases, normalization parameters, and other checkpoint components remain outside the selected storage comparison.
Within each matrix, row-major groups contain 128 values. We promote source values to float32, divide each group's largest absolute value by seven to form a float32 scale, and round divided values to codes between minus seven and seven. The scales are positive. Reconstructed values are the integer codes multiplied by those float32 scales. The operation is lossy relative to the released float16 parameters, even though the subsequent decoding check can be exact relative to this declared rounded reference.
The broader numerical methods in Frantar et al., 2022, Lin et al., 2023, Xiao et al., 2022, Dettmers et al., 2022, and Dettmers et al., 2023 use additional calibration, representation, or implementation choices. None is reproduced by the simple maximum-scale procedure. Their reported accuracy or throughput cannot be transferred to this prototype. We retain the narrow numerical convention so that byte accounting and decoder correctness refer to a fully specified object, rather than a generic label such as four-bit compression.
Numerical issues discussed in Higham, 2002 and Goldberg, 1991 motivate preserving the dtype, rounding order, and scale storage precision. A decoder can faithfully reconstruct a quantized reference while that reference differs materially from the original model. We do not equate exact decoding with lossless compression of source weights, and no downstream language score is inferred from an array-equality test. Prediction effects require their own model execution.
The signed code is offset by seven into the range zero through fourteen. Two such codes occupy the low and high nibbles of each stored byte. Group scales follow the code bytes as little-endian float32 values. Every selected matrix has an even number of values divisible by 128, so the implementation needs no unused terminal half-byte. One nominal code remains unused within the four-bit representation.
The packed payload therefore contains four code bits per weight and thirty-two scale bits per 128 weights, giving 4.25 payload bits per weight. The source payload contains sixteen bits per weight. Across the selected matrices, sizes are 626,688 and 2,359,296 bytes respectively. The packed payload is 26.5625 percent of the float16 payload. This ratio follows directly from the selected code and scale format; the experiment does not present that arithmetic as an unexpected empirical discovery.
The saved byte arrays are not self-describing model files. Matrix identity and shape remain in the external source metadata and analysis record. Headers, file alignment, container structures, checksums, and the unchanged checkpoint tensors are outside the payload count. A whole-model file could therefore have a substantially different reduction. We do not multiply the selected payload ratio by the complete checkpoint size and describe that as an observed archive result.
The raw payloads are actually written to local files and assigned hashes. The packed decoder separates nibbles, converts their offset codes to float32, reshapes into groups, and multiplies by the stored scales. For every matrix, decoded values are exactly equal to the independently constructed rounded reference array. The float16 path is separately checked to reconstruct source values promoted to float32. These correctness checks use different intended references because the two representations have different fidelity.
The implementation uses NumPy 2.5.4, in the array-programming context described by Harris et al., 2020, with Python 3.12.14 on an AMD64 processor reported as Family 25 Model 97. The performance counter has a reported resolution of 0.0000001 seconds. Both paths start from bytes already in memory and return float32 values. The measured call includes array conversion and output allocation. It excludes disk reads, source-header parsing, model execution, and any compilation stage.
Each matrix and method receives ten untimed warmup calls, followed by sixty timed observations. Within a repetition the method order alternates, yielding thirty measurements in each ordering. We summarize each method by its matrix-specific median duration. Timing a memory-resident payload repeatedly can favor cache reuse; this is part of the local design rather than an approximation to cold loading from storage. Background workstation load is not independently controlled.
Figure 1. Source and packed bytes enter different float32 reconstruction paths; exact reference checks precede balanced warm timing and matrix-level summaries.
The hardware-cost perspective in Williams et al., 2009 emphasizes that an application's limiting resource may differ from the one measured here. Attention and serving methods in Dao et al., 2022 and Kwon et al., 2023 optimize other pipeline components. The decoding setting in Leviathan et al., 2022 is also distinct from unpacking stored weights. No such implementation is benchmarked by these array calls. Their relevance is to why a local reconstruction duration cannot alone supply a full language-model speedup estimate.
For every matrix we divide packed-decoder median duration by float16-conversion median duration and take the natural logarithm. An intercept-only ordinary least-squares model estimates the equal-matrix mean log ratio. There are twenty-four rows. Exponentiating the mean gives a geometric mean ratio, which differs from dividing arithmetic averages of the two sets of durations. The statistic weights matrix locations equally rather than weighting them by parameter count or payload bytes.
Covariance clusters by the six transformer layers, with a finite-sample correction and a t reference using five degrees of freedom. The grouping perspective in Cameron and Miller, 2015 motivates accounting for related matrices within a layer. Heteroskedasticity-consistent estimation in White, 1980 alone would not describe that grouping. These intervals summarize a conditional across-matrix comparison; six layers in one model and one workstation session are not six independent machines or independent model-training replications.
Table 1. Mean log packed-Q4 versus source-F16 reconstruction-time ratio, using per-matrix medians.
| Model | Term | Estimate (SE) | 95% CI | N |
|---|---|---|---|---|
| M1 | Mean outcome/contrast | 0.0624 (0.0039) | [0.0523, 0.0725] | 24 |
| M1: fit | R-squared = 0.0000 | - | - | 24 |
Model labels: M1 = Mean log packed-Q4 versus source-F16 decode time.
The mean-only fit has zero R-squared, without an implication that timing is unexplained in a broader performance model. The sixty within-matrix repetitions are not promoted to independent matrix observations. They support the median summaries used here. A separate repeated-session experiment would be required to quantify session-to-session timing variability or test whether the same ratio holds under changing system load.
The mean log ratio is 0.062380909, with layer-clustered standard error 0.003924130 and interval 0.052293611 to 0.072468207. Exponentiating the coefficient gives a ratio of 1.064367694: approximately 6.4 percent greater duration for the packed reconstruction under this equal-matrix summary. The interval is conditional on the measured session and the matrix-group interpretation. It should not be promoted into a universal penalty attached to four-bit weights.
Arithmetic averages of the matrix medians are 73,929.167 nanoseconds for float16 conversion and 76,922.917 nanoseconds for packed reconstruction, approximately 73.93 and 76.92 microseconds. Their ratio differs from the reported geometric mean because matrices have different sizes and timing ratios. We retain the definitions rather than switching summaries to enlarge or diminish the observed overhead. Both are descriptive quantities from the actual recorded calls.
The payload saving and local timing result are compatible. The packed path reads fewer stored code bytes but performs nibble extraction, offset conversion, and scaling to produce the same output dtype. It also allocates temporary arrays. The float16 path performs a different conversion operation and preserves different source fidelity. We have not isolated which instruction, cache effect, allocation, or dtype conversion accounts for the measured difference, so none is presented as a confirmed mechanism.
The published tool context in ggml-org contributors, 2026 illustrates a different kind of evidence involving actual quantized-model throughput. Those external numbers are not merged with this timing table. Runtime systems represented by Paszke et al., 2019 and Wolf et al., 2019 are also outside the measured decoder call. Merely loading a model through a separate library would not make these NumPy payload timings equivalent to an end-to-end comparison through that runtime.
Exact decoded equality verifies the implementation against the intended code-and-scale reference. It does not establish exact restoration of the source weights. The parameter-space reconstruction errors are retained in the data, but this report does not measure generated text, next-token loss, or task correctness. Measurement reasoning in Carroll et al., 2006 cautions against substituting one endpoint for another simply because both are numerical.
Scoring principles in Gneiting and Raftery, 2007 reinforce that performance claims require an outcome matched to the question. A decode call, a stored payload, and a language-model answer can each be evaluated correctly while leading to different practical conclusions. The benchmark therefore avoids a composite efficiency score that would combine bytes and nanoseconds with an unstated value judgment. A real deployment would need actual loading, execution, memory, and quality measurements under its intended workload.
Systems concerns in Sculley et al., 2015 are relevant to external metadata and integration assumptions. The prototype relies on known matrix shapes and a fixed grouping convention, so its raw files cannot be consumed as arbitrary standalone model files. An operational codec would need versioning, validation, and an execution path for its reconstructed values. These are additional implementation requirements, not already measured benefits inferred from the saved byte arrays.
Influence reasoning in Cook, 1977 motivates inspecting unusually slow matrix measurements, but no matrix is removed. Medians reduce sensitivity to individual timed-call outliers without establishing a controlled hardware experiment. Validation concerns in Roberts et al., 2017 and selection bias in Varma and Simon, 2006 also matter if the same timing session is used to tune implementation choices and then presented as independent final evidence. We retain the declared implementation and balanced schedule rather than searching multiple versions for a preferred ratio.
Interpretation under Wasserstein and Lazar, 2016 emphasizes the modest conditional effect and its measurement scope. Multiplicity considerations in Benjamini and Hochberg, 1995 and Benjamini and Yekutieli, 2001 would apply to a broader search over code paths, group sizes, precisions, and devices. The current comparison has one group size and two explicitly defined reconstruction paths. Its interval cannot compensate for an unmeasured system bottleneck or guarantee that another implementation will favor the same path.
Timing also requires a different reproduction standard from byte packing. Numeric payloads and their decoded reference values can be reproduced exactly under the declared format. Nanosecond measurements can change with software, hardware, allocation state, and background work. A successful rerun should preserve the protocol and report its new timing results, not be required to reproduce an old timing CSV byte for byte. Keeping these evidence types separate is necessary for honest performance reporting.
The completed prototype packs selected weight payloads to 4.25 bits per value and reconstructs their declared rounded values correctly. In one warmed CPU session, that reconstruction is modestly slower than source-float16 conversion under the reported matrix-level summary. These observations establish a local storage-and-reconstruction tradeoff, without determining the speed, fidelity, or size of a complete compressed inference system.
Documentation under Gebru et al., 2021 records source tensor identities, code offsets, endianness, scales, excluded metadata, timer resolution, warmups, and execution order. Preservation following Wilkinson et al., 2016 retains actual packed payloads, hashes, numerical checks, and timing repetitions. They permit the implementation and its measurements to be audited. A general compression-efficiency recommendation requires independent workload and hardware evidence beyond this transparent NumPy experiment.