Resource Allocation and Optimization · Vol. 1 No. 1 (2026)
Original publication: October 31, 2026
Brandon Franklin — Belgium; Stephen Young — Austria; Ryan King — Italy
Train-only allocation experiment on public block-count features. Observation count is the resource unit; acquisition time, physical scanning, and a specialized inpainting model are not evaluated.
Large variation in an observed feature does not necessarily make it useful for predicting a missing region. We compared two training-defined observation menus for four concealed Optdigits features. With sixteen visible features, target-directed greedy selection gave test MSE 0.07122, versus 0.08336 for marginal-variance selection. The advantage persisted with thirty-two observations. Both menus were fixed before the 1,797 test images were evaluated. The result supports task-specific allocation within this numerical setting, while leaving physical acquisition cost and the global optimality of the selected menu unresolved.
A limited observation budget forces a distinction between information that varies and information that helps. A high-variance feature can reveal much about the variability of the observed region while contributing little to a particular missing target. Conversely, a less variable feature may have a strong predictive relationship with the target. Selecting observations solely because they are individually variable can therefore spend the budget on the wrong part of the reconstruction problem.
We ask whether a training-defined, target-directed observation menu improves hidden-feature completion over a menu based on marginal variance at the same feature count. The empirical setting uses sixty candidate visible block counts to predict four central block counts in handwritten digits. Two budgets, sixteen and thirty-two observations, allow us to examine whether an allocation advantage persists when the menu expands. The objective is completion of a specific target, rather than general compression of the visible image.
The application motivation comes from restoration of structured symbols. Meng et al., 2025 develop a zero-shot inpainting model for oracle-bone inscription rubbings. We do not implement that model or treat digit blocks as an archaeological dataset. Instead, the present experiment studies an elementary resource question that can arise before sophisticated restoration: which available context should a fixed predictor receive when the observation menu is constrained?
Our proposition is that a menu selected for the hidden-target objective can outperform a menu selected for high individual feature variance. This is an empirical proposition, not a claim that greedy selection is always optimal. Dependencies among visible features, regularization, finite training data, and test-population differences can all alter the result. The analysis evaluates the specified procedures on the same official held partition.
Sensor-selection research explicitly distinguishes observation placement from the estimator that later uses the observations. The Gaussian-process placement work of Krause et al., 2008 studies an information-based objective and algorithmic guarantees in its own setting. The convex-optimization approach of Joshi and Boyd, 2009 treats a related subset-choice problem through a different heuristic. These papers motivate a target-aware allocation question, but our covariance-greedy implementation does not inherit their approximation guarantees or reproduce their objectives.
Regression subset methods provide another comparison. Leaps and bounds Furnival and Wilson, 1974 and optimization-based best subset selection Bertsimas et al., 2016 examine alternatives to unrestricted enumeration of predictor sets. We use a forward-only greedy path rather than claiming the best subset among all possibilities. Ridge regularization Hoerl and Kennard, 1970 stabilizes the selected-feature regression. A superior held result for one greedy path does not prove that its selected menu is the globally best use of the budget.
Low-dimensional representation methods Pearson, 1901 and probabilistic PCA Tipping and Bishop, 1999 can also reduce an observation space. They answer a different operational question: a projected component generally requires access to several original measurements. The current budget counts original block features actually supplied to the completion rule. Replacing them with sixteen arbitrary latent components would not preserve that resource definition and is therefore not an additional policy in this experiment.
Matrix completion with noise Candes and Plan, 2010 and spectral regularization Mazumder et al., 2010 motivate modeling relationships among available and missing entries. Modern imputation systems such as GAIN Yoon et al., 2018, GRAPE You et al., 2020, GRIN Cini et al., 2021, and BRITS Cao et al., 2018 address other missing-data structures and modeling assumptions. None is fitted here. They identify possible extensions, while the present study retains a simple common predictor so that the observation menu remains the main difference between conditions.
Optdigits Alpaydin and Kaynak, 1998 provides 3,823 official training records and 1,797 official test records. Each record contains sixty-four values formed from digit-image block counts. We divide counts by sixteen to work on a zero-to-one scale. The four concealed entries have flattened indices 27, 28, 35, and 36, corresponding to the central two-by-two block of the eight-by-eight feature arrangement. All other sixty features are candidates for observation.
Every allocation rule sees the same training feature matrix, including the four target entries during fitting. This supervision is legitimate for the declared prediction task: the targets are known in training and concealed only at test time. Class labels are not used to select features, fit the regression, or evaluate an allocation objective. The policies are global menus, so every test image receives the same selected positions for a given policy and budget.
We estimate the full training mean and covariance with the covariance divisor equal to the number of training records. This convention matters because the regularization constant is specified on the covariance scale. The ridge value is fixed at 0.01 under that convention. Test data do not contribute to the covariance, standardization, selected indices, regularization choice, or sequence of greedy decisions.
The variance policy orders candidate visible features by their marginal training variance, breaking ties by source feature index. Its first sixteen and thirty-two entries define nested menus. This baseline does not examine covariance with the hidden targets or redundancy between its selected features. It is an intentionally transparent rule for selecting individually variable observations, rather than a claim about an optimal unsupervised observation strategy.
The target-directed policy starts from an empty menu. At each step, it considers adding each remaining candidate and calculates the regularized ridge fitting objective for the four hidden targets. The objective is the total target covariance trace minus the cross-covariance improvement obtained through the selected-feature covariance plus the fixed ridge diagonal. We choose the candidate giving the smallest value, breaking an exact tie by feature index. The first thirty-two choices form one nested path.
This criterion equals a penalized training objective for the corresponding centered linear regression. It is not an unregularized conditional variance, a mutual-information estimate, or a held-out risk. Its trace sums across four target coordinates. In contrast, the reported test MSE averages over those coordinates. The two quantities therefore differ in their normalization and role; the training objective is not printed as though it were a directly comparable test loss.
After a menu is fixed, its predictor uses the same centered ridge solution under either policy. The intercept is the training target mean, and the coefficient matrix is computed from the selected-visible covariance and its cross-covariance with the hidden entries. Predictions are left unclipped. This avoids adding another policy difference after feature selection, although it permits numerical predictions outside the original feature range.
Figure 1 separates the allocation objective from the shared estimator and the held test outcomes. Its arrows show that all menu construction remains inside the training partition. Each policy is evaluated at an equal observation count. The diagram does not imply that these block measurements have equal real acquisition cost or that a physical scanner can obtain only the chosen blocks without seeing the others.
Figure 1. Marginal variance and target-directed greedy selection supply different nested feature menus. A common covariance-ridge predictor evaluates sixteen- and thirty-two-feature budgets on the same hidden test entries.
For each test image and budget, we subtract variance-policy MSE from target-greedy MSE. Two intercept-only OLS models summarize the mean paired differences. HC3 uncertainty White, 1980 and t intervals use 1,796 residual degrees of freedom. The observation unit is an image, not an individual hidden block or a candidate feature. The point estimates condition on one fitted covariance and one selected path per policy.
The source documentation describes distinct training and test writers but provides no per-image writer identifier in these feature files. We cannot implement writer-clustered covariance without inventing a grouping. Cluster-inference guidance Cameron and Miller, 2015 therefore informs a stated limitation: the displayed intervals do not account for residual dependence among images from the same writer. They also do not propagate uncertainty from training-data selection or the greedy search.
Observation count is a meaningful numerical constraint, but it is not a measured financial or hardware cost. A digit block can differ in acquisition difficulty or redundancy from another block. Shannon's sampling perspective Shannon, 1949 motivates considering what an observation represents, without supplying a reconstruction theorem for these nonuniform block features. We do not report wall-clock speed, scanner savings, bandwidth, or energy as outcomes.
At the sixteen-feature budget, the target-greedy test MSE is 0.071221 and the variance-policy MSE is 0.083356. Their paired difference is −0.012135, with 95% interval [−0.014226, −0.010044]. At thirty-two features, the corresponding MSE values are 0.067255 and 0.077235. The mean difference remains negative at −0.009980, with interval [−0.011495, −0.008466].
Table 1. Target-greedy minus marginal-variance hidden-feature MSE.
| Model | Term | Estimate (SE) | 95% CI | N |
|---|---|---|---|---|
| M1 | Mean outcome/contrast | -0.012135 (0.001066) | [-0.014226, -0.010044] | 1797 |
| M1: fit | R-squared = 0.0000 | - | - | 1797 |
| M2 | Mean outcome/contrast | -0.009980 (0.000772) | [-0.011495, -0.008466] | 1797 |
| M2: fit | R-squared = 0.0000 | - | - | 1797 |
Model labels: M1 = Target-greedy minus high-variance allocation at 16 visible features; M2 = Target-greedy minus high-variance allocation at 32 visible features.
Both columns use the same 1,797 official test records, no regression controls, and HC3 t-based uncertainty. The intercept-only R-squared values are effectively zero. The negative coefficients favor target-directed menus under the stated numerical loss. They do not show that every image improves, that additional features always reduce test error, or that the selected subsets are optimal among all possible subsets.
The fitted target-greedy menu at sixteen observations even has lower mean test MSE than the variance menu at thirty-two. This is a descriptive cross-budget comparison from the same experiment, not a complete resource-efficiency frontier. Offline selection cost is omitted, and the policies can have different practical acquisition constraints. The result shows how the chosen allocation criterion can matter as much as the count of observations supplied to this particular predictor.
Unclipped outputs also deserve disclosure. Across the 7,188 hidden-coordinate predictions, the target-greedy menus produce 266 and 290 out-of-range values at the two budgets. The variance menus produce 206 and 240. Thus, the policy with lower mean squared error is not the policy with fewer range violations. We retain the declared loss and report the violations rather than retroactively clipping one method to improve another outcome.
Context-based restoration Pathak et al., 2016, partial convolutions Liu et al., 2018, and gated convolutions Yu et al., 2018b illustrate models that exploit observed context in different ways. Edge-guided completion Nazeri et al., 2019, denoising representations Vincent et al., 2008, and large-mask restoration Suvorov et al., 2021 add other structures. Our small feature experiment cannot compare those architectures, but it identifies an evaluation variable they can leave implicit: the allocation rule determining which context is available.
Measurement-error analysis Carroll et al., 2006 limits interpreting block-count MSE as visual fidelity. The block representation has already discarded stroke-level structure, and the hidden region is a fixed artificial omission. Influence considerations Cook, 1977 also caution that mean error can reflect particular difficult images. We do not label the remaining variation a semantic failure, measure human readability, or infer the recovery of a specific handwritten stroke.
The numerical work uses NumPy Harris et al., 2020 and SciPy Virtanen et al., 2020 with retained source data, selection logs, and per-image errors. FAIR principles Wilkinson et al., 2016 and dataset documentation Gebru et al., 2021 motivate preserving those relationships. The record makes clear that target entries are available for training selection, which would be an information advantage if an allegedly unsupervised comparator were expected to operate without any target training observations.
The greedy path evaluates sixty candidate additions at its first step, fifty-nine at the next, and eventually twenty-nine at the thirty-second. This is a total of 1,424 candidate-menu objectives, excluding the covariance calculation shared by the policies. The variance baseline only orders the marginal variances. We do not time these operations, but their different structures make it clear that an equal test observation count does not imply equal offline selection effort.
That distinction matters when the menu will be used repeatedly. A more expensive training-stage design could be worthwhile if the same menu is deployed over many future images. Conversely, it might be inappropriate when the training distribution changes frequently and the menu must be rebuilt. Neither deployment frequency nor retraining cost is measured here. The experiment therefore establishes a conditional prediction comparison at equal observed-feature counts, rather than a complete economic optimum.
Regularization conventions also need to travel with a reproduction. The constant 0.01 is added to a covariance matrix divided by the training sample count. A library interface that minimizes an unnormalized sum of squared residuals with a penalty called 0.01 would not fit the same model. Reproducing the analysis requires preserving the objective's normalization or scaling the penalty accordingly. Merely copying the displayed number into another ridge-regression implementation can change the predictor and the selected path.
The observation sets are global rather than image-adaptive. The greedy rule does not inspect a test image and decide which block to request next. Every held image receives the same sixteen or thirty-two source positions under a given policy. This limits claims about active sensing but also removes a possible information leak: the menu cannot adapt to hidden test entries or to a conveniently recognized test class. Per-image acquisition would require a sequential protocol describing which values become available before each selection.
Finally, the target choice is substantive. Four central block counts emphasize a particular part of digit structure. A menu selected for that target may be poorly suited to a corner gap or a full-image objective. A future allocation study should define its intended missing-region distribution before fitting the menu and should evaluate transfer to independent damage patterns. The present result cannot be carried to a new target simply by retaining the same list of selected indices.
For a fixed central hidden target, selecting visible features for a supervised ridge objective yields lower held-feature error than selecting them for marginal variance. The advantage remains at two equal-count budgets. The evidence supports distinguishing observation allocation from observation quantity, while limiting the claim to a fixed numerical feature task, fitted menus, and test partition. Physical acquisition cost and broader restoration quality require their own measurements.