Computation and Performance Studies · Vol. 1 No. 1 (2026)
Original publication: October 28, 2026
Dominic Knight — Australia; Tyler Robertson — New Zealand; Evan Anderson — Netherlands
Choosing an image-quality metric requires more than ordering its reported correlations. An exact-name join of five quality panels and two timing columns retains eleven complete methods from a pinned IQA-PyTorch snapshot. Five methods are nondominated for CPU latency and average reported SRCC, while only two remain on the corresponding GPU frontier. The quality-log-latency slopes are uncertain on both platforms. These conditional frontiers describe a published table join, with different quality and timing workloads, rather than a new controlled hardware benchmark or a universal recommendation.
Image assessment is often treated as a ranking problem in which the method with the highest correlation wins. A system choosing a metric for repeated use also faces a computational constraint. A slightly stronger reported correlation may or may not justify a much longer call, and the answer can depend on the platform used to execute it. The research question is which reported methods remain nondominated when a common quality summary is considered jointly with CPU latency and then with GPU latency.
The semantic and distortion distinction proposed in Meng, 2026 motivates attention to what a quality metric measures. It does not determine which computational burden a workflow can accept. We do not run that model or compare its measured speed with existing methods. Our evidence is a pinned public benchmark snapshot from the IQA-PyTorch resource, referenced as Chen and Mo, 2022. It contains separate published quality and efficiency tables that can be joined transparently without pretending that a new experiment has been performed.
The resulting frontier is conditional on several explicit choices. Quality is the arithmetic mean of reported absolute SRCC across five panels. Latency is one of two source columns. Missing records are excluded, metric identifiers must match exactly, and dominance requires no worse values on both objectives with a strict improvement on at least one. A frontier can change when any of those definitions changes. Its purpose is to expose the tradeoff available in the captured records, rather than to hide those choices behind the claim that one metric is best overall.
The computational perspective in Williams et al., 2009 makes platform and workload central to performance interpretation. The benchmarking concerns in Mytkowicz et al., 2009 and the measurement approach in Kalibera and Jones, 2013 further caution against treating a single reported time as a universal constant. We therefore preserve the published timing values as source summaries. We do not attach invented repeated-run standard errors, infer energy use, or equate a reported wall-clock call time with training cost. The source table does not provide the information needed to estimate those quantities.
The reported efficiency helper times path-based metric calls, with source documentation describing average evaluation at 1080 by 800 resolution. Model construction is outside the timed region, and the first image timing is omitted. A current helper also invokes CUDA synchronization, including in its CPU branch, leaving limits on reconstructing all historic table cells from that helper alone. These details matter because our join is a comparison of reported columns, not a claim that a newly executed script reproduced the timing conditions or controlled every platform difference.
The quality side uses five panels: LIVE Challenge, KonIQ-10k, TID2013, FLIVE, and SPAQ. The selected resources include smartphone photographs (Fang et al., 2020), diverse natural images (Hosu et al., 2019), and controlled artificial distortions (Ponomarenko et al., 2015). Other assessment settings include aesthetic judgments (Murray et al., 2012) and perceptual restoration assessment (Gu et al., 2020); those two resources are contextual references and are not panels in this calculation. The five-panel arithmetic mean assigns equal panel weight and does not define a common latent perceptual quantity. It is neither image-count weighted nor a claim about the utility of every application using a metric.
Metric design also varies. Mittal et al., 2012 and Mittal et al., 2013 use visual-statistics approaches, while Talebi and Milanfar, 2017 learns assessment outputs. Yang et al., 2022 uses multidimensional attention and Ying et al., 2019 addresses picture-quality assessment. Their inclusion in a common table enables a resource comparison but does not erase differences in training data, intended targets, or implementation. Those differences can affect both the benchmark correlations and the observed call time. We do not attribute a frontier position to a single architectural feature or infer a causal benefit of attention, scale, or training size.
We read the pinned no-reference quality table and retain methods with nonmissing entries in its first five panels: LIVE Challenge, KonIQ-10k, TID2013, FLIVE, and SPAQ. Each cell stores PLCC, SRCC, and KRCC in a slash-separated format. We extract the SRCC component as reported. The source uses correlation magnitudes; the mean therefore summarizes reported absolute rank association rather than signed association on a common raw score direction. The sparsely reported KonIQ-1024 column is excluded as a separate resolution condition.
Twelve method rows have all five selected quality values. We join them by exact metric identifier to the efficiency table and require both CPU Xe6150 and CUDA V100 timings. Eleven remain after the join. No spelling alias is guessed and no missing latency is imputed. The final cohort contains brisque, nrqm, niqe, ilniqe, nima, pi, cnniqa, paq2piq, dbcnn, topiq_nr, and maniqa. This explicit list defines the comparison population and prevents a partially reported method from being treated as dominated merely because its timing is absent.
The mean quality score assigns equal weight to the five panels. It is calculated once and used for both platform frontiers. Latencies must be positive so that their logarithms can be calculated for the descriptive regressions. Family labels group related metric names for the covariance calculation; the eleven records form ten such groups. Exact-ID joining and family grouping serve different purposes. The former determines which numeric records are compared, while the latter expresses a working dependence structure among those retained methods.
For a given platform, method A dominates method B when A has latency no greater than B and mean quality no lower than B, with at least one strict inequality. Every method not dominated by any retained method is on the platform-specific frontier. This is a finite-table relation and does not require a regression model. Equal values are treated as ties at the published precision. No tolerance is added to imply knowledge of an unreported numerical difference, and no missing method is silently assigned a poor score.
Figure 1 shows the two sources entering an exact-ID join before platform-specific comparisons. The quality summary is held constant across platforms, so changes in frontier membership arise from the reported latency ordering within the retained cohort. This does not make the platform a randomized intervention. The CPU and GPU columns can differ in hardware, execution behavior, and measurement context. They supply alternative operational records for the same method identifiers, not an isolated causal estimate of moving an arbitrary workload to a GPU.
Figure 1. Published quality summaries and timing records are joined by exact method identifier. CPU and GPU dominance are assessed separately, with a common quality aggregation and a complete-case cohort.
Two OLS models relate mean reported SRCC to log latency, one using the CPU column and the other the GPU column. These associations describe whether slower records tend to have higher quality summaries in the retained table. They do not determine the frontier, estimate an efficiency production function, or establish an intrinsic speed-quality law. Carroll et al., 2006 makes measurement uncertainty relevant to such interpretation, but the source lacks image-level score uncertainty and repeated timing records needed to propagate it into a more comprehensive model.
Covariance is clustered by ten declared method families, with finite-sample correction and a t reference of nine degrees of freedom. Cameron and Miller, 2015 explains why related implementations should not automatically be treated as independent observations. White, 1980 supplies the broader heteroskedasticity-aware foundation. Family grouping still omits shared benchmark panels, training ancestry, and source measurement conditions. The intervals are working descriptions of a small captured cohort, conditional on the rounded published numbers, rather than confidence statements about every possible image-quality algorithm.
Table 1. Mean reported absolute SRCC regressed on log published latency separately for CPU and GPU. N is eleven methods. Both models include an intercept, use ten family clusters with finite-sample correction, and use a t reference with nine degrees of freedom. There are no additional controls.
| Model | Term | Estimate (SE) | 95% CI | N |
|---|---|---|---|---|
| M1 | Log reported CPU seconds | 0.015323 (0.021419) | [-0.033130, 0.063776] | 11 |
| M1: fit | R-squared = 0.0510 | - | - | 11 |
| M2 | Log reported GPU seconds | -0.001230 (0.034870) | [-0.080110, 0.077651] | 11 |
| M2: fit | R-squared = 0.0001 | - | - | 11 |
Model labels: M1 = Mean reported SRCC versus log CPU latency; M2 = Mean reported SRCC versus log GPU latency.
The CPU frontier contains brisque, nima, cnniqa, paq2piq, and topiq_nr. The GPU frontier contains paq2piq and topiq_nr. Thus three CPU-nondominated choices become dominated under the alternative timing column, while the two GPU frontier entries are also CPU frontier entries. Six of the eleven methods are dominated under both platform records. These statements are exact for the selected objectives and published precision. They do not mean that the six methods have no useful property absent from the quality and latency summaries.
The CPU log-latency coefficient is positive 0.015323, with standard error 0.021419 and a 95% interval from negative 0.033130 to positive 0.063776. The model R-squared is 0.0510. The GPU coefficient is negative 0.001230, with standard error 0.034870, an interval from negative 0.080110 to positive 0.077651, and R-squared approximately 0.0001. Neither association shows a clear average slope under the stated inference. A near-zero fitted linear slope also does not make every method an equally good resource choice.
That last distinction is central. Dominance is pairwise and permits a nonlinear, irregular frontier, whereas the regression summarizes one global linear association with log time. A table can contain clearly dominated entries even when its overall slope is uncertain. Conversely, a positive slope would not identify which specific method dominates another. Reporting the frontier alongside the model keeps the decision relation and the descriptive trend separate, without using the inferential table as a substitute for checking the actual retained records.
Rank-correlation traditions in Spearman, 1904 and Kendall, 1938 explain the association quantities reported in the source panels, but their presence does not justify averaging them as if all panels measured one latent construct. Demšar, 2006 adds a relevant caution about comparisons over related benchmark tasks. The five panels are not five independent replications of a controlled hardware tradeoff. They provide one chosen quality summary, and a workflow emphasizing a single dataset could legitimately obtain a different frontier from the same source methods.
The frontier deliberately omits deployment utility. Gneiting and Raftery, 2007 motivates stating what a predictive evaluation rewards, and Willmott and Matsuura, 2005 clarifies that a numerical error summary depends on its objective. Here mean absolute SRCC rewards rank association averaged across the selected panels. It does not quantify the cost of a mistaken image decision, calibration of a quality threshold, or performance on a rare distortion. A method could be dominated in the present two-dimensional summary while remaining attractive under one of those unmeasured criteria.
Other visual evaluation targets illustrate that limitation. Wu et al., 2023 concerns preference-related generated-image assessment, while Zhang et al., 2018 supplies a perceptual distance between images. Neither is reducible to the exact five-panel no-reference average used here. We therefore do not use frontier status as a recommendation for human preference, semantic alignment, or image restoration fidelity. Likewise, a smaller reported call time does not establish lower peak system memory, energy, or end-to-end application latency once loading, batching, and data movement are included.
Numerical and diagnostic caution also matters. The measurement discussion in Stevens, 1946 calls for identifying the meaning of every axis, and Cook, 1977 reminds us that a small regression can depend strongly on particular records. We did not delete influential metrics or tune the aggregation to strengthen a trend. The reporting perspective in Wasserstein and Lazar, 2016 favors retaining the uncertain slopes and conditional membership lists instead of turning either into a binary significance story. Rounding could affect a near-tie, and missing methods could change the frontier if complete records became available.
The snapshot, join cohort, parsing decisions, quality means, platform values, and family labels are retained according to documentation aims in Gebru et al., 2021 and reuse principles in Wilkinson et al., 2016. Those records allow a reader to change a panel weighting or a completeness rule explicitly. They do not establish that every historical timing cell can be regenerated from the current helper. A controlled follow-up would need common workloads, repeated timings, image-level quality outputs, and an application-specific objective before it could support a stronger method recommendation.
Completeness is part of the result, rather than a neutral preprocessing convenience. Requiring all five quality panels and both timings removes records that may be competitive under a smaller evidence requirement. We cannot infer their frontier positions without those missing entries. Conversely, using each method's available panels would compare different quality summaries and could reward selective reporting. The complete-case rule makes the current comparison coherent, but its cost is a narrower method cohort and an explicit inability to rank incompletely reported alternatives.
The frontier also supplies no preferred point among its surviving entries. Choosing between a faster lower-quality record and a slower higher-quality one requires a latency budget, a quality floor, or an application utility. None was elicited for this study. We consequently retain the full nondominated list for each platform and avoid a recommendation disguised as a geometric property. An institution can apply its own constraint to these captured summaries, provided it first checks whether the source workloads and dataset mixture resemble the intended use.
Joining the published tables yields five CPU-nondominated methods and two GPU-nondominated methods under the same quality summary. The average quality-log-latency associations remain uncertain, emphasizing that a frontier and a global regression answer different questions. The useful result is a transparent, platform-conditional comparison of complete reported records. Its practical interpretation depends on the selected panels, the timing workload, missing entries, and the decision objective that a real application needs to serve.