The engineering that turns a model into a product.
11 topics · 13 curated works
No prior grounding assumed.
Green AI
Roy Schwartz, Jesse Dodge, Noah A. Smith & Oren Etzioni · 2020
Argues the field should report and optimise for computational cost alongside accuracy, and proposes measuring a paper's efficiency directly rather…
Assumes you know the vocabulary.
Hidden Technical Debt in Machine Learning Systems
Sculley et al. · 2015
The model is a tiny box in a large diagram; almost all the cost and almost all the failures live in the plumbing around it.
+7 more at this level
Primary sources and full treatments.
Large Scale Distributed Deep Networks
Jeffrey Dean et al. · 2012
Describes DistBelief, Google's framework for training networks with billions of parameters across thousands of machines using both model and data…
+3 more at this level
12 of 13 works
Roy Schwartz, Jesse Dodge, Noah A. Smith & Oren Etzioni
Argues the field should report and optimise for computational cost alongside accuracy, and proposes measuring a paper's efficiency directly rather than treating ever-larger models as free progress.
link checked 17 Sept 2026Sculley et al.
The model is a tiny box in a large diagram; almost all the cost and almost all the failures live in the plumbing around it.
link checked 17 Sept 2026Geoffrey Hinton, Oriol Vinyals & Jeff Dean
Trains a small model to match the soft output probabilities of a large trained model rather than its hard labels, showing that the large model's mistakes carry information a small model can learn from directly.
link checked 17 Sept 2026Manasi Vartak et al.
Proposes tracking every trained model, its pipeline and its metrics automatically as models are created, arguing model management is a systems problem that should not depend on data scientists remembering to log anything.
link checked 17 Sept 2026Daniel Crankshaw et al.
Puts a model-agnostic serving layer between applications and the frameworks that trained the models, arguing that caching, batching and adaptive model selection belong in that layer rather than duplicated inside every framework.
link checked 17 Sept 2026Neoklis Polyzotis, Sudip Roy, Steven Euijong Whang & Martin Zinkevich
Surveys the data-side problems — collection, cleaning, validation, drift detection — that dominate the engineering cost of a production ML pipeline once the modelling problem itself is largely solved.
link checked 17 Sept 2026Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang & Joel S. Emer
Surveys the hardware options for running neural networks — from general-purpose processors to custom accelerators — and derives the data-movement and dataflow choices that actually determine energy and latency, not just peak FLOPs.
link checked 17 Sept 2026Orr et al.
Argues that feature stores designed for tabular data are unprepared for the shift to self-supervised embeddings as model inputs, and identifies embedding versioning, quality evaluation and drift monitoring as unsolved infrastructure problems.
9 pageslink checked 17 Sept 2026Kreuzberger, Kühl & Hirschl
Pulls a definition of MLOps out of practitioner interviews and tooling surveys, as a set of principles, components and roles rather than a category of product.
link checked 17 Sept 2026Jeffrey Dean et al.
Describes DistBelief, Google's framework for training networks with billions of parameters across thousands of machines using both model and data parallelism with asynchronous parameter updates.
link checked 17 Sept 2026Benoit Jacob et al.
Works out how to train a network so both its weights and activations can run entirely in 8-bit integer arithmetic at inference, closing most of the accuracy gap that naive post-training quantisation leaves open.
link checked 17 Sept 2026Mohammad Shoeybi et al.
Splits a transformer's own matrix multiplications across GPUs, rather than splitting layers or data, to train language models too large for any single accelerator's memory with only small, targeted communication.
link checked 17 Sept 2026