The Information Budget: How Many Independent Things Do Thousands of Datasets Actually Say?

The catalog of data that can be bought is now longer than any research group can read. Card transactions, web traffic, satellite imagery, job postings, product prices, shipping manifests, the text of every filing and every news wire: each arrives as a separate product with a separate contract and a separate claim to be predictive. It is natural to count datasets and to treat the count as a measure of informational advantage. It is also wrong. A dataset is a way of observing the economy, and most ways of observing the economy see the same few things. Two panels of consumer spending are not two sources of information about consumption; they are two noisy views of one.

This piece asks how much genuinely independent information survives once thousands of datasets are netted against the common economic exposures they share and against each other. We work in a stylized universe rather than a real one, because the question is about structure, not about any particular product. The answer, in that stylized world, is that the count of independent dimensions is smaller than the catalog by roughly two orders of magnitude, that the price of an independent dimension varies by close to an order of magnitude across dataset families, and that this count, rather than the length of the catalog, is the budget a systematic research process actually operates within.

Breadth Is Counted in Independent Bets

The reason the count matters comes from the arithmetic of active management. The value of a forecasting process depends on the skill of each forecast and on the number of independent forecasts it makes.^[1] The second term is breadth, and breadth is not the number of datasets or the number of signals. It is the number of bets that are not already implied by the other bets. Ten datasets that all track consumer spending, each with a different lag and a different sampling error, add skill to a single consumption forecast. They do not add breadth. Once that forecast is made, the eleventh spending panel is worth only the marginal reduction in its noise.

The structure of a dataset universe can be read from its correlation matrix. If every dataset were independent, the eigenvalues of that matrix would be roughly equal, and the cumulative variance explained by the leading principal components would rise along a diagonal (a straight line on linear axes; on the logarithmic component axis used in Figure 1 it bows upward toward the right). If a handful of common exposures dominated, the first few components would explain most of the variance and the curve would rise steeply, then flatten. Figure 1 draws that curve for a stylized universe of two thousand dataset-level series built from twelve common factors and a long tail, and draws it again after the twelve factors have been regressed out of every series.

Two things stand out. The raw curve reaches roughly seventy percent of the variance by the twelfth component, which is what a universe dominated by the market, by sector, by the consumer cycle, and by a few macroeconomic conditions looks like. Everything a dataset says about those exposures is already said by prices, by earnings, and by the other datasets. The residual curve is far flatter, but it is not the diagonal. It bends above the pure-noise benchmark for its first several dozen components, after which it rises only with the noise floor and the benchmark slowly catches up with it. That bend is the part of the universe that is both independent of the common factors and shared across several datasets in a structured way: the candidate independent dimensions. In this construction there are on the order of fifty of them, out of two thousand series.^[2]

Principal components count dimensions of variance, not dimensions of information. A direction that explains a great deal of residual variance may forecast nothing; a direction that forecasts something useful may be small. The scree is therefore an upper bound on the count of independent things a universe can say, and the true count is found only by testing each candidate direction out of sample against the forecasts the firm already makes. That is slow, and its slowness is the first constraint on the information budget.

Two Layers of Redundancy

The scree collapses every dataset into one picture. The more useful decomposition is by family, because redundancy has two layers and they weigh differently on different kinds of data. The first layer is the common economic exposure: the share of a dataset's variance explained by the factors every dataset sees. The second is the within-family layer: the share of what remains that is shared with other datasets of the same kind, because three card panels sample the same consumers, three geolocation providers count the same visits, and every weather feed reports the same weather.

Figure 2 stacks the two layers for ten stylized families. The residual slice, the part of a dataset's variance that is specific to that dataset after both layers are removed, is what a firm actually pays for when it buys a second or third source in a family it already holds. The parameters are chosen to illustrate the mechanism, not estimated from any real data, but the ordering follows from the nature of what each family measures.

The pattern repays study. Sentiment extracted from news and social text loads heavily on the common factors, because most of what moves sentiment is what moves the market, and the several providers who extract it apply similar methods to the same text; the dataset-specific residual is small. Weather sits at the opposite extreme on the first layer, since weather does not care about the business cycle, and at the same extreme on the second, since every provider is measuring the same atmosphere; a second weather dataset is almost entirely redundant with the first. Satellite imagery and filings text carry the largest dataset-specific residuals in this construction, because what is extracted from an image or a document depends more on the extraction than on the underlying object, and different extractions genuinely disagree.

None of this says which families are valuable. A small residual can still be a valuable residual if it forecasts well, and a large one can be pure measurement noise. It says where the marginal dataset is likely to add a new dimension and where it is likely to add a noisier copy of one the firm already holds. In a research process that must choose what to evaluate, that is most of what is needed.

The Price of an Independent Bit

Data is priced per dataset. Information arrives per independent dimension. The gap between those two units is where the economics of alternative data lives. Figure 3 works the arithmetic for the same ten stylized families: how many datasets are screened, how many survive screening for coverage, history, and point-in-time integrity, how many independent dimensions the survivors add once they are netted against the common factors and against each other, and what that implies for the cost of a dimension relative to the cost of a dataset. We use the word bit loosely here, to mean one independent dimension of predictive information rather than a unit of entropy.^[3]

The cost per independent dimension is not the cost per dataset. Families whose datasets are highly redundant, card panels and sentiment feeds in this construction, produce the most expensive dimensions, because the firm pays for several sources to obtain one or two directions that are new. Families whose datasets are cheap and structurally diverse, product catalogs and filings text, produce the cheapest dimensions after weather, whose datasets are close to free, even though no single dataset in either family looks impressive. The spread across families in the table is nearly an order of magnitude, and it does not follow the spread in headline prices: the most expensive dataset in the table, satellite imagery, buys a mid-priced dimension, while a mid-priced sentiment feed buys the second most expensive one.

There is a reason the expensive data tends to be the redundant data. A dataset commands a high price because many buyers want it, and many buyers want it because it measures something obviously important. Something obviously important is, almost by definition, something the market already tracks through other channels. The residual is small because the exposure is large. The economics of a data catalog therefore have the same shape as the economics of a crowded trade: the sources that are easiest to justify buying are the ones whose content is most likely to be already in the price.

The Budget

Put the pieces together and the size of the information budget becomes visible. Figure 4 totals the stylized universe: the datasets screened, the datasets retained, the independent dimensions they add, and the ratio between the first and the last.

One dimension per hundred datasets screened is not a pessimistic number; in a stylized world with two layers of redundancy it is the natural one. What it implies is that the size of the catalog is not the measure of a data program. The measure is the count of independent dimensions the program has validated, the rate at which it can add to that count, and the cost of each addition. All three are bounded, and the bounds tighten as the inventory grows, because every candidate must be evaluated against everything already held. The hundredth dataset has to be tested for novelty against ninety-nine; the thousandth against nine hundred and ninety-nine. The cost of admission rises with the size of the room.

The same arithmetic constrains the models built on top of the data. A learning algorithm given thousands of features does not receive thousands of dimensions; it receives a few dozen, wrapped in noise, with the noise dressed up as structure. The number of parameters it can support without fitting that noise is set by the independent information in the inputs, not by their count, and the multiple-testing problem that afflicts signal research afflicts feature selection in exactly the same way.^[4] This is the practical reason we regularize by the information budget rather than by the feature count: a model is allowed as much complexity as the independent dimensions can pay for, and no more. Adding features that add no dimensions does not make the model smarter. It makes the false-discovery arithmetic worse.

What This Means for How We Work

Four practices follow. The first is to residualize before evaluating: a candidate dataset is judged on what it says after the common factors and the existing inventory have been removed, never on its raw correlation with returns, because the raw correlation is mostly a restatement of exposures already held. The second is to price per dimension: the question put to a data budget is not what a dataset costs but what an independent dimension in its family costs, and the answer is allowed to reject an inexpensive dataset in a redundant family and accept an expensive one in a diverse family. The third is to retire: a dataset whose residual has been absorbed by a newer or cleaner source is a cost without a dimension, and the inventory is pruned as deliberately as it is grown. The fourth is to keep the count explicit. A research group that knows how many independent things its data can say, and how fast that number is changing, can allocate its attention. One that counts datasets cannot.

The catalog will keep growing. The number of independent things the economy does will not grow with it. The distance between those two facts is the information budget, and a systematic research process is, in the end, an organization for spending it well.