Help us understand you so we can provide a tailored experience at your fingertips.
United States
United States
Canada
Mexico
South America
United Kingdom
European Union
Norway
Switzerland
Australia
New Zealand
Japan
Singapore
Saudi Arabia
Location Not Listed
Institutional Investor
Gain access to institution-level resources, investment strategies and risk-management tools designed for sophisticated portfolios.
Financial Advisor
Find tailored resources, fund information and materials to support your advisory practice and client needs.
Individual Investor
Explore straightforward fund details, educational content and tools to help you make informed personal investment decisions.
Loading...
Vol. 3, 2026
Real-Time Refresh
The Prospects and Problems of Real-Time Data Sources
Fed Chair Kevin Warsh announced a task force to improve data quality and timeliness by using alternative real-time data sources. Government data revisions and release delays have indeed misled policymakers, but can real-time data plug the holes? We survey the situation and find that more data is not always better.
By clicking “Sign Up”, you agree to the relevant privacy policy for your location and consent to receive updates.
Share
Fed Chair Kevin Warsh announced a task force to improve data quality and timeliness by using alternative real-time data sources. Government data revisions and release delays have indeed misled policymakers, but can real-time data plug the holes? We survey the situation and find that more data is not always better.
It's April 2001. The 31-year-old future Federal Reserve (Fed) Chair, Kevin Warsh, arrives at his midtown Manhattan office to analyze the latest Gross Domestic Product (GDP) release. The data showed that during Q1 2001, real GDP grew at a solid 2.0% pace.
Today, a quarter century later, we know the U.S. economy actually shrank by 1.3% in Q1 2001 as the dot-com bubble burst.
A similar story played out in September 2008. By then, Kevin Warsh was the youngest Fed Board Governor in history; he and his Fed colleagues discussed raising rates at that month's FOMC meeting. Data available to policymakers at the time showed that U.S. GDP growth had averaged 2.2% in the first two quarters of 2008.
As we now know, and later revisions revealed, by September of 2008, the U.S. economy was already ten months into the worst recession in nearly a century.
These were not the only two instances in history in which data misled policymakers in real time. Nearly 25% of quarters since 1965 have seen GDP revisions of more than 2 percentage points (see Figure 1).
Nearly 25% of quarters since 1965 have seen GDP revisions of more than 2 percentage points.
Fig. 1
It's Always Time For Revisions
Share Of GDP Absolute Revisions By Percentage Point Range
Sources: Bureau of Economic Analysis, Federal Reserve Bank Of Philadelphia
Perhaps it’s not a coincidence that one of Warsh's first actions as Fed Chair in 2026 was to announce the creation of a task force to “improve the quality and timeliness of real economic signals.”
In today’s digitized world, there's surely enough data to improve the “stale” survey methods the government has used for decades.
After all, more data is always better, right?
Not quite. Measuring a $30 trillion economy is harder than it looks. Quality, coverage, and aggregation matter far more than sheer volume.
Below, we explain why more data doesn’t always mean more clarity. Today, economists and investors have more real-time data at their fingertips than ever but still have to live with uncertainty and make their best guess about the future.
Guessing ‘Da Product?
GDP is the holy grail of macroeconomic data, but it has often misled policymakers and investors, and for good reason: it’s a single number that tries to capture the entire output of a $30 trillion U.S. economy, produced just a month after the quarter ends.
To pull this off, the Bureau of Economic Analysis (BEA) combines over 300 data sources, stitching together hundreds of surveys, tax records, and industry reports, stripping out anything counted twice, and filling in gaps with estimates to produce a single GDP print.
The result of this labyrinthine process is that when the preliminary GDP estimate is released, over half of the data are “guestimated” indirectly or imputed from previous trends.
When the preliminary GDP estimate is released, over half of the data are “guestimated” indirectly or imputed from previous trends.
So, aggregating real-time data from all sectors of the economy may be hard, but can we at least capture consumer spending trends using every bank’s consumer debit and credit card data? After all, consumer spending accounts for 68% of U.S. GDP, so knowing how consumers are faring is key to estimating growth.
Many banks already aggregate their spending data. For example, Bloomberg's aggregated private-sector consumer spending data from credit or debit card transactions tracks retail sales trends closely (see Figure 2).
Fig. 2
Private Purchases
Bloomberg Second Measure Consumer Spend* Versus Nominal Retail Sales
Sources: U.S. Census Bureau, Bloomberg
*Bloomberg Consumer Spend takes into account billions of U.S. consumers' credit and debit card transactions derived from a panel of 23+ million consumers
But retail sales are the easiest part of the spending equation, since they capture mostly goods spending, while much spending on services is not “card-able.” For example, payments such as mortgages, rent, utilities, financial services, employer-paid insurance, healthcare, and tuition are not necessarily made with credit cards. Together, these categories make up 44% of quarterly consumer spending.
In a perfect world, if we had transaction data from every bank and service provider, could we make an accurate prediction?
Researchers have tried. In 2025, a BEA researcher combined data from five payment vendors and found that real-time data remained less accurate than the BEA’s final estimate and was much noisier. However, they do provide early signals in a financial crisis, exhibiting a spending decline in March 2020 weeks before official data arrived.
Perhaps real-time spending data can be good leading indicators and contribute to the GDP report, but more signal can also mean more noise, and to date, no alternative can replace comprehensive spending data in getting the whole GDP picture right.
The Good Old Jobs
Okay, tabulating GDP is a monumental undertaking, but what about jobs? Just count the jobs, right?
Officially, the Employment Situation report (also known as the jobs report) tries to do exactly that. They survey 119,000 firms about their employment levels for the month and extrapolate the results to infer net nonfarm payroll (NFP) job growth for over 12 million U.S. establishments.
Surveys will naturally generate estimation errors, especially since the jobs report is usually released a week after month-end. About a third of the months since 1970 saw job revisions of over 100k per month in either direction, representing 40% of the average monthly job growth in the U.S. economy over the last few decades.
But these days, most larger firms use electronic payroll providers. If we count everyone who receives a paycheck each month, isn’t that a timelier and more accurate measure of jobs?
Take Automatic Data Processing (ADP), an electronic payroll provider serving more than half a million U.S. companies. ADP gets employment data from more worksites than the jobs report, but its average absolute monthly revisions are nearly three times larger than those for Bureau of Labor Statistics (BLS) data.
Worse, ADP even fails to compete with the BLS’s first release of nonfarm payroll job gains (see Figure 3). Other private-sector measures of jobs, such as Revelio Labs, which infers employment status by screening people’s online profiles, underperform the first release of NFP even more (see Figure 3 again).
Fig. 3
Big Data...Bigger Errors?
Average Tracking Error* Of Various Job Growth Measures Relative To Final BLS Nonfarm Payroll Data
Source: Bureau of Labor Statistics, ADP, Revelio Lab
*Tracking error measured as measured by the Root Mean Squared Error (RMSE) from 2010 to 2024, excluding March 2020 to March 2021. Revelio Lab data is from March 2021 to 2024. RMSE is a widely used metric in statistics to measure the accuracy of model predictions.
So why do alternative measures underperform government measures, even with more data?
We can suggest a few reasons. Estimation errors may be concentrated where electronic payroll data is weakest: firms with fewer than 50 employees make up 96% of U.S. firms and employ 25% of private-sector workers, and many don't use electronic payrolls at all.
Firms with fewer than 50 employees make up 96% of U.S. firms and employ 25% of private-sector workers.
Real-time data also shares one of the jobs report's biggest blind spots: it struggles to capture jobs gained when businesses open, and jobs lost when they close in real time. As a result, firm births and deaths account for over half of the average measurement error in nonfarm payrolls (see Figure 4).
Fig. 4
One More Round
Total Revisions To Nonfarm Payroll Employment, 12-Month Moving Average
Source: Bureau of Labor Statistics
The BLS initially estimates their impact using a five-year average, which can miss economic turning points. More accurate tax records can take over a year to arrive, leading to large revisions.
Payroll providers may improve the timeliness of estimates, but errors remain: new firms don't always use electronic payroll right away, and a client that drops out may be switching vendors rather than going out of business.
So, counting the economy's net job gains each month is more complicated than it seems. Private payroll providers have more data, but that doesn’t solve all problems.
Pesky Prices
While survey-based data like payrolls may still fall short of real-time accuracy, surely prices, which anyone can check by walking into a store or visiting a public website, are the easiest to improve with more data?
Turns out many debates have occurred over inflation measures, even among existing government-produced inflation indices. For example, over the last 12 months, the Consumer Price Index (CPI), the inflation measure used to adjust Social Security payments, and the Personal Consumption Expenditures (PCE) Price Index, the Fed's official inflation measure, have diverged markedly—one pointing to inflation moderation, the other to acceleration (see Figure 5). So, which one is more accurate?
Fig. 5
What's "True" Inflation?
Gap Between Core PCE And Core CPI Inflation
Sources: Bureau of Labor Statistics, Bureau of Economic Analysis
The answer: all of the above. The difference is in their scopes and weights. PCE inflation covers more, including spending made on households' behalf, such as employer-paid insurance and Medicare, while CPI counts only out-of-pocket spending and weighs housing costs much more heavily.
Alternative real-time price indicators, such as Truflation and PriceStats, suffer a similar issue—even when they scrape extensive online data and also obtain data from retailers and service providers, they still lack the scope and depth of government data. As with spending, many service prices simply aren't available online.
What’s more, real-time price indicators capture more data points but also bring more noise. Sure, they provided early signals during the inflation surge in 2022, but they also sounded alarms on surges or dips that never happened (see Figure 6).
Fig. 6
More Data, More Noise
Core PCE Inflation Versus Alternative Inflation Indicators
Sources: Bureau of Labor Statistics, Tru Labs. Inc.
So more data doesn't always mean clarity—it can also mean more noise and uncertainty.
Join The Party
While real-time data indicators don’t outperform government series, they can complement them when used correctly, and government researchers haven't ignored them.
For example, as early as 2019, Fed researchers tried integrating raw payroll data from ADP with the official jobs report. They found that ADP’s microdata helps reduce nonresponse and sampling errors, making it a useful complement to public sector data.
On inflation, the BLS started using web-scraped or purchased transaction data for gasoline, new and leased vehicles, and wireless telephone services in 2025. Other categories, such as medical services, airfare, hotels, and housing, are in development, while apparel, general merchandise, and food away from home are in early research.
Since September 2020, the Census Bureau has also been blending survey responses with point-of-sale scanner data to increase the frequency of state-level retail sales analysis, although the integration is still experimental because the purchased point-of-sale receipts data has not met all of the Census Bureau’s quality standards.
Progress Pending
So, if private-sector data can’t replace surveys but can still improve them, why hasn’t the government started integrating it into all its reports?
The legal process of adding more private-sector data sources may be even more complicated than the data itself, given the limitations on data privacy and sharing among federal agencies.
Beyond legal barriers, data integration also isn’t as easy to implement as you might think. For example, integrating private-sector data for CPI inflation took the BLS over a year for new vehicle data and almost four years for telephone services, while medical services still remain in research. And, by the way, these are just the beginning—there are 211 groups of items total in CPI.
Globally, the Australian Bureau of Statistics (ABS) was among the first government agencies to publish private-sector payroll statistics. But transforming payroll data into a monthly jobs measure took nearly a decade. The statistics bureau only began publishing one in 2026, and it still sits alongside, not in place of, the survey-based labor force report.
We, like Fed Chair Warsh and every investor, hope for better data and see the value in integrating private-sector data. But we recognize that designing and producing the extensive public-sector data we enjoy today is much more complicated than just adding up the numbers.
For all its allure, real-time data is not a miracle cure for today's data problems. Both investors and the Chairman’s task force, hoping that more data means more clarity, may be in for another disappointment.
5. Mallatt, J. (2025). The use of real time spending sources for economic measurement: An analysis using multiple data sources (BEA Working Paper No. WP2025-4). U.S. Bureau of Economic Analysis.https://www.bea.gov/sites/default/files/papers/BEA-WP2025-4.pdf
6. U.S. Bureau of Labor Statistics. (n.d.). Technical notes for the CES national benchmark. U.S. Department of Labor. Retrieved September 28, 2026, fromhttps://www.bls.gov/web/empsit/cestn.htm
9. Truflation is a private-sector U.S. inflation index, updated daily, that aggregates price data from a wide range of third-party sources. According to its methodology paper, it relies on partners across retail, housing, energy, autos, and travel to improve coverage, granularity, and accuracy. These include NielsenIQ, Amazon, Walmart, Zillow, CoreLogic, the Energy Information Administration, J.D. Power, and hotel chains such as Hilton and Hyatt. Meanwhile, PriceStats, produced in partnership with State Street, grew out of MIT's Billion Prices Project, founded by economists Alberto Cavallo and Roberto Rigobon. It scrapes prices daily from retailers with both online and physical stores, since research shows retailers tend to adjust online prices first. The result is a daily inflation index designed to be comparable to the official CPI.
12. U.S. Government Accountability Office. (2025). Highlights of a forum: Expert views on the federal statistical system (GAO-25-107124).https://www.gao.gov/assets/gao-25-107124.pdf
This material reflects the firm’s current opinion and is subject to change without notice. Sources for the material contained herein are deemed reliable but cannot be guaranteed. This material is for illustrative purposes only and does not constitute investment advice or an offer to sell or buy any security. Past performance is no guarantee of future results. Point of View articles may not be reprinted without permission. We welcome your comments and feedback at editor@payden.com.
This material has been approved by Payden & Rygel Global Limited which is authorised and regulated by the Financial Conduct Authority. This material has been approved by Payden Global SIM S.p.A.. which is authorised and regulated by CONSOB.
