Factory with thought bubbles representing future expectations

Decoding Productivity: How Subjective Expectations Can Revolutionize Business

"New research reveals how tapping into a company's collective intuition can unlock hidden potential and drive unprecedented growth."


The ‘production function,’ representing how inputs translate into outputs, has long captivated economists. Understanding this process is crucial for analyzing technological change, productivity dispersion, firm markups, and the impact of policy. Recent productivity growth slowdowns have only amplified the need for accurate and reliable methods.

For years, economists have grappled with estimating production functions, facing challenges like input endogeneity – the unobservable correlation between a firm's productivity and its input choices. Traditional methods, including fixed effects models and lagged input values, often fall short, yielding implausible results or requiring extensive data.

A new approach is emerging, leveraging data on firms' subjective expectations of future output and inputs. This method offers a way to obtain consistent production function parameter estimates while relaxing assumptions about input demand policies. By tapping into a company's collective intuition, this innovative technique promises to revolutionize how we understand and estimate productivity.

AI Search Multiple angles on this topic

The Productivity Imperative in Context

While reliable, widely cited statistics on the productivity gap that subjective expectations might help close remain scarce and context-dependent, productivity itself has become central to debates over business competitiveness and economic growth. Sector analyses and practitioner reports suggest that a substantial share of output gains in modern firms flows from how work is organized and how performance is perceived, rather than from raw capital or labor inputs alone. Because such figures vary by industry, region, and methodology, they are best read as indicative rather than definitive. What is less disputed is that even modest, expectation-driven improvements in planning and engagement tend to have measurable consequences for output quality and timeliness over time.

How Production Is Defined Today

In mainstream economics and business education, production is understood as "a process of value addition" that transforms inputs such as labor, raw materials, capital, energy, and information into finished products and services of proper quality and quantity, as described by the resource EducationLeaves. Complementary accounts emphasize the major forms of this activity, including market production, public production, and household production, which together are said to explain much of the origin of economic well-being. Dictionaries reinforce the same frame, with Merriam-Webster defining production simply as "something produced" and broader usage extending the term to the arts, entertainment, and media, where it denotes assembling, crafting, or presenting a work. A common limitation of these accepted definitions is their focus on physical transformation and countable outputs, which leaves the subjective expectations and perceptions shaping how productivity is assessed largely outside the standard model.

A Long-Standing Dual Meaning

The dictionary record shows that "production" has long carried a dual meaning: the process of making or growing goods to be sold, and the amount of something that is made, as documented in the Cambridge Dictionary. This pairing of process and quantity reflects an enduring emphasis on goods produced for exchange rather than on the productivity of the people doing the producing. That orientation persisted across agricultural, manufacturing, and service economies, anchoring early productivity metrics in countable units of output. Only in more recent discourse, the evidence suggests, has attention shifted toward the subjective expectations behind that output, a theme that classical dictionary definitions do not yet acknowledge.

Why Subjective Expectations Matter: A New Perspective on Productivity

Factory with thought bubbles representing future expectations

Traditional methods for estimating production functions, such as those pioneered by Olley and Pakes (OP) and Levinsohn and Petrin (LP), rely on assumptions about input choices. They often require a strictly monotonic relationship between a firm's input demand and productivity. This can be problematic, as factors like input adjustment costs, varying prices, and optimization errors can undermine this relationship.

The new method, outlined in a recent research paper, introduces a different approach. It uses data on firms’ expectations of future output and inputs to estimate production function parameters. This leverages the idea that a firm’s expectations contain information about its anticipated future productivity, providing a new way to control for unobserved variables.

Here's how this new method stacks up against traditional approaches:
  • Relaxed Assumptions: Unlike OP, LP, and ACF, this new approach does not require firms' decisions to be perfectly optimal.
  • Flexibility: It can accommodate non-linear productivity dynamics, while dynamic panel methods typically require linearity.
  • Data Efficiency: The method can identify production function parameters from a single cross-section of data, removing the need for multiple observations per firm.
AI Search Multiple angles on this topic

An Evolving Research Frontier

Current research on subjective expectations and productivity is still relatively early-stage, with findings emerging across behavioral economics, organizational psychology, and management science rather than coalescing around a single accepted framework. Much of this work, as reported in recent reviews, suggests that managers' and employees' beliefs about outcomes can shape actual performance through planning, effort allocation, and feedback loops. Because methodologies and sample sizes differ markedly across studies, individual results should be treated as provisional, converging evidence rather than settled fact. The field appears to be moving toward more quantitative field experiments that test expectation-based interventions in real business settings.

Skepticism and Known Limitations

Critics of expectation-centered productivity models argue that subjective beliefs are difficult to measure reliably and may be confounded with optimism, motivation, or self-report bias. Attempts to act on such expectations have also been linked to failures, including overconfidence, misallocation of effort, and targets that become decoupled from objective performance. Skeptics further note that many productivity differences are driven by hard factors, such as capital investment and technology, that no change in perception can replace. A balanced view is that expectation-based approaches are a complement to, not a substitute for, structural improvements in how work is organized.

Comparison with Conventional Productivity Models

Relative to conventional models that treat productivity as a ratio of outputs to inputs, an expectation-based approach emphasizes the beliefs and forecasts that precede and shape those outputs. The conventional view offers objectivity and ease of measurement, while the expectation-based complement explains variation that purely mechanical input-output accounting cannot, such as why similar teams with identical resources perform differently. Each framework carries trade-offs in measurement cost, interpretability, and predictive power. Most thoughtful treatments therefore propose integrating the two, using standard metrics as the baseline and subjective expectations to explain the residual gap.

While this new method requires data on firms’ subjective expectations, such data are becoming increasingly available through surveys like the Management and Expectations Survey (MES). This makes the approach a viable and attractive alternative to traditional methods.

A New Era of Productivity Analysis

This research offers a compelling new approach to estimating production functions. By leveraging firms' subjective expectations, it relaxes restrictive assumptions about input choices and opens new avenues for understanding productivity. As data on subjective expectations become more readily available, this method has the potential to transform how businesses and policymakers analyze and promote economic growth.

AI Search Multiple angles on this topic

Bringing the Threads Together

Commentators synthesizing the academic and practitioner literature tend to converge on the view that productivity is not purely a technical property of inputs but is also a consequence of the expectations people hold and act upon. The strongest accounts treat measurement rigor from classic production economics as the necessary foundation, then layer subjective-expectation insights on top to explain otherwise puzzling performance differences. Experts caution, however, that the evidence base remains uneven and that enthusiasm should not outpace replication. The emerging consensus is that expectation management is best treated as a cross-cutting input, like capital or skill, rather than a novelty.

Where the Conversation Is Heading

Looking ahead, plausible next frontiers include the use of richer data, such as organizational analytics and longitudinal surveys, to test expectation-based hypotheses more rigorously. Advances in measurement technology may make beliefs about performance more visible and comparable across firms and industries than ever before. Researchers anticipate closer integration with behavioral design, where nudges and feedback systems are engineered specifically to calibrate expectations. Early signs point to a more personalized approach, tailoring expectation-setting practices to team context and individual differences.

Wider Economic and Organizational Pressures

The discussion of subjective expectations sits within broader systemic pressures, including stagnant measured productivity growth in several advanced economies, rising demands for worker well-being, and the accelerating adoption of automation. These pressures complicate any single explanation for performance gaps and make context-specific, expectation-aware strategies more relevant. Systemic challenges such as data fragmentation, reporting incentives that distort self-assessment, and short-termism can also undermine efforts to manage expectations honestly. Addressing these structural conditions appears necessary for expectation-based approaches to deliver durable gains.

What It Means for People at Work

At the human level, the shift toward subjective expectations implies that how goals are set, communicated, and believed can materially affect daily work experience and output. Real-world accounts suggest that when people internalize credible, well-calibrated expectations, they direct effort more deliberately and respond productively to feedback. Conversely, poorly managed expectations are associated with burnout, disengagement, and turnover. The practical promise of this perspective is that it treats the beliefs of employees and managers as a genuine, addressable resource rather than a by-product of production.

Estimating production functions from firms' expectations

Populationfirms[1]
SettingUK panel data[1]
Main outcomeproduction function parameter estimates[1]
Effect sizeThe share of cross-firm TFP dispersion accounted for by persistent productivity differences is substantially larger using the proposed estimator's parameter estimates.[1]
Times cited0[1]

How the expectations estimator compares with other approaches

On UK panel data, the proposed estimator produced results described as similar to or more credible than commonly used alternatives; this is a comparison of estimates, not a claim that it wins in every setting.[1]

The paper’s simulations add a specific comparison: when firms’ material-input choices include optimization error, the estimator outperformed alternatives. The authors also report that it can be used with a single cross-section, unlike dynamic panel methods.[1]

The paper notes that the Levinsohn–Petrin approach uses material-input demand as a proxy to address a selection issue associated with zero investment. This helps explain one reason proxy methods differ, beyond the assumptions already described in the article.[2]

What the estimates change about productivity differences

Using the proposed estimator’s parameters, persistent productivity differences account for a substantially larger share of cross-firm TFP dispersion. This means the estimated explanation of productivity gaps depends in part on the estimation method; it does not establish that expectations themselves caused those gaps.[1]

Taken together, the UK comparison and simulations suggest the method may be useful where input decisions are imperfect, while leaving open how well its relative performance carries to other data and conditions.[1]

About this Article -

Written with AI assistance from published research, and reviewed by the Mystum team. See our About page for more information.

This article is based on research published under:

DOI-LINK: https://doi.org/10.48550/arXiv.2407.07988,

Title: Production Function Estimation Using Subjective Expectations Data

Subject: econ.em

Authors: Agnes Norris Keiller, Aureo De Paula, John Van Reenen

Published: 10-07-2024

Everything You Need To Know

1

What is a production function and why is it important in business and economics?

The production function represents how inputs translate into outputs. It's a fundamental concept for understanding technological change, productivity dispersion, firm markups, and the impact of policy. Accurately estimating the production function is critical for analyzing economic growth and understanding how efficiently businesses utilize resources. Recent slowdowns in productivity growth have amplified the need for reliable methods to analyze this relationship and make informed decisions.

2

What are the challenges in estimating a production function using traditional methods?

Traditional methods face challenges like input endogeneity, where a firm's productivity and input choices are correlated in unobservable ways. Methods like fixed effects models and lagged input values often fall short, yielding implausible results or requiring extensive data. Moreover, these approaches, such as those pioneered by Olley and Pakes (OP) and Levinsohn and Petrin (LP), rely on restrictive assumptions about input choices, such as a strictly monotonic relationship between input demand and productivity. These assumptions can be problematic due to factors like input adjustment costs, varying prices, and optimization errors.

3

How does the new method using subjective expectations differ from traditional approaches like Olley and Pakes (OP) and Levinsohn and Petrin (LP)?

The new approach leverages data on firms' subjective expectations of future output and inputs to estimate production function parameters. Unlike OP, LP, and ACF, this method relaxes assumptions about input choices, accommodating non-linear productivity dynamics, and requiring less data. It can identify production function parameters from a single cross-section of data. Traditional methods often require multiple observations per firm and make more restrictive assumptions about how firms make decisions.

4

What is the role of subjective expectations in the new approach to estimating production functions?

The new method utilizes firms' expectations of future output and inputs. These expectations contain information about a firm's anticipated future productivity, providing a new way to control for unobserved variables, like input endogeneity. By tapping into a company's collective intuition, this innovative technique promises to revolutionize how we understand and estimate productivity. The Management and Expectations Survey (MES) is an example of how such data is being collected to facilitate this new approach.

5

How can businesses and policymakers benefit from this new method of analyzing productivity?

By leveraging firms' subjective expectations, the new method offers a more flexible and accurate approach to understanding productivity. This allows businesses to make better-informed decisions about resource allocation, investment, and innovation. Policymakers can use this method to gain deeper insights into economic growth drivers and the impact of various policies. As data on subjective expectations becomes more readily available, this method has the potential to transform how businesses and policymakers analyze and promote economic growth.

Newsletter Subscribe

Subscribe to get the latest articles and insights directly in your inbox.