For much of the 20th century, R&D was viewed as a long-term investment in scientific discovery, often insulated from day-to-day operational pressures. Success was measured in patents, breakthrough technologies, and the promise of future growth. Today, that view no longer holds.
Across energy, chemicals, and broader industrial sectors, R&D has been reshaped by a combination of macroeconomic shocks, shareholder expectations, and rapid technological change. The result is a new reality in which innovation is no longer a standalone function, but a tightly managed, performance-driven capability embedded within the business.
The changing role of R&D
The pressures shaping R&D today did not emerge overnight. They began to build as early as the 1980s, when globalization, increased competition, and rising capital intensity forced companies to become more disciplined in how they invested. Through the 1990s, these pressures continued to simmer, as organizations began to rethink the role of centralized R&D and the need to connect innovation more directly to business outcomes.
That shift accelerated significantly over the past decade. Following commodity price volatility and sustained margin pressure in the mid-2010s, companies became far more rigorous in how they allocated capital. R&D was increasingly reframed as a tool to support the core business, with a stronger emphasis on cost reduction, operational efficiency, and asset optimization. The focus moved toward initiatives that could deliver immediate, measurable financial impact.
This improved financial discipline, but it also narrowed how companies defined innovation. Efforts that could not clearly demonstrate near-term value became harder to justify, and longer-term initiatives were often deprioritized or delayed.
In the early 2020s, a new set of pressures began to shift the picture again. The energy transition, regulatory expectations, supply chain disruption, and advances in digital technologies forced companies to re-engage with longer-term priorities. R&D agendas expanded to include decarbonization, advanced materials, and new value chains that sit further out on the horizon.
However, this was not a return to the past. Companies did not step back from the discipline they had built. Instead, they layered these longer-term ambitions on top of an already constrained R&D environment. As a result, R&D is now expected to support both near-term performance improvements and longer-term transformation.
Exhibit 1: Three Horizons framework for innovation.
Innovation is under pressure
These shifts have created a new set of challenges that many industrial companies are still working through.
Stricter financial expectations. The business case for R&D is far more demanding than it was historically. Innovation investments are now expected to demonstrate clear financial returns, with metrics tied directly to profitability rather than activity. This naturally favors initiatives that deliver near-term value and makes longer-term efforts more difficult to sustain.
A persistent gap between ideas and scale. While companies continue to generate strong ideas and early-stage innovation, the ability to translate those ideas into commercially viable outcomes remains limited. Moving from pilot to full-scale deployment requires significant capital, operational alignment, and market confidence, all of which introduce friction and delay.
Rising complexity in how R&D is executed. Innovation is no longer confined to centralized research teams. It now spans digital, operations, and commercial functions, while also increasingly relying on external ecosystems. Companies are combining internal R&D with open innovation models, corporate venturing, startup partnerships, university collaborations, and offshore or specialized R&D centers. These approaches expand capabilities, but they also make coordination, governance, and execution significantly more complex.
The challenge of scaling new technologies. Advances in AI, data, and simulation are changing how R&D is conducted, but many organizations struggle to scale these capabilities beyond isolated use cases. Integrating digital tools into core processes and realizing enterprise-level impact remains a major hurdle.
External pressures shaping internal priorities. Commodity cycles, regulatory requirements, geopolitical uncertainty, and evolving customer expectations are all influencing R&D decisions. Innovation is no longer driven solely by internal strategy. It is increasingly shaped by external constraints that can change quickly.
While long-term innovation remains a strategic priority, it is often deprioritized in practice in favor of initiatives that deliver near-term results. Over time, this leads to fragmented portfolios, stalled scaling efforts, and a growing disconnect between innovation ambition and business impact. For many companies, the challenge is no longer generating ideas, but converting them into outcomes that can compete for capital alongside core business initiatives.
How leading companies are responding
While many organizations face similar constraints, leading companies are beginning to respond in more deliberate ways.
Some are restructuring how their R&D portfolios are managed, creating clearer distinctions between short-term and longer-term initiatives. Arkema, for example, has shifted its innovation focus toward higher-value specialty materials and sustainable product lines while maintaining discipline in capital allocation. This has allowed continued investment in areas such as battery materials and low-carbon solutions without forcing those efforts to compete directly with near-term operational priorities.
Others are integrating R&D more directly with commercial and operational teams. At Element Solutions, parts of the R&D function have been reorganized to work more closely with application engineering and customer-facing teams. Aligning innovation earlier with customer requirements has improved the company’s ability to move new products into commercial use and reduced the likelihood of stalled pilots.
There is also a shift toward more focused use of external innovation ecosystems. BP has increasingly relied on targeted partnerships to complement internal R&D, particularly in areas such as digital twins and asset optimization. These efforts are typically concentrated on specific capabilities that can be deployed within existing operations, improving the chances that innovation translates into practical impact.
At the same time, organizations are rethinking how digital tools are applied within R&D. DuPont has expanded the use of data, analytics, and AI-driven platforms across its innovation processes, linking laboratory work more directly with product development and commercialization. This has helped improve development efficiency and increase the contribution of newer products to overall revenue.
Finally, some companies are placing greater emphasis on execution discipline. In advanced materials markets, Daxin Materials has focused on close customer collaboration and targeted product development tied directly to end-use applications. This approach has helped reduce fragmented innovation efforts and improved the ability to scale products more efficiently.
What comes next?
The examples above point to a broader shift in how industrial R&D is being managed. Leading companies are not necessarily investing more, but they are becoming more deliberate in how innovation is structured, prioritized, and connected to the business.
This marks a departure from the traditional model of R&D as a largely autonomous function. Instead, innovation is increasingly embedded across the organization, shaped by financial discipline, operational needs, and external constraints. The challenge is no longer generating ideas or expanding pipelines, but ensuring that innovation efforts can compete for capital, scale effectively, and deliver measurable impact.
For many companies, this transition remains incomplete. The underlying pressures identified earlier are still present, and the complexity of managing R&D continues to grow. Bridging the gap between ambition and execution requires more than incremental adjustments. It requires a clearer approach to portfolio design, stronger integration across functions, and a more disciplined focus on outcomes.
As these demands intensify, the ability to manage R&D effectively will become a more meaningful differentiator across industrial sectors. Companies that can align innovation with business priorities and consistently translate ideas into scalable results are more likely to capture value from their R&D investments. Those that cannot will continue to face stalled initiatives, fragmented portfolios, and growing pressure on performance.
— Piercen Hoekstra
About ADI Analytics
ADI is a prestigious, boutique consulting firm specializing in oil and gas, energy, and chemicals since 2009. We bring deep expertise in a broad range of markets where we support Fortune 500, mid-sized and early-stage companies, and investors with consulting services, research reports, and data and analytics, with the goal of delivering actionable outcomes to help our clients achieve tangible results.
We also host the ADI Forum that brings c-suite executives together for meaningful dialogue and strategic insights across the oil & gas, energy transition, and chemicals value chains. Learn more about the ADI Forum.
Subscribe to our newsletter or contact us to learn more.
