In my last two stories about how venture capitalists are using AI, we discussed how investors are currently leveraging data science to help source and make decisions about potential investments, including why VCs have been slow to adopt AI. We also explored the results of a survey that highlighted how, where, and why data scientists on VC teams are using AI, and what effect it’s having on the work of the firms they work at.

Given this work, I’d like to close out this series with a short overview of three potential futures we see for AI in VC, each with vastly different outcomes for the industry.

 

Future #1: Leveraging data will create a sustainable competitive advantage

In this scenario, we assume that it is genuinely possible to find real competitive advantage over other firms using AI. Firms who see the biggest advantage will identify narrow areas in which to use AI to directly impact their differentiation from others. To identify those areas, recall our four-tier framework to help categorize the different types of opportunity AI poses to venture firms:

In this scenario, any firm that invests in the upper two quadrants will be able to build or sustain a competitive advantage over its peers. As a reminder, here are some examples of efforts that fall under those categories:

Operational Leverage: Aggregating and synthesizing data for easier company analysis, automating data entry, consolidating relationship data for more informed outreach, synthesizing portfolio health report.

Real Competitive Advantage: Creating predictive models for niche investment themes, algorithmically finding ways to expand networks, automating insights extraction from diligence documents, identifying emerging trends faster than competitors.

It’s worth noting that the lower two quadrants, which cover everything from email automation and common data vendors (low-hanging fruit) through CRM and network mapping (foundational) will be table stakes in the near-to-medium term.

Regardless of which future we find ourselves in, efforts that offer operational and competitive leverage should be the most important pieces of a VC data scientist’s job. In this specific future, the list of tasks above could be mixed, matched, and tailored to a fund’s specific strategy, making them less generalizable but highly valuable when aligned with a firm’s unique strengths.

VCs have a lot of different directions to take their automation and data efforts but under this scenario, firms that fail to make strides in those two categories will also fail to gain a competitive advantage. Right now, our industry is focusing too much on the “low-hanging fruit” opportunities, which would explain why our survey showed that firms are not seeing a lot of tangible impact from AI at the moment. The takeaway here is that firms should be bold and make strategic bets in AI — and do so sooner than later because, as we’ll see in our next hypothesis, timing will likely be important.

 

Future #2: In the short term, there will be an advantage for first movers

As we’ve discussed, there is a lot of friction in VCs’ work processes, and AI offers a unique chance to remove a lot of it. However, under this scenario, the ability to autonomously aggregate and synthesize company data for analysis, automate data entry, and other “operational” opportunities will become commoditized over time. As a result, the only real advantage will go to the VC firms whose data scientists move first.

This scenario assumes that there is no durable, long-term differentiation to be found for firms that invest in AI efforts, even those we categorized as “real competitive advantage.” The only advantages will play out over the short term, and will only flow to those who invest early and move fast. The advantage will be short-lived, however, as best practices spread through the industry and those efforts become commoditized.

 

Future #3: All AI efforts in VC will become commoditized

This hypothesis is simple: all potential applications of AI in VC will be relatively easy and quick to implement, and will offer no durable advantage over others. In this scenario, the only “losers” will be firms that fail to invest in AI at all.

If you believe in this outcome, you believe that all tooling that VCs can build for themselves will be based on similar systems and ideas, and there will be a low ceiling on how creative industry players can get with them. The result would be a race to the bottom, where everyone builds similar tooling with few competitive rewards on offer beyond mere survival. The result will be a VC landscape similar to the one we currently live in, where there will be little to differentiate funds other than brand and fund size — capital remains capital, regardless of who’s handing it over.

Regardless of how the future of data in VC plays out, the truth underlying all these realities is that data (and specifically data science) is going to play a role in the future of our industry. The degree of the potential advantage will vary widely for those who invest, but the main losers will be those funds that don’t incorporate data and AI into their workflows at all.

In our previous article, we explored how data in venture capital is reaching a pivotal moment. With various market forces driving more VCs to invest in data efforts, we outlined several key areas where data can be leveraged effectively, and discussed different levels of impact these efforts are having.

To build on that foundation, we conducted a survey among our peers in the industry to understand how they are leveraging data, engineering, and automation, and evaluate the effectiveness of these efforts. This article delves into the survey results to highlight what areas are most important to VCs, where data initiatives have been most impactful, and how firms can interpret these findings to optimize their own strategies.

---

Survey Insights

Q1: Have you increased investment in data/engineering over the past 5 years?

The response is clear: 75 percent of the more than 50 firms surveyed have indeed increased their investment in data and engineering in the past five years. This trend aligns with our discussion in the last article about the growing emphasis on data-driven strategies within VC.

Q2: What are the key areas where you apply data science/engineering, and in what ways do you leverage it?

We asked respondents to categorize their data efforts into the five key areas discussed in our previous article.

At a high level, the places where people seem to be putting the most effort seem to be those where there is a high availability of data, room to build impactful predictive models, and a clear and direct impact to investing performance. Here are some additional details:

Q3: In what area do you think your data efforts have yielded the most impact and automation?

If you look at the proportion of people who believe their efforts in each area have yielded some or a lot of impact, it more or less correlates to what firms focus on most. Data efforts in “sourcing” and “portfolio management” are perceived as the most impactful, and the rest don’t have as much consistent value. Despite this, only 50 to 60 percent of firms reported achieving meaningful impact from their efforts, suggesting a gap between potential and realized outcomes. Though this looks like an existential crisis for data driven VC, we believe this gap stems from a focus on low-hanging fruit and foundational efforts rather than more impactful, differentiated projects.

Here are some examples of lower level projects vs. those that create differentiated value:

Sourcing

🍇 Low-hanging fruit:Scraping popular public lists of companies

🏆Competitive advantage: Building models to predict the relevance of companies to your firm’s strategies, the likelihood of them responding to a cold outreach, the likelihood of them getting through stages of your pipeline

Portfolio Management

🏗️Foundational: Aggregating portfolio data in a consistent structured format

🏆Competitive advantage: Accurately automating aspects of portfolio support requests (customer requests, hiring requests), modeling portfolio metrics against past and new non-portfolio companies that you meet

Pipeline Management

🍇 Low-hanging fruit: Creating automated email and linkedin campaigns for cold outreach

🏆 Competitive advantage: Consistently reprioritizing your early pipeline based on signals like the likelihood of the company looking to raise soon and fit with your firm’s strategy

Company evaluation

🏗️ Foundational: Organizing all files from companies you meet with and making it queryable

🏆Competitive advantage: Compare the business models, markets, and so on with new companies to similar past companies seen in the pipeline, synthesize an initial analysis of a company based on company files

Managing people networks

🏗️ Foundational: Mapping the experiences and expertise of everyone in your network and being able to search/filter it

🏆Competitive advantage: Proactively finding relevant holes in your network and finding secondary connections to fill them

Automation vs. Impact

The survey also highlighted automation trends across these areas. Generally, automation levels are relatively low across all domains, with “sourcing” and “portfolio management” seeing the most automation. However, there are key insights to consider:

1. Limits of Automation: There is a cap on how much automation is feasible in any given area. This limitation arises from the complexities involved in obtaining quality data, generating reliable insights, creating valuable outputs, and developing effective workflows. For example, while the potential for automation in “sourcing” is high due to the availability of structured data, areas like “company evaluation,” where data is sparse and unstructured, have lower automation potential.

2. Automation vs. Impact: It’s important to note that automation does not directly equate to impact. Even if an area cannot be fully automated, the impact can still be significant. For instance, “human-in-the-middle” systems, such as research tools that help query past company data, may only automate the information retrieval part of the process but can greatly reduce research effort and prevent the loss of valuable insights.

3. Automation as Low-Hanging Fruit: Currently, most automation efforts fall into the “low-hanging fruit” category, providing immediate efficiency gains. However, as data teams move towards more foundational, operational leverage, and competitive advantage projects, the scope for full automation diminishes, and its direct impact becomes less pronounced. More complex, high-impact efforts often require a balance of automation and human insight while also being more bespoke.

These findings suggest that while automation can offer quick wins, the real value lies in combining automation with strategic, high-impact data efforts that provide long-term differentiation and competitive advantages. As firms mature in their data capabilities, focusing on areas with the greatest potential for sustained impact will be crucial.

The survey results highlight a clear trend: data teams in venture capital are concentrating their efforts in areas with abundant data, opportunities for impactful predictive modeling, and a direct link to enhancing investment performance. This makes sense. However, it’s important to note that despite their efforts, many firms do not perceive much impact. This is likely due to efforts being spent on low-hanging fruit and foundational projects.

But as data capabilities mature, it is important for VC firms to move beyond basic automation and low-impact projects and towards differentiated efforts. Beyond organizing, pulling structured insights, and building predictive mechanisms from external data sources like Harmonic and LinkedIn, this means doing the same for your rich internal datasets like company diligence documents and portfolio company data.

There are dozens of data and AI initiatives you can pursue in each of our core categories, but if you prioritize them in the framework we have described, you’ll be more likely to reap value for your fund. In our next article, we will explore the industry’s expectations for the future of data, engineering, and automation in VC, and how firms can prepare for the next wave of innovation in this space.

Venture capitalists are playing a key role in the ongoing boom in artificial intelligence, helping provide capital and guidance for startups seeking and capitalizing on exceptional use cases for AI. So it may come as a surprise to learn that data and AI adoption among VCs themselves has lagged behind the industries they invest in.

I’m a data scientist for F-Prime’s tech fund, and this is the first in a series of three blog posts in which I will explore why AI adoption among VCs has lagged behind the tech industry, how and why that is starting to change, and what the future looks like. As part of this effort, my colleagues over at Eight Roads and I conducted a survey among our fellow data scientists and engineers at other venture capital firms to gauge where they have been most impactful.

 

A Slow Start in Data Utilization

Data adoption in venture capital has lagged behind other industries. Historically, data sources were limited, expensive, and not very actionable. About 10-15 years ago, VCs primarily used niche data for market trends and consumer behavior, but the accuracy was questionable, and integrating this data into actionable insights was challenging.

At the same time, there was a scarcity of skilled data and engineering talent. Most of the talent gravitated towards industries with more tangible problems, like software and public finance which, ironically, were often funded by the same VCs. Finally, limited management fees and small fund sizes further constrained VCs, leaving little room for dedicated data or engineering teams. Consequently, only a small fraction of VC firms globally (five percent) have a software engineer, data engineer, or data scientist on staff. Even when looking at medium to large-sized firms (those with more than ten team members), this number rises to just 20 percent.

A Turning Point for Data in VC

Despite these historically low numbers, we believe VC is approaching an inflection point where concerted data and engineering effort is about to accelerate. Several factors are driving this change:

1. Improved Data Quality and Accessibility: The availability of data has expanded significantly, with data providers mapping more companies and team members, making it easier to track and analyze market trends and potential investment opportunities. Access to non-traditional data sources like credit card transactions and web trends has increased, and data is now more accessible through user-friendly interfaces and APIs. Accuracy and freshness have improved, and increased competition among providers has lowered costs. On top of all of that, GenAI tools further simplify data extraction and structuring, accelerating the trend of making high-quality data more accessible and affordable.
2. Easier Access to High-Quality Talent: The pool of data and engineering talent is growing as more individuals receive formal education in computer science and data science. Technological advancements have abstracted complex tasks like building data pipelines, integrating systems, and developing predictive models, making these processes more accessible even to those with basic training. Again, GenAI tools accelerate trends here, allowing engineers to achieve more with less effort — even average engineers can become “10x engineers.”
3. Increased Management Fees and Larger Fund Sizes: As VC funds grow in size, more resources are available for data and automation initiatives. AI technologies also reduce costs associated with research, due diligence, and operational work, freeing up more resources for data projects. This increased financial flexibility allows VCs to invest in comprehensive data strategies.

 

The Current State of Data Use in Venture Capital

As more venture capital firms begin to build out their data capabilities, the big question becomes: How are we leveraging data to create a real impact for our firms? We’ve created a framework to think through where firms are focusing their efforts and how we think about the impact of those efforts.

What follows is a high-level overview of the framework and we’ll dive deeper into examples — and explore how the industry thinks about data in VC — in the next two articles.

Where we’re focusing our data/engineering efforts:

1. Sourcing: Use data to identify high-quality, timely investment opportunities.
2. Pipeline Management: Improve the management and prioritization of companies at each pipeline stage through data-driven approaches.
3. Network Management: Map and analyze the firm’s networks to identify gaps and track changes in real time.
4. Company Evaluation: Systems that help more easily diligence companies, with the goal of eventually automating out some diligence task.
5. Portfolio Management: Automate portfolio health tracking, covering performance, competitive landscape, and team changes.

 

Evaluating Impact: A Four-Tier Framework

Let’s define some of the terminology you see in the graphic above, and I’ll offer my opinion on how much importance they should hold for VC data scientists and engineers.

Low Hanging Fruit 🍇
Definition: Projects that offer short-term advantages by reducing operational time and costs. These benefits are typically temporary.
Examples: Email/LinkedIn automation (pipeline management), scraping public company lists (sourcing)
Our Opinion: While these are easy to implement and can provide quick wins, their value diminishes as more VCs adopt similar strategies. Over-reliance on these can create dependencies on low-impact solutions, so smart firms will not invest too heavily in “low-hanging fruit” use cases.

Foundational Projects 🏗️
Definition: Essential projects for any VC aiming to build a data-centric approach. These efforts don’t immediately boost efficiency but set the stage for future data-driven initiatives.
Examples: Mapping and filtering networks (network management), setting up a CRM system (pipeline management, network management), extracting and structuring portfolio data (portfolio management), creating a research portal on top of files you’ve collected from startups (company evaluation)
Our Opinion: These projects are vital for enhancing a fund’s capabilities, even though they might not provide immediate competitive benefits. They are necessary to establish a solid base for more advanced data efforts.

Operational Leverage 📈
Definition: Projects that have a lasting impact by reducing time and costs related to specific tasks. These are often customized to fit a firm’s existing workflows developed in the foundational phase.
Examples: Aggregating and synthesizing data for easier company analysis (company evaluation), automating data entry (pipeline management), consolidating relationship data for more informed outreach (network management), synthesizing portfolio health reports (portfolio management)
Our Opinion: These initiatives are worth investing in because they reduce costs and time in the long run, which helps maintain efficient operations, even if they don’t fundamentally change the firm’s overall operations. They typically build upon foundational efforts and tend to be more precise, only improving aspects of workflows.

Real Competitive Advantage 🏆
Definition: Projects that significantly impact operations and differentiate a fund’s strategy. They build on a fund’s existing strengths or offer unique advantages.
Examples: Creating predictive models for niche investments (sourcing), algorithmically finding ways to expand networks (network management), automating insights extraction from diligence documents (company evaluation), identifying emerging trends faster than competitors (sourcing)
Our Opinion: These are the most impactful projects and should be prioritized. They are often tailored to a fund’s specific strategy, making them less generalizable but highly valuable when aligned with a firm’s unique strengths. Focusing on these can create lasting competitive advantages.

Understanding where and how to invest in data initiatives is crucial for VCs looking to stay competitive. That is why we have created this framework; to make it simpler for industry players to understand where to focus efforts to reap the most impact. But in order to get a more holistic understanding of what is important, we sent a set of peer VCs a survey asking how they have leveraged data and how it has impacted their firm, based on this framework. In the next article we will discuss the results of this survey and understand where the majority of value lies when it comes to leveraging data for VC.

Enterprises demand better integration from SaaS vendors to support unified data repository initiatives

Enterprises are making significant investments to build robust data foundations to get ready to power their AI initiatives. Modern cloud-based data platforms like Snowflake, Google BigQuery, and Microsoft Fabric have provided the technical means to consolidate datasets scattered across various platforms into unified repositories — often known as Lakehouses or Data Warehouses.

At the same time, these enterprises are also the largest buyers of SaaS solutions. SaaS platforms power critical enterprise functions like CRM, HR, finance, and marketing. However, the data created or consumed by SaaS solutions often exists in silos, creating a conflict with this architecture and governance objective.

Each SaaS deployment is an island of data, with critical enterprise data trapped inside the product. This data is difficult to discover, challenging to govern, and, most importantly, difficult to extract and merge with other datasets of an enterprise for powering analytics and AI.

SaaS vendors have been slow to provide mechanisms to extract data easily and move it to analytics data stores. They have provided APIs and flat file interfaces, but these are inadequate and inefficient to meet the needs of enterprise initiatives to build a true enterprise-wide data platform capable of driving analytics and AI.

Often the SaaS vendor requires significant amounts of sensitive data from clients to provide a specific service. For example, many CDP SaaS vendors require customer 360 profiles to be transferred out to their databases. Transferring millions of records of sensitive customer data is a deal breaker for most enterprises.

Over the last few years, enterprises are getting more aggressive and demanding that their SaaS vendors provide features that would allow the easy transfer of data back to them. Incumbent vendors are facing increasing threats of being replaced in favor of alternatives who enable mechanisms for seamless data sharing. New RFPs are increasingly making data sharing a critical feature to win the business.

What is Zero-Copy Data Sharing, and Why Is It Important?

Zero-copy data sharing is a mechanism for sharing data between two databases without physically moving or making a copy of the data. This eliminates the error-prone and costly steps of building a pipeline to move data from the source to the target. In many cases, this pipeline is currently as arcane as downloading the data from the source, moving it via secure FTP to the consumer’s environment and loading the data into the target database.

Here are the key distinguising features of a zero-copy data architecture:
No physical data movement: Data remains in its original location, eliminating the need for a separate physical copy.
Elimination of ETL processes: Consumers access data directly as tables, columns, and relationships, rather than handling CSV files.
Zero latency: New data is instantly visible to consumers as soon as it becomes available at the source.
Enhanced data quality: The elimination of process steps and code to extract, format, and transfer data improves quality.
Cost efficiency: Reduces expenses related to storage, coding, and data pipeline operations.
Control over sensitive data: Allows SaaS software to access sensitive data without it leaving the client’s control.

Snowflake pioneered this architecture, using it to power their data marketplace and data cleanrooms. Other cloud databases have followed and built their own capabilities in this space. Many SaaS companies like Salesforce, Simon Data, and (recently) ServiceNow have zero-copy data sharing partnerships with Snowflake. But data sharing across data cloud vendors is still a challenge. And for a SaaS company, it’s expensive to publish data products to support all major cloud data platforms’ proprietary formats.

But in the past year, most cloud database vendors have announced support for the open-source Apache Iceberg table format. By providing a common table format, Iceberg enables seamless data sharing between different cloud platforms and services. This ensures that data can be easily integrated and accessed across various environments and provides an opportunity for SaaS vendors to publish standard schemas in a database vendor-agnostic format that can be shared with the client.

What Does All This Mean for a SaaS Platform Vendor?

In the zero-copy architecture, the data from the SaaS product would appear in the enterprise warehouse, structured as a canonical schema, with complete metadata. In other words, the SaaS vendor is providing their clients with a data product, ready for consumption with no extra investment in coding and infrastructure from the client’s technology teams.

Here are some imperatives for SaaS companies:
Zero-copy data sharing is a must-have: Enterprises are now actively asking SaaS vendors to integrate seamlessly with their analytics platform. Zero-copy data sharing is becoming a critical feature in RFPs.
Revisit your product roadmap: Smart SaaS companies have already adopted this paradigm — or are actively working on it. To defend your market share or to win new customers, put this on your roadmap as a priority.
Strategic decisions are necessary: You have decisions to make. Do you adopt Iceberg and stay data cloud vendor-agnostic, or do you directly support sharing mechanisms provided by a specific data cloud vendor? If one or two data cloud platforms have dominant market share in your industry, then the latter might be a better place to start.
Complexity is inevitable: It is going to be messy as this is all still very new. The control plane to manage data sharing, especially across multiple CSPs and data cloud vendors is still not mature. But this is the future of data integration.

Much like APIs became table stakes for operational system integrations, zero-copy data sharing will soon define successful integration with enterprise data.

Zero-copy data sharing is no longer a nice-to-have — it is an essential feature for SaaS vendors looking to retain existing customers and win new ones.

SaaS vendors must act decisively, investing in data sharing capabilities that align with the needs of their customers. By adopting cutting edge technologies like Iceberg or partnering with leading data cloud platforms, venors can position themselves as critical enablers of enterprise-wide data strategies in the AI era.

---

Mihir is a venture partner with F-Prime Capital and Eight Roads Ventures, and an Advisor-in-Residence at Ernest & Young. He recently retired from Fidelity Investments where he was the CIO responsible for “All Things Data” for the firm. He is currently advising VCs, startups, and large corporations on their data and analytics strategy.