Meta is developing a search engine to power its AI Chatbot

Could Meta build its own search engine, and if it did, what would it look like?

There have been reports of that Meta is looking to build its own search tool for Meta AIto reduce its reliance on Google and Microsoft’s Bing for web searches in its AI chatbot.

There are a few potential angles to the news, but it appears Meta is looking to expand its data collection process to continue building more powerful real-time AI data tools. Which could result in a broader Meta search tool that provides real-time insights into its apps. This is something that could be of particular value to Threads, providing up-to-the-minute notes on emerging news stories to complement in-app engagement.

When Meta first announced its Meta AI chatbot in September last yearexplained how it would use web search results from Bing to help expand its answers:

“In text-based chats, Meta AI has access to real-time information through our search partnership with Bing and offers a tool for image generation.”

When Meta AI Answers uses Bing for information for context, Meta provides a web link to the search query.

Then in April of this year, Meta announced a further partnership with Google to expand its web search answers even further by providing a similar web link on Google-powered answers.

And now according to The informationMeta is looking to build its own web crawler to drive search queries independently to reduce its reliance on its search partners for Meta AI.

As per The information:

“As meta-platforms trying to keep up with OpenAI in the development of artificial intelligence, the Facebook owner is working on a search engine that crawls the web to provide conversational answers about current events to people who use its Meta AI chatbot.”

Conceptually, this would give Meta an alternative if Microsoft or Google, both of which compete with Meta for generative AI development, decide to end their search deals.

But that would also mean that Meta would have to build a complex search system that crawls the web in the same way as both Google and Bing. Which is a big move and why Meta went with these partnerships in the first place.

But could Meta actually make a web crawler that works the same way, and would it be good enough to provide adequate answers in Meta’s chatbot?

Meta already has different web crawlers that collect information from external websites, and those crawlers have been increased activity in recent monthsda Meta is looking to collect more data for its AI projects. They could also already collect data for this new search engine, but a dedicated Facebook search engine would again be a big project.

Although a dedicated search engine is a logical move in the broader sense of information gathering, which would also give Meta more data for its AI language models going forward. Maybe that’s actually the bigger picture here, as opposed to just driving in-stream search.

It’s more likely that Meta supports its data sources to power its AI tools and uses its existing crawlers to scrape whatever data they can from third-party providers that haven’t updated their robots.txt protection. That would help Meta build on its already large data stores, and if it already collects that information, it also makes sense to reduce its reliance on third-party search providers where it can.

So perhaps it’s probably less about building a competitor to these providers and more about making the most of your own data collection processes. But still, it’s a significant undertaking that could provide expanded opportunities for Zuck and Co. if they can get it right.

In reality, though, it’s not an unexpected move given the competition in the space. Again, Meta is battling these two providers for AI supremacy, and as the race continues to heat up, it will come as no surprise if they stop helping Meta in this regard.