EP. 234 How to Choose the RIGHT Vector Database?

Jesse Hall hosts Vijay and Pankaj from Airwave on MongoDB TV, discussing Airwave's innovative use of AI and MongoDB technology to empower field technicians working on heavy industrial machinery. Pankaj, CEO, and Vijay, CTO, emphasize AI's role in transforming technicians into experts by enabling them to access and understand complex service manuals efficiently. They detail the challenges of processing vast amounts of technical documentation and the integration of hybrid search capabilities to improve accuracy and retrieval. The conversation highlights Airwave's impact on reducing errors and enhancing productivity, with a focus on future advancements in schematic interpretation and international language support.

• Airwave is a platform designed for technicians working on heavy industrial machinery.
• The platform leverages AI to empower technicians by turning them into experts.
• MongoDB technology is integral to the functioning of Airwave.

"Airwave is built specifically for technicians out in the field that work on heavy industrial machinery."

• Airwave's primary goal is to improve the efficiency and expertise of field technicians.

• AI offers significant benefits to frontline workers by providing them with necessary information and support.
• Technicians often encounter machinery they are unfamiliar with, and AI helps them quickly understand and resolve issues.
• AI is a tool that enhances human capability rather than replacing it.

"There is tremendous interest in what AI can bring to the table in terms of helping these folks do their jobs better."

• AI assists technicians in diagnosing and fixing problems by providing instant access to relevant information.

• The onboarding process for new customers involves processing thousands of documents, which is simplified by AI.
• AI makes it feasible to handle vast amounts of information that would be overwhelming manually.

"With this support of AI, I think it's making their life easier."

• AI facilitates the management and retrieval of information from extensive documentation, enhancing the work of service technicians.

• Airwave's concept originated from the need to capture and transfer knowledge from experienced technicians to newcomers.
• The initial application was an asynchronous communication tool that evolved with the integration of AI capabilities.

"The use case came from actually one of our early customers... they wanted to make sure that they had access to him."

• The transition from a communication tool to a comprehensive AI-powered platform was driven by customer needs and technological advancements.

• The team faced challenges when tasked with processing 5000 service manuals.
• Initial attempts to use AI models for document processing were cost-prohibitive.

"I fine-tuned... it costed us like $90 actually."

• The solution involved building a robust infrastructure using a combination of technologies, including a vector database and OpenAI.

• The team implemented a Retrieval-Augmented Generation (RAG) infrastructure to handle large volumes of documents.
• This infrastructure allowed efficient processing and retrieval of information from service manuals.

"We started to put together this entire RAG infrastructure with whatever available at the time."

• The choice of vector databases and the integration of AI models were crucial to the platform's success.

• AI significantly reduces the time and effort required for technicians to find information.
• The platform's ability to process and retrieve data from thousands of documents is transformative for the industry.

"The customer was actually pretty excited because it was our first time that... we need a huge infrastructure to deal with actually."

• Airwave's AI-driven approach exemplifies how technology can address complex challenges in industrial settings.

• Extracting information from scanned documents, especially old manuals, poses significant challenges due to the format and structure of the documents.
• Service technicians require precise information, like part numbers, from these documents, making accuracy critical.
• The process involves dealing with tables, schematics, and wiring diagrams, which require specialized handling.

"When we built the rack platform, we had a basic text scrapping from PDF, and then that was actually, we thought, okay, it's going to be okay. But that wasn't actually enough."

• Initial attempts at text extraction were inadequate, highlighting the complexity of the problem.

"We have a huge team around the computer vision area itself to deal with it."

• A dedicated team is necessary to tackle the challenges of extracting information from complex documents.

• The task involves correlating text data, like part numbers, with image data, such as schematics, to ensure accurate retrieval of information.
• The complexity increases with documents that cover multiple serial numbers or models within a single manual.

"You have to make sure that you have the right part number. You have to also make sure you pull in the right schematic for that part number."

• Ensuring the correct association between text and image data is crucial for accurate information retrieval.

"If you ask a question about, hey, can you give me some information about this particular model number for this particular machine? We have to make sure that we actually use the page number five, information across to get that information."

• The process requires navigating through various sections of a document to gather relevant information.

• Years of experience in handling millions of pages of manuals contribute to the expertise required for effective information extraction.
• The ability to quickly provide comprehensive information, including manuals, part numbers, and schematics, is a significant advantage.

"No one does this incredibly well. Like, yeah, but years of experience over years of doing it, you kind of remember all these things or you've seen it before."

• Experience is a key factor in efficiently managing complex document information.

"Here's the manual, the installation manual. Here's the table that has the part number, price, even availability. And here's the schematic of what it all looks like."

• The integration of various data points into a cohesive output is a major achievement.

• Managing large volumes of documents and scaling vector databases are significant challenges.
• Performance optimization is crucial, especially when dealing with large datasets and ensuring quick retrieval times.

"I have a folder with 100 gigs. Yeah, how long will this take?"

• The sheer volume of data poses logistical challenges in processing and retrieval.

"We actually like, want to like go one step at a time, actually. So I think the scaling of vectors is one of the challenge that we face apart from, you know, retrieval."

• Scaling and retrieval are ongoing challenges that require careful management and optimization.

• The initial proof of concept faced issues with retrieving correct part numbers due to limitations in the search method.
• Transitioning to a hybrid search approach, combining keyword and semantic search, improved accuracy significantly.

"The initial proof of concept that we did was not great at retrieving the right part number."

• Initial solutions were inadequate, necessitating a change in approach.

"We wanted something which does both keyword search and also semantic search. So that's when we decided, okay, fortunately, Mongo was actually like rolling up the, rolling out the vector support for databases."

• The hybrid search approach allowed for more accurate retrieval by leveraging both keyword and semantic searches.

"Having the hybrid search actually made a huge difference in terms of accuracy, actually."

• The shift to hybrid search was a pivotal moment in improving the system's accuracy and reliability.

• Integration of operational and vector databases is crucial to reduce latency and improve efficiency.
• Combining full-text keyword search with vector search allows for hybrid search capabilities.
• MongoDB's ability to integrate these searches is a key differentiator in handling specific data types like part numbers.

"You have your operational database over here and your vector database over here. Obviously there's going to be latency there. So having that in one spot... And recently adding vector search, now you have that hybrid search when you combine the two."

• The quote highlights the importance of integrating operational and vector databases to minimize latency and enhance search capabilities.

• Semantic search is not suitable for exact number lookups, such as part numbers.
• Vector search can fail in scenarios requiring precise information, necessitating alternative approaches.
• Continuous learning and adaptation are required to address these challenges.

"We didn't even know, like, okay, we will have issues with like semantic search, especially with the part numbers. Then after digging into it, then we learned, okay, this is why it actually fails, because vector search isn't designed for like lookups, like exact number lookups."

• This quote explains the limitations of vector search in handling exact data, emphasizing the need for continuous learning and adaptation.

• Accuracy in AI systems is a work in progress and can be improved in various ways.
• Different use cases require tailored methods to enhance accuracy.
• The process involves proving the system's reliability to users.

"It hallucinates quite a bit. Right? But, you know, look how far we've come. Like, it's getting better and better and better. And that's one of the main things. Like everyone, you have to prove it that it's going to be accurate."

• The quote discusses the evolving nature of AI accuracy and the importance of demonstrating reliability to users.

• The hybrid search approach involves using both keyword and vector search simultaneously.
• A preprocessing step is employed to clean and format user queries for better matching with the database.
• This approach minimizes the risk of incorrect information retrieval.

"We have like a massaging system where it actually, like, cleans up the question and then creates, you know, multiple formats of the question that actually matches with what we have in our rag infrastructure."

• This quote describes the preprocessing system used to optimize query matching and enhance search accuracy.

• Simplifying user queries is essential for effective information retrieval.
• The system predicts and suggests potential questions based on minimal input.
• This approach reduces the need for users to be prompt engineers.

"We spent a lot of time trying to, like, okay, well, can we take what they've given us and figure out the question that they're trying to ask in this very specific context?"

• The quote highlights the effort to simplify user queries and improve the system's ability to predict user intent.

• OpenAI models are used for their efficiency and speed in processing queries.
• The system remains flexible to accommodate customer preferences and advancements in AI technology.

"We use all OpenAI. OpenAI for? So we use this standard because it just works pretty well and pretty fast, actually, recently."

• This quote underscores the reliance on OpenAI models for efficient query processing and the system's adaptability to new technologies.

• Technicians share answers and solutions within a communication platform to enhance collective knowledge.
• This collaborative approach reduces the need for backend caching systems.

"They can actually share that answer with the tech, like in a channel within airwave. So now everyone has access to that."

• The quote illustrates the collaborative sharing of information among technicians to improve problem-solving and reduce system load.

• A feedback system is in place to gather user reactions and improve the system.
• Dislikes are analyzed to identify and rectify inaccuracies in the information provided.
• The feedback system is a core metric for system enhancement.

"When our system, rack system respond with the information that they don't like, they will dislike it. And that is our, one of the core usage, core, core metric that we use for analyzing what is going on with the rack system."

• This quote highlights the importance of user feedback in continuously improving the system's accuracy and reliability.

• Documentation serves as a starting point, but real-world conditions often deviate from original specifications.
• The system must adapt to changes and modifications made over time to machinery and equipment.

"When a machine has been operating for 20 years, 24 hours a day, the factory spec that it came with is usually not no longer the norm."

• The quote emphasizes the need for the system to adapt to the realities of long-term equipment use and modifications.

• Airwave captures institutional knowledge from technicians, allowing them to access collective insights beyond manuals and schematics.
• The platform enhances decision-making by providing comprehensive information from past experiences and comments made by previous technicians.

"So if some technician goes out there and is like, look, the manual says connect the seven s five to the seven x nine, but make sure you don't jump the orange cable or you let the capacitor drain before you actually connect it, that institutional knowledge also gets captured in airwave."

• This quote illustrates how Airwave captures detailed and crucial procedural knowledge that may not be explicitly stated in manuals, ensuring technicians have access to vital information for accurate problem-solving.

• Airwave's development is driven by customer and user needs, with a focus on making complex schematics more understandable.
• The engineering team consists of 15 remote members, emphasizing a global and distributed work environment.
• Airwave is used exclusively for internal communication, eliminating the need for email, Slack, or Teams, and promoting asynchronous audio communication.

"We are very customer and user driven. So I try to get out in a service truck once a month to go help folks fix the things that they're fixing."

• This quote highlights the customer-centric approach in Airwave's development, emphasizing real-world testing and feedback to enhance the platform's utility.

"Yeah, and actually it's been really great because we don't have a lot of meetings because we just use asynchronous audio and it gives it a really personal feel."

• The effectiveness of asynchronous audio communication is captured here, indicating how it reduces the need for meetings while maintaining a personal touch.

• Airwave is continuously seeking talent, particularly individuals with expertise in mechanics, electronics, and coding.
• The company values experience in computer vision, Python, and knowledge of sentence transformer models and LLMs for information retrieval tasks.

"We are always looking for talented people, particularly in the intersection between. Do you know a lot about mechanics? Do you know a lot about electric? And can you also code like those people are."

• This quote underscores the company's need for multidisciplinary talent to tackle complex technological challenges.

"So we're looking for people who worked in computer vision, side with experience in Python, and then if they have exposure to sentence transformer models or any LLMs, especially with information retrieval."

• The specific skill set required for potential recruits is detailed here, highlighting the focus on advanced technologies and applications.

• Airwave's LLM-based translation capabilities enable technicians to ask questions in their native language and receive answers from English manuals, facilitating cross-language communication.
• This feature is particularly beneficial for training technicians in foreign countries and overcoming language barriers.

"What it can actually do is when you ask a question in, like, Farsi, it will transcribe it in Farsi, it will go find the answer in an English manual and then transcribe it back in Farsi."

• This quote emphasizes the transformative impact of Airwave's translation feature, allowing seamless communication and learning across different languages.

"So the one thing that we are fortunate enough is like, you know, this translation works pretty well because their questions are usually, like, very specific about a machine or a part number."

• The effectiveness of translation for technical queries is highlighted, demonstrating how Airwave manages to bridge language gaps effectively.

• Airwave helps minimize risks associated with technicians experimenting due to a lack of knowledge by providing reliable information quickly.
• Preventing costly mistakes is a significant benefit, as technicians can find answers without resorting to potentially damaging trial-and-error methods.

"And so just limiting that, like, don't try it unless you actually ask. Jarvis has been pretty transformational for our customers."

• This quote reflects how Airwave's information accessibility reduces the likelihood of costly errors, promoting safer operational practices.

• The podcast concludes with acknowledgments to the team behind Airwave and gratitude for the support from viewers and listeners.
• Emphasis is placed on the satisfaction derived from seeing the product effectively solve real-world problems.

"I just want to say all the credit goes to Vijay and his team. I get to go out and find really hard problems and then I'll just message Vijay on airwave and be like, hey, can you do it?"

• This quote acknowledges the collaborative effort and problem-solving capabilities of the Airwave team, highlighting their role in the platform's success.