Mastering Mistral AI: A Comprehensive Guide for Building Intelligent Applications

This chapter provides a comprehensive introduction to Mistral AI and its powerful suite of tools for building intelligent applications. From basic chat completions to advanced techniques like Retrieval Augmented Generation (RAG) and function calling, this guide will equip you with the essential knowledge and practical skills to leverage Mistral AI’s capabilities. This course, created by Per Borgan from Scrimba in collaboration with Mistral AI, offers hands-on experience with Mistral’s open-source and commercial models, empowering you to master AI engineering paradigms and create sophisticated conversational user experiences. You will even learn to run AI models locally on your own computer.

1. Introduction to Mistral AI

Welcome to the world of Mistral AI! This course is designed to teach you how to build “magical stuff” using JavaScript and Mistral AI.

1.1 What is Mistral AI?

Mistral AI is a company specializing in building foundational models.

Foundational Models: Large AI models trained on vast datasets, capable of performing a wide range of tasks and serving as the base for more specialized applications.

In 2023, Mistral AI gained significant recognition within the AI community by launching small, open-source foundational models that rivaled the performance of the best closed-source models available. This achievement underscores Mistral AI’s commitment to developing powerful yet accessible AI technologies.

1.2 Why Mistral AI Matters for AI Engineers

As an AI engineer, Mistral AI is a company you should definitely pay attention to. Their focus on creating high-performing, open-source models makes them a valuable resource for developers and researchers alike. This course will guide you through understanding and utilizing Mistral AI’s platform and tools effectively.

2. Exploring the Mistral AI Platform and Models

This section provides an overview of the Mistral AI platform and the diverse range of models they offer.

2.1 Platform Overview

We will begin by exploring the Mistral AI platform itself before delving into the practical aspects of using their API and JavaScript SDK.

API (Application Programming Interface): A set of rules and specifications that software programs can follow to communicate with each other. In the context of AI, APIs allow developers to access and utilize AI models and services.

SDK (Software Development Kit): A collection of software development tools in one installable package. For AI, SDKs often include libraries, code samples, documentation, and guides to facilitate the integration of AI models into applications.

While this course focuses on JavaScript, it’s important to note that Mistral AI also offers a Python SDK with similar functionalities, ensuring the course’s value extends to Python developers as well.

2.2 Mistral AI Models: A Comprehensive Range

Mistral AI offers a variety of models to cater to different needs and use cases. These include:

• Open-Source Models:

  • Mistral 7B: Mistral AI’s initial open-source model, boasting impressive performance despite its smaller size (7 billion parameters).

  • Mistral 8x7B (Mixtral 8x7B): A more advanced open-source model with a unique Mixture of Experts architecture.

  Mixture of Experts (MoE): An architecture in neural networks where multiple specialized sub-networks (experts) are used to process different parts of the input, leading to increased model capacity and efficiency.

  These models are released under the open-source Apache 2.0 license, making them freely available for experimentation and use.

• Commercial Models (Optimized Enterprise-Grade):

  • Mistral Tiny: Designed for low-latency use cases, prioritizing speed and efficiency.
  • Mistral Small: Optimized for language-based tasks requiring a balance of performance and cost.
  • Mistral Medium: Suitable for more sophisticated language understanding and generation tasks.
  • Mistral Large: Mistral’s most powerful model, designed for complex and demanding AI applications requiring advanced reasoning.

• Embedding Model:

  • Mistral AI also provides a state-of-the-art embedding model for generating text embeddings, crucial for working with vector databases.

  Vector Databases: A type of database specifically designed to store and efficiently query vector embeddings, enabling similarity searches and semantic understanding of data.

2.3 Model Selection and Use Cases

Choosing the right model depends on your specific application requirements. Mistral AI’s commercial models generally offer higher performance compared to their open-source counterparts. However, cost and latency considerations also play a crucial role.

• Mistral Small: Ideal for tasks like classification and customer support.
• Mistral Medium: Suitable for intermediate tasks such as data extraction, summarization, and content creation (e.g., job descriptions).
• Mistral Large: The go-to model for complex tasks demanding significant reasoning capabilities, such as building AI agents and implementing RAG.

3. API Basics and JavaScript SDK

This section dives into the practical aspects of interacting with the Mistral AI API using their JavaScript SDK.

3.1 Setting up API Access

To interact with the Mistral API, you will need an API key.

API Key: A unique identifier used to authenticate requests to an API, verifying the user or application making the request and granting access to the API’s functionalities.

You can obtain an API key through the Mistral AI platform (“La Platform”). The process involves:

1. Navigating to the Mistral AI homepage and clicking “Build Now.”
2. Authenticating using your preferred method.
3. Creating a workspace and specifying whether you are a solo creator or part of a team.
4. Adding a payment method (card) to enable API access. Mistral AI uses a pay-as-you-go pricing model, meaning you only pay for the resources you consume.
5. Creating a new API key with a name and expiration date. Crucially, save your API key securely as it will only be shown once.

3.2 Using the Chat Completion API

The Chat Completion API is a fundamental tool for building conversational AI applications.

Chat Completion API: An API endpoint that allows you to send conversational prompts to an AI model and receive generated text completions, suitable for building chatbots and conversational interfaces.

The Mistral AI JavaScript SDK simplifies interaction with this API.

• Installation and Setup: Install the Mistral AI JavaScript client and instantiate a Mistral AI client using your API key, storing it securely as an environment variable.

• Basic Chat Request:

  • Utilize the client.chat method to send chat requests.
  • The request body requires two essential parameters:
    • model: Specify the Mistral model to use (e.g., “mistral-tiny”).
    • messages: An array of message objects representing the conversation history. Each message object includes:
      • role: Indicates the message sender (“user” for user input, “system” for system prompts).
      • content: The actual text message.

• Temperature Parameter: Control the creativity and randomness of the generated text using the temperature parameter (value between 0 and 1).

  • Higher temperature (closer to 1): More random and creative responses.
  • Lower temperature (closer to 0): More focused and deterministic responses.

• System Prompts: Guide the model’s behavior using system messages. Set the role to “system” and content to provide instructions or prompts to the model. System prompts are placed before user messages in the messages array.

3.3 Streaming Responses

For a more interactive user experience, you can implement streaming of responses.

Streaming: A technique where data is transmitted in a continuous flow, allowing for immediate processing and display of information as it becomes available, rather than waiting for the entire dataset to be transmitted. In AI chat applications, streaming enables token-by-token display of the AI’s response, improving perceived responsiveness.

• chatStream Method: Use client.chatStream instead of client.chat.

• Async Iteration: The chatStream method returns an async iterable.

  Async Iterable: An object that can be asynchronously iterated over using for await...of loops in JavaScript, yielding values one at a time as they become available.

• Token-by-Token Processing: Use an async for...of loop to iterate over the response chunks. Access the message content within each chunk using chunk.choices[0].delta.content.

3.4 JSON Response Format

Mistral AI API allows you to request responses in JSON (JavaScript Object Notation) format.

JSON (JavaScript Object Notation): A lightweight data-interchange format that is easy for humans to read and write and easy for machines to parse and generate. It is commonly used for transmitting data in web applications.

• response_format Parameter: Set the response_format parameter in the chat request to { type: "json_object" }.
• Prompt Instruction: Include an instruction in your prompt to indicate that you expect a JSON response (e.g., “Reply with JSON”).

This feature is particularly useful for integrating AI responses directly into applications, as JSON format facilitates easy parsing and data extraction.

4. Advanced AI Engineering Paradigms

This section explores advanced AI engineering techniques that you can implement with Mistral AI.

4.1 Retrieval Augmented Generation (RAG)

Retrieval Augmented Generation (RAG) is a powerful technique to enhance AI models with domain-specific knowledge or information they were not trained on.

Retrieval Augmented Generation (RAG): An AI framework that combines information retrieval with text generation. It involves fetching relevant information from a knowledge source (retrieval) and then using that information as context to generate more informed and accurate responses (generation).

RAG is particularly useful for:

• Providing AI applications with access to proprietary company data.
• Incorporating real-time information.
• Enabling in-depth knowledge on niche subjects.

RAG Process:

1. Retrieval: Fetch relevant data based on the user’s query. This is often done using a vector database.

   Semantic Search: A search technique that aims to understand the meaning and context of search queries and documents, rather than just relying on keyword matching. Vector embeddings and similarity search in vector databases enable semantic search.

   • Vector Database and Embeddings: Data is converted into embeddings (numerical representations of text) and stored in a vector database. User queries are also converted into embeddings. Semantic search in the vector database identifies embeddings similar to the query embedding, retrieving relevant data.

     Embeddings: Vector representations of text or other data, capturing semantic meaning and relationships in a numerical format that AI models can understand and process.

2. Generation: Combine the retrieved information with the user’s query to create a comprehensive prompt for the LLM (Large Language Model).

   LLM (Large Language Model): A type of AI model, often based on deep learning architectures, trained on massive amounts of text data to understand and generate human-like text. Mistral AI models are LLMs.

   • The LLM uses the retrieved context to generate a human-readable response, often including references or footnotes to the source data, enabling fact-checking.

Understanding Embeddings in Detail:

• Embeddings are generated by passing text (e.g., “hello world”) through an AI model.

• The output is a vector – a long array of numbers.

• Mathematically similar vectors represent semantically similar text.

• Vector Space: Embeddings exist in a high-dimensional vector space, where the proximity of vectors reflects semantic similarity.

  Vector Space: A mathematical space where vectors are located, and relationships between vectors (like distance and direction) can be measured and interpreted. In the context of embeddings, vector space represents the semantic relationships between words, phrases, or documents.

• Example: Word Relationships in Vector Space:

  • “Cat” and “Feline” vectors are close.
  • ”Kitten” vector is near “Cat” and “Feline” but slightly further.
  • ”Dog” vector is in a different region but still within the “domesticated animals” domain.
  • ”Building” vector is far away, representing a semantically unrelated concept.
  • Vector Arithmetic: Embeddings enable mathematical operations to explore semantic relationships (e.g., Vector(“King”) - Vector(“Man”) + Vector(“Woman”) ≈ Vector(“Queen”)).

• Real-World Applications of Embeddings:

  • Search Engines: Semantic search beyond keyword matching.
  • Recommendation Systems: Personalized suggestions (e.g., Netflix, Spotify, Amazon).
  • Healthcare: Medical image analysis.
  • Finance: Financial data analysis and prediction.

4.2 Text Splitting for Embeddings

When working with large documents for RAG, it’s crucial to split the text into smaller, meaningful chunks before generating embeddings.

• Importance of Chunking: Embedding entire large documents results in broad, less semantically specific embeddings, hindering effective semantic search.

• Langchain Text Splitter: Utilize libraries like Langchain for efficient and intelligent text splitting.

  Langchain: A popular framework for developing applications powered by language models. It provides tools and components for various tasks, including prompt management, chains of operations, data augmentation, and agent creation.

  • Recursive Character Text Splitter: Langchain’s RecursiveCharacterTextSplitter is recommended for its ability to split text recursively based on characters and separators while considering sentence boundaries.
  • Chunk Size and Overlap: Configure chunk size (e.g., 250 characters) and overlap (e.g., 40 characters) to balance semantic specificity and contextual awareness.
  • Chunk Content Cohesion: Aim for chunks that focus on a single subject or theme to create high-quality embeddings. Avoid chunks that are “polluted” with multiple themes.
  • Experimentation: Finding the optimal chunk size and overlap often requires experimentation to achieve the best results for your specific data and application.

4.3 Creating Embeddings with Mistral API

Mistral AI provides an embedding model accessible via their API to generate text embeddings.

• client.embeddings Method: Use the client.embeddings method from the Mistral AI JavaScript SDK.
• Model Parameter: Specify the embedding model: "mistral-embed".
• Input Parameter: Pass an array of strings (text chunks) as the input parameter.
• Response Structure: The API response contains an array of embedding vectors, each corresponding to an input text chunk.

4.4 Vector Databases and Superbase

Vector databases are essential for efficiently storing, indexing, and querying vector embeddings in RAG applications.

Vector Database (Vector Store): A specialized type of database optimized for storing and querying vector embeddings. It uses similarity metrics to efficiently find vectors that are semantically similar to a query vector.

• Superbase: A full-fledged open-source backend platform that includes a PostgreSQL database with the PGVector extension, making it suitable for vector database functionalities.

  PGVector: A PostgreSQL extension that adds support for storing and querying vector embeddings within a traditional relational database.

• Superbase Setup:

  1. Create a Superbase account and a new project.
  2. Enable the PGVector extension in your project’s database settings.
  3. Obtain your Superbase project URL and API key from the project settings.
  4. Set these as environment variables in your development environment.

• Creating a Vector Table in Superbase: Use SQL to create a table in Superbase to store your text chunks and their corresponding embeddings. Define columns for:

  • id (primary key, big serial)
  • content (text, plain text)
  • embedding (vector, vector[dimensions of your embedding model])

• Inserting Data into Superbase: Use the Superbase JavaScript SDK to insert data into your vector table. Structure your data as an array of objects, each containing content (text chunk) and embedding (vector).

• Vector Similarity Search in Superbase: Superbase with PGVector enables efficient vector similarity searches using SQL functions.

  • match_handbook_docs Function (Example): Create a SQL function (using Superbase’s SQL editor) that takes a query embedding as input and performs a similarity search against the embedding column in your vector table. It returns the most similar documents based on a defined match threshold and number of matches.
  • Remote Procedure Call (RPC): Use Superbase’s rpc method in your JavaScript code to call the match_handbook_docs function and perform similarity searches from your application.

4.5 Building a RAG Application: End-to-End Flow

Putting it all together, a RAG application using Mistral AI and Superbase involves the following steps:

1. User Input: Obtain the user’s query (e.g., “Do I get an extra day off if Christmas is on Sunday?”).
2. Query Embedding: Generate an embedding of the user’s query using Mistral AI’s embedding model.
3. Retrieve Matches (Context): Use Superbase’s RPC method to call the match_handbook_docs function, performing a similarity search in your vector database using the query embedding. Retrieve relevant text chunks (context) from Superbase.
4. Generate Chat Response: Combine the retrieved context and the user’s original query into a prompt. Send this prompt to Mistral AI’s Chat Completion API (e.g., using mistral-large-latest model).
5. Display Response: Present the AI-generated response to the user.

5. AI Agents and Function Calling

Function calling empowers AI models to interact with external tools and APIs, creating intelligent AI agents.

Function Calling: A capability of AI models that allows them to identify when a function call is needed to fulfill a user’s request and to generate the necessary function call parameters. This enables AI models to interact with external tools and APIs.

AI Agent: A software entity that can perceive its environment, reason, and take actions to achieve specific goals. In the context of function calling, AI agents can use external tools and functions to perform actions based on user requests.

5.1 Function Calling Architecture

1. User Query: User interacts with the AI agent (e.g., “Is my package on its way?”).
2. LLM with Tools: The user query is sent to the LLM along with a description of available tools (functions).
3. Function Call Instruction: The LLM analyzes the query and, if necessary, determines that a function call is required. It instructs the application to call a specific function with specific parameters (e.g., “call fetch_order function with order_id”).
4. Function Execution: The application executes the function call, retrieving data from an external source (e.g., order database).
5. Data Return: The function returns the retrieved data to the LLM.
6. Response Generation: The LLM uses the retrieved data to generate a human-readable response for the user (e.g., “Yes, your order is expected to arrive…“).

5.2 Implementing Function Calling with Mistral AI

• Tools Definition: Define functions as JavaScript functions (e.g., getPaymentStatus, getPaymentDate). Create a tools.js file to store these functions.

• Tools Schema: Describe each function in a structured format (JSON-like object) within a tools array. This schema includes:

  • type: “function”
  • function: An object containing:
    • name: Function name (string).
    • description: Function description (string).
    • parameters: Function parameters (object) defining parameter properties (e.g., type, description, required).

• Chat Request with Tools: When making a chat request to Mistral AI API, include the tools array in the request body.

• Tool Call Response: If the model determines a function call is needed, the API response will indicate this through the finish_reason being set to "tool_calls" and will include a tool_calls array in the response. This array contains instructions on which function to call (function.name) and with what arguments (function.arguments).

• Function Invocation: In your application code, check for the "tool_calls" finish reason. If present:

  1. Extract the function name and arguments from the API response.
  2. Use the function name to dynamically invoke the corresponding function from your availableFunctions object (e.g., using bracket notation: availableFunctions[functionName](functionArgs)).
  3. Obtain the function response (data).

• Sending Tool Response Back to Model: Push a new message to the messages array with the following:

  • role: "tool"
  • tool_call_id: The ID of the tool call from the API response.
  • name: Function name.
  • content: The function response (data) as a string.

• Looping for Multi-Turn Function Calls: Wrap the entire process in a loop (e.g., for loop with a limited number of iterations) to handle scenarios where the model might require multiple function calls to fulfill the user’s request. The loop continues until the finish_reason in the API response is "stop", indicating the model has a final response for the user.

6. Running Mistral Models Locally with Ollama

Ollama is a tool that simplifies running large language models locally on your computer.

Ollama: An application that allows you to easily download, manage, and run large language models locally on your computer, providing a user-friendly interface for interacting with these models offline.

6.1 Setting up Ollama

1. Download and install Ollama from ollama.com.
2. Open your terminal and run ollama run mistral. This command will download and run the Mistral model locally.

6.2 Interacting with Mistral via Ollama in the Terminal

Once Ollama is running, you can interact with the Mistral model directly in your terminal by typing messages after the >>> prompt.

6.3 Using Ollama with JavaScript (Example Project)

You can integrate Ollama with your JavaScript projects to run Mistral models locally within your applications.

• Ollama SDK: Use the Ollama SDK (or similar libraries) in your JavaScript code.

• API Resemblance: The Ollama SDK API often resembles the Mistral AI API, making the transition relatively straightforward.

• Local Inference: Perform inference (generating responses) locally on your computer, eliminating reliance on cloud-based APIs and ensuring data privacy.

  Inference: The process of using a trained AI model to generate predictions or outputs on new, unseen data. In the context of LLMs, inference is the process of generating text responses based on user prompts.

• Example Express.js Project: A simple Express.js project can demonstrate how to interact with Ollama programmatically. The project would:

  1. Set up an Express router to handle requests.
  2. Use the Ollama SDK to send chat requests to the local Mistral model.
  3. Extract the model’s response and send it back to the client (e.g., display it in the browser).

7. Conclusion and Next Steps

Congratulations on completing this comprehensive guide to Mistral AI! You have gained a solid understanding of:

• Mistral AI platform and models.
• Chat Completion API and JavaScript SDK.
• Advanced AI techniques: RAG and function calling.
• Local model execution with Ollama.

This course provides a strong foundation for building innovative AI applications. Continue experimenting, building, and exploring the rapidly evolving world of AI. Share your achievements, connect with the AI community, and keep learning to unlock the full potential of AI engineering! Happy building!