Learn how to create a conversational AI chatbot locally using Microsoft .NET and Ollama

This tutorial guides you through the process from setup to running your own AI assistant.

 

Artificial intelligence is rapidly changing the tech landscape, and now you can harness its power without relying on cloud-based services.

This guide will walk you through building a local AI chatbot using .NET and Ollama. We’ll leverage the Microsoft.Extensions.AI library for an abstraction layer, allowing you to easily swap out underlying AI models with minimal code changes.

This project is perfect for technology professionals, software engineers, IT managers, and students looking to understand and experiment with local AI development.

What You'll Learn

1. Setting up Ollama locally
2. Creating a .NET console application using the Microsoft.Extensions.AI library
3. Connecting to a local AI model (phi3:mini)
4. Building a basic conversational interface
5. Understanding AI abstractions and their benefits
6. Implementing chat history for improved interactions

Lesson 1: Setting Up Ollama - Your Local AI Engine

Let’s start by installing Ollama, a tool that allows you to run various AI models locally. Ollama simplifies the process of downloading and running these models on your machine.

Prerequisites

• .NET 8.0 or higher installed on your system. You can download it from https://dotnet.microsoft.com/en-us/download.
• Ollama Installed Locally. Follow the instructions on the Ollama website https://ollama.com to install it for your operating system (Windows, macOS, or Linux). 
• Visual Studio Code (Optional). While not strictly required, VS Code with the C# extension is a great development environment. 

Steps

1. Verify Ollama Installation: Open a terminal window (Command Prompt on Windows, Terminal on macOS/Linux) and type: "ollama". If Ollama is installed correctly, you’ll see a list of available commands. 
2. Start Ollama: Run the following command to start the Ollama server: "ollama serve" This will initiate the Ollama server, which will handle running your chosen AI model. You'll see output indicating that it's running.
3. Pull the "phi3:mini" Model: This is the AI model we’ll use for this tutorial. Run the following command to download it: "ollama pull phi3:mini ". This will download the model, which can take some time depending on your internet connection.
4. Run the Model: Once the download is complete, start the "phi3:mini" model: type the following command: "ollama run phi3:mini". Ollama will start the model and provide a prompt in the terminal for you to interact with it. You’ll see something like: "Welcome to phi3:mini! Type your prompt and press Enter. "

Common Beginner Mistakes & How to Avoid Them

• Ollama Not Found: Ensure Ollama is correctly installed and added to your system's PATH environment variable. 
• Port Conflicts: Ollama typically runs on port 11434. If another application is using this port, Ollama won’t start. Try stopping the conflicting application or changing Ollama’s port configuration (refer to the Ollama documentation).

Lesson 2: Creating the .NET Console Application

Now that we have Ollama running locally, let's build a .NET console application that interacts with the "phi3:mini" model. 

Steps

1. Create a New Project. Open a terminal window and navigate to an empty directory on your device. Create a new .NET console application using the following command: "bash dotnet new console -o LocalAI". This will create a folder named `LocalAI` containing the project files.
2. Change Directory. Navigate into the newly created folder: "cd LocalAI"
3. Add the OllamaSharp Package. Install the "OllamaSharp" NuGet package to your project. This package provides the necessary abstractions for interacting with Ollama. "dotnet add package OllamaSharp"
4. Open in Editor. Open the "Program.cs" file in your chosen code editor (e.g., Visual Studio Code).
5. Replace Code. Replace the contents of "Program.cs" with the following C# code:

         using Microsoft.Extensions.AI; 

         using OllamaSharp;

         IChatClient chatClient =

             new OllamaApiClient(new Uri("http://localhost:11434/"), "phi3:mini"); 

             // Start the conversation with context for the AI model

             List chatHistory = new();

             while (true)

             {

             // Get user prompt and add to chat history 

             Console.WriteLine("Your prompt:"); 

             var userPrompt = Console.ReadLine();

             chatHistory.Add(new ChatMessage(ChatRole.User, userPrompt));

             // Stream the AI response and add to chat history 

             Console.WriteLine("AI Response:"); 

             var response = ""; 

             await foreach (ChatResponseUpdate item in

                 chatClient.GetStreamingResponseAsync(chatHistory)) 

             { 

                 Console.Write(item.Text);

                 response += item.Text; 

             }

    chatHistory.Add(new ChatMessage(ChatRole.Assistant, response));

    Console.WriteLine();

}

Common Beginner Mistakes & How to Avoid Them

• NuGet Package Not Found: Ensure you have the internet connection and that the "OllamaSharp" package is available on NuGet. Try cleaning and rebuilding your solution.
• Incorrect Uri: Double-check that the URI in the "OllamaApiClient" constructor is correct (`http://localhost:11434/`). This is the default port Ollama uses.

Lesson 3: Running and Interacting with Your Chatbot

Now, let's run your .NET application and start chatting with the local AI model!

Steps

1. Run the Application: In your terminal window, run the following command to build and run the application: "dotnet run" 
2. Interact with the AI: The application will start, and you’ll see the prompt: "Your prompt:". Type your question or statement into the console and press Enter. The AI model ("phi3:mini") will generate a response, which will be displayed in the console.

Example Interaction:

Your prompt:

Tell me three facts about .NET. 

AI Response:

1. **Cross-Platform Development:** One of the significant strengths of .NET, particularly its newer iterations (.NET Core and .NET 5+), is cross-platform support. It allows developers to build applications that run on Windows, Linux, macOS, and various other operating systems seamlessly, enhancing flexibility and reducing barriers for a wider range of users.

2. **Rich Ecosystem and Library Support:** .NET has a rich ecosystem, comprising an extensive collection of libraries (such as those provided by the official NuGet Package Manager), tools, and services. This allows developers to work on web applications (.NET for desktop apps and ASP.NET Core for modern web applications), mobile applications (.NET MAUI), IoT solutions, AI/ML projects, and much more with a vast array of prebuilt components available at their disposal.

3. **Type Safety & Reliability:** .NET's CLI model enforces strong typing and automatic garbage collection, mitigating many runtime errors that are common in languages like C/C++. It also enables features such as garbage collection, thus relieving developers from manual memory management. These characteristics enhance the reliability of .NET-developed software and improve productivity by catching
issues early during development.

Key Concepts

• Abstraction: The "OllamaApiClient" provides an abstraction layer, allowing you to switch AI models without changing the core logic of your application.
• Streaming Response: The "GetStreamingResponseAsync" method allows the AI model to send responses incrementally, improving the user experience.
• Chat History: The "chatHistory" list stores the conversation's context, allowing the AI to maintain a coherent dialogue.

Conclusion

Congratulations! You’ve successfully built a local AI chatbot using .NET and Ollama. This project demonstrates the power of local AI  development and provides a foundation for exploring more advanced concepts like fine-tuning models, integrating with different AI services, and building complex conversational applications.

The use of abstractions like the "Microsoft.Extensions.AI" library is crucial for future scalability and adaptability. Keep experimenting, learning, and pushing the boundaries of what’s possible with local AI!