Basic LLM Interaction

Your first step into the world of AI agents

Learn how to load and interact with local language models

What is a Local LLM?

A Local LLM is an AI language model that runs entirely on your computer, without needing internet or external APIs. Think of it as having your own AI assistant that never leaves your machine.

Privacy

Your data never leaves your machine

Cost

No per-token API charges

Control

Full control over model selection

Offline

Works without internet connection

How LLMs Generate Responses

User Input

"Hello"

→

Model

Processes

→

Generation

Token by token

→

Response

"Hi there!"

LLMs don't generate entire sentences at once. They predict one token (word piece) at a time, building the response incrementally.

What is a Token? (Beginner-Friendly Explanation)

A token is a piece of text that the model processes. Think of it like this:

Simple Words:

"Hello" = 1 token

"world" = 1 token

"Hello world" = 2 tokens

Complex Words:

"programming" = 1 token

"unprogrammed" = 2-3 tokens

"antidisestablishmentarianism" = 4+ tokens

Key Insight:

Tokens are not exactly words. Common words are usually 1 token, but rare or long words get split into multiple tokens. The model processes text token-by-token, not word-by-word.

Context Window: The Model's Memory

System Prompt (if any)

User: "do you know node-llama?"

AI: "Yes, I'm familiar..."

(Space for more conversation)

Limited size (e.g., 2048, 4096, or 8192 tokens)

Limited Size

Has a maximum capacity (e.g., 4096 tokens)

When Full

Old messages must be removed

Influence

All previous messages affect the next response

Common Beginner Mistakes

Mistake 1: Forgetting to Dispose Resources

❌ Bad:

const session = new LlamaChatSession(...);

await session.prompt("Hello");

// Memory leak! Resources not cleaned up

✅ Good:

const session = new LlamaChatSession(...);

await session.prompt("Hello");

session.dispose(); // Always clean up!

Mistake 2: Expecting Memory Between Calls

❌ Wrong Expectation:

await session.prompt("My name is Alice");

await session.prompt("What's my name?");

// Expects: "Alice" | Reality: Model doesn't remember!

✅ Correct Understanding:

// Each prompt is independent

// To maintain context, you must pass full conversation history

// (We'll learn this in later examples)

Mistake 3: Using Vague Prompts

❌ Too Vague:

"node-llama-cpp"

// Model doesn't know what you want

✅ Clear and Specific:

"What is node-llama-cpp and how does it work?"

// Model understands the request

Key Concepts for AI Agents

Stateless Processing

Each prompt is independent unless you maintain context. The model has no memory between different script runs. To build an "agent", you need to keep context alive and maintain conversation history.

Prompt Engineering

The way you phrase questions affects the response:

❌ Poor: "node-llama-cpp"

✅ Better: "do you know node-llama-cpp"

✅ Best: "Explain what node-llama-cpp is and how it works"

Resource Management

LLMs consume significant resources. Models need gigabytes of RAM/VRAM, inference takes time, and you must clean up properly to avoid memory leaks.

Why This Matters for Agents

🧠

Agents need LLMs to "think"

The model processes information and generates responses

💾

Agents need context

To maintain state across interactions

🔧

Agents need structure

Later examples add tools, memory, and reasoning loops

📖 Quick Reference

Basic Code Pattern

1. Load model

2. Create context

3. Create session

4. Prompt the model

5. Dispose resources

Key Takeaways

• LLMs are stateless - no memory between calls
• Tokens are processed one at a time
• Always dispose resources to prevent leaks
• Clear prompts = better responses

What's Next?

In the next lesson, you'll learn about using cloud-based models (OpenAI) and how they compare to local models.

System prompts Function calling Memory ReAct patterns

Back to Overview Next: OpenAI APIs