Minecraft Integration

Autonomous AI bot with cognitive architecture for Minecraft

Overview

The Minecraft integration is an intelligent bot built on a four-layered cognitive architecture inspired by cognitive science. AIRI can understand natural language commands, explore the world autonomously, gather resources, craft items, and interact with players.

Features

• Cognitive Architecture: Perception → Reflex → Conscious → Action layers
• Natural Language Understanding: Chat with the bot using plain English
• Autonomous Gameplay: Gather resources, craft items, build structures
• World Awareness: Navigate terrain, avoid obstacles, track entities
• Combat System: Fight hostile mobs and defend itself
• Inventory Management: Smart item collection and crafting
• Debug Dashboard: Web-based real-time monitoring and MCP integration
• Mineflayer Viewer: Watch the bot’s POV in your browser

Prerequisites

• Node.js 18 or higher
• Minecraft Java Edition server (1.20 recommended)
• OpenAI API key (or compatible LLM provider)
• AIRI server runtime (optional, for integration)

Setup

1. Prepare Minecraft Server

You need a Minecraft server where the bot can connect. Options:

• Local server: Download from minecraft.net
• Aternos: Free hosting at aternos.org
• Existing server: Use any Java Edition server

The bot supports Minecraft Java Edition versions 1.12 - 1.20. Version 1.20 is recommended.

2. Configure Environment

Navigate to the Minecraft service:

cd services/minecraft
cp .env .env.local

Edit .env.local:

# Minecraft Server
BOT_USERNAME='AIRI'
BOT_HOSTNAME='localhost'
BOT_PORT=25565
BOT_VERSION='1.20'
BOT_AUTH='offline'  # Use 'microsoft' for online mode

# OpenAI Configuration
OPENAI_API_KEY='sk-...'
OPENAI_MODEL='gpt-4o'
OPENAI_REASONING_MODEL='o1-mini'
OPENAI_API_BASEURL='https://api.openai.com/v1'

# AIRI Server (optional)
AIRI_WS_BASEURL='ws://localhost:6121/ws'
AIRI_CLIENT_NAME='minecraft-bot'

3. Install Dependencies

From project root:

pnpm install

4. Start the Bot

pnpm run -F @proj-airi/minecraft-bot start

You should see:

[Bot] Connecting to localhost:25565...
[Bot] Successfully logged in as AIRI
[MineflayerViewer] Started on http://localhost:3007
[DebugService] Debug server started on http://localhost:3008

Usage

In-Game Commands

Talk to the bot in Minecraft chat:

Gather 10 oak logs
Craft a wooden pickaxe
Find some iron ore
Build a small house
Follow me

Web Viewer

Open http://localhost:3007 to see the bot’s first-person view in real-time.

Debug Dashboard

Open http://localhost:3008 for:

• Real-time bot status
• Cognitive layer monitoring
• Tool execution interface
• MCP server integration

Cognitive Architecture

The bot operates on four distinct layers:

Layer A: Perception

Location: src/cognitive/perception/

Collects and processes raw Mineflayer events into normalized signals:

• Chat messages
• Health/hunger changes
• Entity spawns
• Block interactions
• Combat events

Layer B: Reflex

Location: src/cognitive/reflex/

Handles immediate, instinctive reactions:

• Auto-eat when hungry
• Dodge incoming attacks
• React to sudden damage
• Idle behaviors (looking around)

Layer C: Conscious

Location: src/cognitive/conscious/

LLM-powered reasoning and planning:

• Interprets user commands
• Plans multi-step tasks
• Manages goal hierarchies
• Handles conversation

Layer D: Action

Location: src/cognitive/action/

Executes concrete tasks in the world:

• Movement and pathfinding
• Block breaking/placing
• Item crafting
• Entity interaction
• Inventory management

Code Examples

Bot Initialization

From services/minecraft/src/main.ts:22:

import { initBot } from './composables/bot'
import { CognitiveEngine } from './cognitive'
import { Client } from '@proj-airi/server-sdk'

async function main() {
  // Initialize Mineflayer bot with plugins
  const { bot } = await initBot({
    botConfig: config.bot,
    plugins: [
      MineflayerPathfinder,
      MineflayerCollectBlock,
      MineflayerTool,
      MineflayerPVP,
    ],
    reconnect: {
      enabled: true,
      maxRetries: 5,
    },
  })

  // Connect to AIRI server
  const airiClient = new Client({
    name: config.airi.clientName,
    url: config.airi.wsBaseUrl,
  })

  // Load cognitive engine
  await bot.loadPlugin(CognitiveEngine({ airiClient }))
}

Configuration Loading

From services/minecraft/src/composables/config.ts:62:

export function initEnv(): void {
  config.bot = {
    username: getEnvVar('BOT_USERNAME', 'airi-bot'),
    host: getEnvVar('BOT_HOSTNAME', 'localhost'),
    port: getEnvNumber('BOT_PORT', 25565),
    auth: getEnvVar('BOT_AUTH', 'offline'),
    version: getEnvVar('BOT_VERSION', '1.20'),
  }

  config.openai = {
    apiKey: getEnvVar('OPENAI_API_KEY', ''),
    baseUrl: getEnvVar('OPENAI_API_BASEURL', ''),
    model: getEnvVar('OPENAI_MODEL', ''),
    reasoningModel: getEnvVar('OPENAI_REASONING_MODEL', ''),
  }
}

Skills System

The bot has modular skills in src/skills/:

Movement (src/skills/movement.ts):

// Go to coordinates
await bot.skills.movement.goto(x, y, z)

// Follow entity
await bot.skills.movement.follow(player)

// Come to player
await bot.skills.movement.comeToPlayer(username)

Gathering (src/skills/blocks.ts):

// Collect specific block type
await bot.skills.blocks.collectBlock('oak_log', 10)

// Mine blocks in area
await bot.skills.blocks.mineBlocks('iron_ore', {
  count: 5,
  searchRadius: 32
})

Crafting (src/skills/crafting.ts):

// Craft items
await bot.skills.crafting.craft('wooden_pickaxe', 1)

// Ensure item exists (craft if needed)
await bot.skills.crafting.ensure('crafting_table')

Combat (src/skills/combat.ts):

// Attack entity
await bot.skills.combat.attack(entity)

// Defend against attacker
await bot.skills.combat.defend()

Plugin System

Extend bot capabilities with plugins:

interface BotPlugin {
  name: string
  install: (bot: Bot) => Promise<void> | void
  uninstall?: (bot: Bot) => Promise<void> | void
}

// Example plugin
const myPlugin: BotPlugin = {
  name: 'my-plugin',
  install(bot) {
    bot.on('chat', (username, message) => {
      if (message.includes('hello')) {
        bot.chat('Hello!')
      }
    })
  }
}

await bot.loadPlugin(myPlugin)

Advanced Features

Query DSL

Query the world from conscious layer:

// Find nearest entities
const zombies = query.entities()
  .type('zombie')
  .distance('< 10')
  .alive()
  .list()

// Check inventory
const hasDiamonds = query.inventory()
  .item('diamond')
  .count('> 3')
  .exists()

// Find blocks
const ironOres = query.blocks()
  .type('iron_ore')
  .radius(32)
  .list()

MCP Integration

The debug server exposes Model Context Protocol tools:

// Available MCP tools:
- minecraft_status     // Get bot status
- minecraft_execute    // Execute arbitrary code
- minecraft_move       // Move to coordinates
- minecraft_chat       // Send chat message
- minecraft_collect    // Collect blocks
- minecraft_craft      // Craft items

Troubleshooting

Bot can’t connect to server

1. Verify server is running and accessible
2. Check BOT_HOSTNAME and BOT_PORT
3. For online servers, use BOT_AUTH='microsoft'
4. Ensure Minecraft version matches: BOT_VERSION='1.20'

Bot is stuck or unresponsive

1. Check debug dashboard for task status
2. Send stop command in chat
3. Restart the bot service
4. Check logs for errors

LLM errors

1. Verify OPENAI_API_KEY is valid
2. Check API rate limits
3. Ensure OPENAI_MODEL is accessible
4. Try a different model endpoint

Pathfinding fails

1. Bot may be in difficult terrain
2. Increase search radius
3. Clear obstacles near bot
4. Use /tp to relocate bot

Performance Tips

• Use o1-mini for complex reasoning tasks
• Set shorter tick intervals for faster reactions
• Reduce perception event frequency for lower CPU usage
• Enable viewer only when debugging

Next Steps

Factorio Integration

Automate Factorio gameplay

API Reference

Full Minecraft API documentation