bradliebs/ollama-agent-harness/testing

Context

Testing skill for the Ollama Agent Harness. Uses Jest with TypeScript. Tests cover the core subsystems: agent loop, tool dispatch, permission evaluation, context management, subagent delegation, and session persistence.

Patterns

Test File Naming

• Tests live alongside source files: src/core/queryLoop.test.ts
• Integration tests in tests/integration/
• Test fixtures in tests/fixtures/

Testing the Agent Loop

Mock the Ollama client to return controlled responses. Test the loop's behavior with:

• Text-only responses (should stop)
• Tool-call responses (should dispatch and continue)
• Multiple tool calls (should handle concurrency classification)
• Error responses (should trigger recovery)

Testing Permissions

Test deny-first rule ordering:

• A deny rule overrides a more specific allow rule
• Unrecognized actions escalate to ask/deny
• Permission modes cascade correctly

Testing Context Management

• Verify budget reduction truncates oversized tool results
• Verify compaction appends summary, never mutates history
• Verify context assembly includes all required sources

Testing Session Persistence

• Verify append-only writes to JSONL
• Verify resume rebuilds conversation from transcript
• Verify fork creates new session without inheriting permissions

Mocking Ollama

const mockOllama = {
  chat: jest.fn().mockResolvedValue({
    message: { role: 'assistant', content: 'Done', tool_calls: [] },
    done: true,
  }),
};

Assertion Style

• Use descriptive test names: it('denies tool when deny rule matches before allow rule')
• One assertion per logical concept
• Prefer toEqual for object comparison, toBe for primitives

Examples

Agent Loop Test

describe('queryLoop', () => {
  it('stops when model returns text-only response', async () => {
    mockOllama.chat.mockResolvedValueOnce({
      message: { role: 'assistant', content: 'All done.' },
      done: true,
    });
    const events = [];
    for await (const event of queryLoop(config)) {
      events.push(event);
    }
    expect(events).toHaveLength(1);
    expect(events[0].type).toBe('text');
  });
});

Permission Test

describe('permissions', () => {
  it('deny rule overrides specific allow rule', () => {
    const rules = [
      { type: 'allow', tool: 'bash', pattern: 'npm test' },
      { type: 'deny', tool: 'bash' },
    ];
    const result = evaluate(rules, { tool: 'bash', input: 'npm test' });
    expect(result.decision).toBe('deny');
  });
});

Anti-Patterns

• Testing implementation details instead of behavior.
• Depending on real Ollama API calls in unit tests — always mock.
• Skipping permission system tests when adding new tools.
• Not testing error/recovery paths in the agent loop.