koshaकोश / Model registrynpmv1.12.1

DOCUMENTATION / 05

Architecture

How discovery, normalization, caching, and pricing fit together.

Read the source ↗
On this page

कोश (kosha) – treasury, storehouse of knowledge

1. Overview

Kosha-discovery uses a two-layer architecture to provide comprehensive, accurate model information across all supported providers.

Layer 1: Dynamic API Discovery

The first layer queries provider APIs in real time to get the authoritative list of available models. Each provider has a dedicated discoverer that authenticates, calls the model-listing endpoint, and normalizes the response into a common ModelCard format. This layer answers: “What models exist right now?”

Layer 2: Static Enrichment

The second layer augments dynamic data with static metadata from the LiteLLM model cost database – token pricing, context window sizes, max output limits, and capability flags. This layer answers: “What can each model do, and what does it cost?”

Why Two Layers?

Provider APIs are inconsistent. Some return pricing (OpenRouter), most do not. By combining live discovery with a curated static dataset, kosha provides a uniform, complete view regardless of which provider you query.

    Provider APIs (live)        LiteLLM Database (static)
           |                            |
           v                            v
    +--------------+            +---------------+
    |  Discoverers |            |   Enrichers   |
    +--------------+            +---------------+
           |                            |
           +--------+    +-------------+
                    |    |
                    v    v
              +---------------+
              |  ModelRegistry |
              +---------------+
                     |
           +---------+---------+
           |         |         |
           v         v         v
        Library     CLI    HTTP API

2. Discovery Flow

Steps

  1. Registry Initialization – ModelRegistry loads discoverers for the requested providers (or all by default).
  2. Credential Resolution – The credential resolver locates API keys per provider by walking a priority chain (see Section 3).
  3. Parallel Discovery – Discoverers run concurrently via Promise.allSettled(). A failing provider does not block others.
  4. Normalization – Raw API responses are converted into ModelCard.
  5. Enrichment – LiteLLM enricher fills pricing, context, and capabilities.
  6. Caching – Results are serialized to disk with a TTL. Subsequent requests within the TTL skip steps 2-5.
  7. Alias Resolution – Short names (“sonnet”, “gpt4”) are mapped to canonical model IDs.
  Request
    |
    v
  [Cache valid?] --yes--> Return cached models
    |no
    v
  [Resolve credentials] --> [Discover in parallel] --> [Normalize]
    --> [Enrich] --> [Write cache] --> [Build alias index] --> Return models

3. Credential Resolution

Credentials are resolved per-provider using a priority chain. The first source that yields a valid key wins.

Priority Chain

Explicit Key --> Env Var --> CLI Token --> Config File --> None (skip provider)

Per-Provider Sources

Provider Env Var CLI Token Path Config Path
Anthropic ANTHROPIC_API_KEY ~/.claude/credentials.json ~/.config/claude/settings.json
OpenAI OPENAI_API_KEY — ~/.config/openai/auth.json
Google GOOGLE_API_KEY / GEMINI_API_KEY ~/.config/gemini/credentials.json ~/.config/gcloud/application_default_credentials.json
OpenRouter OPENROUTER_API_KEY – –
Vercel AI Gateway AI_GATEWAY_API_KEY / VERCEL_OIDC_TOKEN – –
Ollama OLLAMA_HOST – Default: http://127.0.0.1:11434

All providers also accept an explicit key via options.apiKey (highest priority).

4. Module Map

src/
├── index.ts              Public API: createKosha(), ModelCard, ModelRegistry
├── types.ts              ModelCard, ModelMode, ModelPricing, ProviderName, etc.
├── registry.ts           ModelRegistry orchestrator (discovery + enrichment + cache)
├── aliases.ts            Alias resolution: short names --> canonical IDs
├── cache.ts              File-based JSON cache with TTL (~/.cache/kosha/)
├── cli.ts                CLI entry point, arg parsing
├── cli-format.ts         ANSI formatting + table rendering
├── cli-commands.ts       list, resolve, info, cache-clear, serve
├── server.ts             Hono REST API (/api/models, /api/resolve/:alias, etc.)
├── discovery/
│   ├── base.ts           Abstract BaseDiscoverer interface
│   ├── anthropic.ts      Anthropic /v1/models
│   ├── openai.ts         OpenAI /v1/models
│   ├── google.ts         Google /v1beta/models
│   ├── ollama.ts         Ollama /api/tags (local)
│   ├── openrouter.ts     OpenRouter /api/v1/models
│   └── index.ts          createDiscoverer() factory
├── enrichment/
│   ├── litellm.ts        LiteLLM pricing/context enrichment
│   └── index.ts          Enricher exports
└── credentials/
    ├── resolver.ts       Multi-source credential discovery
    └── index.ts          Credential exports

5. Data Flow Diagram

 EXTERNAL SOURCES                 KOSHA INTERNALS                  CONSUMERS
 ================                 ===============                  =========

 +----------------+
 | Anthropic API  |--+
 +----------------+  |        +----------------+
 | OpenAI API     |--+------->|  Discovery     |     +----------------+
 +----------------+  |        |  Layer         |     |                |
 | Google AI API  |--+        |  (parallel     |---->| ModelRegistry  |
 +----------------+  |        |   fetch +      |     |  .discover()   |
 | OpenRouter API |--+        |   normalize)   |     |  .list()       |
 +----------------+  |        +----------------+     |  .resolve()    |
 | Ollama (local) |--+               |               +-------+--------+
 +----------------+                  v                   |    |    |
                              +----------------+         v    v    v
 +----------------+           |  Enrichment    |       Lib  CLI  HTTP
 | LiteLLM JSON   |--------->|  Layer         |
 +----------------+           +----------------+
                                     |
 +----------------+                  v
 | Env / CLI /    |           +----------------+
 | Config files   |---------->|  Cache Layer   |
 +----------------+           |  (disk, TTL)   |
                              +----------------+

Transformation Pipeline

Provider API Response (raw JSON)
  --> Extract fields (id, name, description, created, owner)
  --> Normalize to ModelCard (standardize names, infer mode, set provider)
  --> Enrich from LiteLLM (pricing, context, maxOutput, capabilities)
  --> Index & Alias (build lookups, register aliases, sort)
  --> Final ModelCard[] ready to query

6. Adding a New Provider

Step 1: Define the Provider Name

Add to the ProviderName union in src/types.ts:

export type ProviderName =
  | "anthropic" | "openai" | "google"
  | "openrouter" | "ollama"
  | "newprovider";  // <-- add here

Step 2: Create the Discoverer

Create src/discovery/newprovider.ts:

import { BaseDiscoverer } from "./base.js";
import type { ModelCard, DiscoverOptions } from "../types.js";

export class NewProviderDiscoverer extends BaseDiscoverer {
  readonly provider = "newprovider" as const;

  async discover(options: DiscoverOptions): Promise<ModelCard[]> {
    const apiKey = options.credentials?.["newprovider"];
    if (!apiKey) return [];

    const response = await fetch("https://api.newprovider.com/v1/models", {
      headers: { Authorization: `Bearer ${apiKey}` },
    });
    if (!response.ok) return [];

    const data = await response.json();
    return data.models.map((m: any) => ({
      id: m.id,
      name: m.display_name ?? m.id,
      provider: this.provider,
      mode: this.inferMode(m),
      capabilities: this.extractCapabilities(m),
      contextWindow: m.context_length ?? 0,
      maxOutputTokens: m.max_output ?? 0,
      pricing: undefined,
      aliases: [],
      source: "api",
    }));
  }
}

Step 3: Register the Discoverer

Update src/discovery/index.ts:

case "newprovider":
  return new NewProviderDiscoverer();

Step 4: Add Credential Resolution

Update src/credentials/resolver.ts:

case "newprovider":
  return options.apiKey ?? process.env.NEWPROVIDER_API_KEY ?? undefined;

Step 5: Add Aliases

Update src/aliases.ts with short names for the new provider’s models.

Step 6: Write Tests

Create test/discovery/newprovider.test.ts covering:

  • API response mocking and normalization verification
  • Graceful failure handling
  • Missing credentials (returns empty array)

Step 7: Update Documentation

  • Add the provider to the table in SKILL.md
  • Add credential sources to Section 3 of this document
  • Update the README with new environment variables

Appendix: Key Design Decisions

File-based caching – Kosha is used across CLI invocations. Disk cache means kosha list in one terminal benefits from discovery in another.

LiteLLM as enrichment source – Most comprehensive open-source database of model pricing and capabilities, community-maintained and frequently updated.

Promise.allSettled() over Promise.all() – A single provider failure should not prevent results from others. Graceful degradation by design.

Hono for HTTP – Lightweight, fast, multi-runtime (Node/Bun/Deno), minimal dependency weight with a complete routing and middleware stack.