effGen

Where v0.3.0 hardened the framework, v0.3.1 sands down the edges real professionals hit the moment they sit down with it. No new providers or subsystems — the things you already reach for are now more predictable, measurable, and consistent. No breaking API changes — every change is additive or makes a previously-silent failure surface a clear, typed error.

from effgen import create_agent, LegalDomain

# Grounded research: sources/citations come from the URLs the tools retrieved.
agent = create_agent("research", "openai:gpt-5-nano")
r = agent.run("What is the capital of France? Cite a source.")
print(r.text)                  # "...Paris (Source: https://en.wikipedia.org/wiki/Paris)."
print(r.sources)               # ['https://en.wikipedia.org/wiki/Paris']
print(r.metadata["cost_usd"], r.metadata["latency_ms"])

# A knowledge domain becomes a runnable agent in one call.
legal = LegalDomain().to_agent("openai:gpt-5-nano")
print(legal.run("What does an NDA confidentiality clause protect?").text)

effgen run --json -q "What is 25 * 17?" | jq .output   # pure-JSON stdout for CI
effgen models status                                    # physical GPU memory; which card is free

Full v0.3.1 changelog →

effGen v0.3.0 made the framework production-safe from the inside out. No breaking API changes.

• Fail-closed Agent.run() — no silent success; typed, redacted errors; smarter retries and loop detection.
• Self-updating, drift-aware model catalog — effgen models refresh reconciles the local snapshot against live provider lists (chat models only; never persists ft: ids).
• Real GPU support — deterministic temperature=0, a deadlock-free allocator, clean multi-GPU use.
• Fail-closed API server — forged/expired/wrong-alg JWTs rejected; secure CORS, metrics, RBAC, and budget enforcement.
• Hardened built-in tools — Python REPL timeout, one shared SSRF guard, path confinement, and no unsafe pickle/eval.
• Faster & quieter — import effgen in ~20 ms, faster streaming + agent loop, a scriptable CLI, and a live "thinking" UX.

Full v0.3.0 changelog →

effGen v0.2.9 ships the full observability and reliability stack. All telemetry is async/non-blocking — a failed export never fails inference.

Structured JSON logging with secret redaction. Every log line is a JSON object: {ts, level, module, event, attributes, trace_id, span_id}. The built-in Redactor strips OpenAI, Anthropic, Cerebras, Google, HF, Groq, Bearer, Slack, and Discord webhook patterns at the encoder — no secret ever appears in a log file.

from effgen.observability import get_logger
log = get_logger(__name__)
log.event("model.call.started", provider="cerebras", model="gpt-oss-120b", cached_tokens=0)
# → {"ts": "2026-05-23T...", "level": "INFO", "event": "model.call.started", ...}

Prometheus histograms + SLO tracking. effgen_model_call_latency_seconds, effgen_tool_call_latency_seconds, effgen_agent_iteration_latency_seconds, and effgen_tokens_total now expose histogram buckets at /metrics. SLOTracker maintains a rolling-window error budget and burn_rate() at /slo.

Configurable OTel samplers + canonical span spec. Choose AlwaysOn, AlwaysOff, TraceIdRatio(p), or RateLimited(per_second) in config. effgen/observability/spans.py is the single source of truth for every span attribute name.

Reliability primitives. Four layers now protect every adapter call:

Deterministic chaos harness. Inject NetworkTimeout, Http5xx, Http429, SlowResponse, PartialResponse, or MalformedJSON faults with Chaos(seed). Four canonical scenarios — fallback on 5xx, Retry-After honoured, timeout fires cleanly, AllProvidersFailed — all pass deterministically across 10 seeds.

Fuzz suite. Hypothesis runs 500 examples against all 66 BaseTool subclasses, random ContentPart message sequences, and the router's provider-availability logic. No unhandled exceptions, no secret leaks.

Load-testing CLI + Alertmanager rules.

# Run a 30-second load test (JSON report prints to stdout by default)
effgen loadtest --concurrency 10 --duration 30 --scenario fixed

# Or write the report to a file with --output
effgen loadtest --concurrency 10 --duration 30 --output report.json

# Integrate with Alertmanager
cp docs/observability/alert_rules.yaml /etc/prometheus/rules/effgen.yaml

See docs/observability/overview.md, docs/observability/metrics.md, and docs/observability/alerting.md.

effGen v0.2.8 accepts image, audio, and video as input types. Send them to any vision-capable provider through a unified Message schema — the adapter handles the translation, not your code.

Image input — Gemini, OpenAI gpt-4o, Groq, Anthropic (code-only), Together, HF. Automatic resize/MIME validation via image_pre.py. Raises CapabilityNotSupportedError cleanly when the provider doesn't support vision.

Audio input — Gemini native inline audio, OpenAI Whisper transcription + gpt-4o audio, HF Inference ASR. Auto-downsamples to 16 kHz mono; chunks files over provider max duration. Anthropic raises CapabilityNotSupportedError.

Video input — Gemini native video for providers that accept raw video; frame-sampling fallback (ffmpeg) for all others. MissingSystemDependency with install hints when ffmpeg is absent.

Unified message schema — TextPart, ImagePart, AudioPart, VideoPart form a typed ContentPart union. Message.content is always a List[ContentPart]; backwards-compatible string constructor still works.

multimodal preset — create_agent("multimodal", model) wires Gemini Flash-Lite (primary) + OpenAI gpt-4o-mini (fallback) with ImageInfo, ImageCaption, OCR, AudioTranscribe, MultimodalDescribeTool, and the full tool suite.

5 cookbook walkthroughs — image Q&A, audio transcribe + reason, video summarize, OCR + LLM structured extraction, chart reading from an image. All in docs/cookbook/.

from effgen import image_from, audio_from
from effgen.presets import create_agent
from effgen import load_model

model = load_model("gemini-3.1-flash-lite", provider="gemini")
agent = create_agent("multimodal", model)

# Image question — pass media through inputs=
img = image_from("https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png")
result = agent.run("What is in this image?", inputs=[img])
print(result.output)

# Audio transcription
aud = audio_from("/tmp/clip.mp3")
result = agent.run("Transcribe and summarize.", inputs=[aud])

effgen run --preset multimodal "Describe this image" --file /tmp/photo.jpg
python -c "from effgen.models.capabilities import Capability; print(Capability.vision)"

See docs/multimodal/overview.md and docs/cookbook/README.md.

effGen v0.2.7 adds a curated, domain-organized Prompt Library with reusable templates, paired with a golden evaluation harness and an interactive playground CLI. See the full gallery.

Research — literature review (zero-shot + CoT), paper summary, citation extraction, methodology critique. Coding — code review, bug diagnosis, refactoring plan, test generation, docstring fill. Data / SQL — NL-to-SQL with warnings, SQL explain, SQL optimize, data profile, ETL plan. Legal — contract summary, clause classify, research brief. All templates include mandatory legal disclaimer. Medical — symptom triage, drug interaction, medical literature synthesis. All templates include mandatory medical disclaimer. Creative — story continuation (zero-shot + few-shot), poetry forms, character bio, world building. Business — meeting summary, email draft (formal/casual), OKR generation, SWOT analysis, elevator pitch.

effgen prompts list
effgen prompts list --domain research
effgen prompts show research.literature_review.v1.cot
effgen prompts eval --domain coding --live --model gpt-oss-120b
effgen prompts playground

from effgen.prompts.library import registry

p = registry.get("data.sql_from_nl.v1")
sql_prompt = p.template(
    schema_ddl="CREATE TABLE orders (id INT, customer TEXT, total FLOAT, created_at DATE)",
    question="Total revenue per customer this month",
    dialect="postgresql",
)

See docs/prompts/gallery.md and docs/prompts/library.md.

effGen v0.2.6 adds 14 new built-in tools across document, media, and communication categories, and two new presets (media, notify).

1. OCR — OCRTool (Tesseract local + OCR.space fallback; OCRBackendUnavailable raised with install instructions).

   import asyncio
   from effgen.tools.builtin.ocr import OCRTool
   result = asyncio.run(OCRTool().execute(operation="extract", image_path="/tmp/scan.png"))
   print(result.output["text"])
   

2. Audio Transcription — AudioTranscribeTool (faster-whisper local; HF Inference fallback; GPU auto-detected).

3. Image Analysis — ImageInfoTool (Pillow metadata, zero network) + ImageCaptionTool (vision-capable model router).

4. Document Parsing — PDFTool (pypdf + pdfplumber), DOCXTool (python-docx), ExcelTool (openpyxl + pandas). Added to research and general presets.

   import asyncio
   from effgen.tools.builtin.pdf import PDFTool
   result = asyncio.run(PDFTool().execute(operation="text", path="/tmp/paper.pdf"))
   

5. Geo / Weather — WeatherTool (Open-Meteo, free, no auth), GeocodeTool (Nominatim/OSM, 1 req/s), MapsTool (staticmap PNG renderer).

6. Email & Webhooks — EmailSMTPTool, EmailIMAPTool, SlackWebhookTool, DiscordWebhookTool. All in the new notify preset. Webhook URLs are redacted in logs.

See the full tool gallery.

effGen v0.2.5 adds 13 free, no-auth-required tools. All integrate with the research and general presets.

1. Academic Research — PubMedTool (NCBI, 3 ops, built-in rate limiting), ArXivTool (Atom feed + PDF download), SemanticScholarTool (search + citations + references).

   import asyncio
   from effgen.tools.builtin.arxiv import ArXivTool
   result = asyncio.run(ArXivTool().execute(operation="search", query="transformer attention", max_results=5))
   

2. News & RSS — RSSFeedTool (any RSS/Atom feed), NewsTool (BBC, Reuters, HN, NPR, etc. + optional NewsAPI.org key).

3. YouTube — YouTubeTranscriptTool (captions without Google API key), YouTubeMetadataTool (via yt-dlp, public content only).

4. Social Media — RedditTool (public JSON, no OAuth), HackerNewsTool (Firebase API, no auth).

5. Translation & Language Detection — TranslateTool (LibreTranslate + offline argostranslate fallback), LanguageDetectTool (55+ languages, fully offline).

6. QR Codes — QRGenerateTool (generate locally), QRReadTool (decode from image, with OpenCV fallback if zbar is unavailable).

See the full tool gallery.

1. PolicyBasedRouter — composable routing engine with three built-in policies. Pick the cheapest provider within your budget, the fastest under your SLA, or simply the first available.

   from effgen import PolicyBasedRouter, RoutingContext, CostBasedPolicy, LatencyBasedPolicy
   from effgen.models.capabilities import Capability
   
   router = PolicyBasedRouter(policies=[LatencyBasedPolicy(), CostBasedPolicy()])
   ctx = RoutingContext(
       prompt_tokens_estimate=500,
       user_budget_usd=0.01,
       latency_budget_ms=3000,
       required_capabilities={Capability.chat},
   )
   decision = router.route(ctx)
   print(decision.chosen)      # e.g., ProviderModelPair("cerebras", "gpt-oss-120b")
   print(decision.eliminated)  # [(pair, reason), ...] — fully explainable
   

2. Transparent failover — route_and_execute(ctx, fn) retries on rate-limits / 5xx / timeouts and moves to the next-best provider. Each hop fires a RouterEvent to registered subscribers.

3. Cross-process SQLite rate-limit coordination — share a single rate-limit budget across multiple workers via RateLimitCoordinator(SQLiteRateLimitStore(...)) (WAL-mode, BEGIN IMMEDIATE).

4. Persistent cost tracking + effgen cost CLI — every API call persists to SQLite:

   effgen cost today          # per-provider per-model table
   effgen cost week           # rolling 7-day view
   effgen cost by-provider    # lifetime totals
   effgen cost set-budget 1.0 # set $1/day cap (BudgetExceededError at 100%)
   

5. Fully explainable decisions + budget guard — RouterDecision records every eliminated provider and why ("rate_limited", "no_key", "cost_exceeds_budget", "latency_exceeds_sla"), and fails over to a free-tier provider when the budget is hit.

1. 5 new cloud backends — GroqAdapter, TogetherAdapter, FireworksAdapter, ReplicateAdapter, HFInferenceAdapter — each with streaming, native tools, rate-limit coordination, and cost tracking. 9 providers total.

   model = load_model("openai/gpt-oss-20b", provider="groq")
   model = load_model("Qwen/Qwen2.5-72B-Instruct", provider="hf")
   

2. Unified ProviderRegistry — list_providers(), list_models(provider), lookup(model_id) consolidated across all 9 adapters. AmbiguousModelError on bare IDs shared across providers.

3. effgen doctor — new CLI command showing which providers have API keys configured.

4. Backend parity matrix — canonical agentic task ("(17 × 23) + sqrt(144) = 403") runs identically across all providers; streaming and error surfaces verified uniform. See docs/providers/parity.md.

5. HuggingFace Router support — HFInferenceAdapter with 124-model dynamic catalog, refresh_models() + check_drift(), ModelUnavailableError with suggest_alternatives(), and custom Inference Endpoint URL.

1. Gemini 3.x/2.5/2.0 + Gemma families — full model registry with correct context windows, output limits, and feature flags; SDK migrated to google-genai>=1.0.0.

2. Gemini thinking_budget — activate Gemini's internal reasoning with GenerationConfig(thinking_budget=8192, include_thoughts=True); thinking trace surfaces in ModelResponse.metadata["thinking"].

3. Gemini grounding + Files API — GenerationConfig(grounding=True) injects Google Search; upload_file(path) passes PDFs/images to the model with a 2 GiB guard.

4. Gemini native tools — GoogleSearchTool, GeminiUrlContextTool, GeminiCodeExecutionTool activate server-side Gemini capabilities in any Agent. Parallel function calls handled automatically.

5. Anthropic Claude 4.7, extended thinking, prompt caching — full Claude 4.x registry; GenerationConfig.thinking for extended reasoning; mark_cached() + AgentConfig.cache_system_prompt/cache_tools for cache_control; cache tokens surfaced in usage.

1. Cerebras backend — the models the live API currently serves (gpt-oss-120b, zai-glm-4.7) with streaming, native function-calling, automatic RPM/TPM/RPD/TPD rate-limit coordination, and per-call cost tracking. pip install effgen[cerebras] and set CEREBRAS_API_KEY. Run effgen models refresh --provider cerebras to pick up catalog changes.

   from effgen import load_model
   model = load_model("gpt-oss-120b", provider="cerebras")
   

2. OpenAI gpt-5 / gpt-5.4-nano / o-series reasoning models — full registry coverage with reasoning_effort (minimal/low/medium/high) and max_reasoning_tokens on GenerationConfig. Reasoning payloads are routed only to reasoning-capable models.

3. OpenAI prompt caching surfacing — cached_input_tokens exposed on ModelResponse.usage; AgentConfig.stable_system_prompt=True keeps the system prompt anchored at position 0 to maximize OpenAI's automatic ≥1024-token prefix cache hit rate.

4. Structured outputs v2 — OpenAIAdapter.generate_structured() with strict JSON Schema; to_openai_schema(pydantic_model) inlines $refs and forces additionalProperties: false; refusals raise ModelRefusalError.

5. OpenAI native tools — OpenAIWebSearchTool, OpenAICodeInterpreterTool, OpenAIFileSearchTool route through OpenAI's Responses API and compose with effGen's local tools in the same agent. ToolIncompatibleError fires at Agent init when paired with a non-OpenAI model.

1. Native Tool Calling — Qwen, Llama, Mistral models use built-in function calling instead of text parsing. Set tool_calling_mode="native" or "hybrid". Structured JSON/Pydantic output validation included.

2. Guardrails & Safety — PII detection, prompt injection blocking, toxicity filtering, tool permissions. One-liner: get_guardrail_preset("strict").

3. Production RAG Pipeline — Ingest PDF/DOCX/HTML/Markdown, semantic+BM25 hybrid search, reranking, inline citations. create_agent("rag", model, knowledge_base="./docs/").

4. Production API Server — OpenAI-compatible /v1/chat/completions, request queuing, agent pooling, multi-tenancy, API keys. Drop-in OpenAI replacement with local SLMs.

5. Apple Silicon Native — MLX & MLX-VLM backends for M1/M2/M3/M4. Metal GPU acceleration, unified memory. pip install effgen[mlx].