EthosPower.ai
Pattern Context
The AI-native web inverts the traditional request-response model. Instead of serving static HTML or JSON, every HTTP request can trigger an LLM inference cycle that generates contextual, natural language responses. The LLM sits in the hot path, not as a background job or async worker.
I first implemented this pattern in 2023 for a midwest utility's outage management system. Their operators needed to query historical storm response data using natural language—"show me restoration times for ice storms in district 4 over 5 inches"—rather than writing SQL or clicking through dashboards. The AI-native approach meant every query hit Ollama running Llama 3.1 70B locally, which generated both the SQL and a natural language summary. No cloud APIs, no data leaving the building.
This pattern works when you need dynamic, context-aware responses that can't be pre-computed. It fails when you need sub-100ms latency or when LLM costs dominate your infrastructure budget. At EthosPower, we've deployed it for document Q&A systems, SCADA alarm analysis, and compliance report generation—but never for real-time protection relay logic.
The Problem
Traditional web applications separate content from presentation. You store data in PostgreSQL, render it through templates, cache aggressively, and serve static assets. This works beautifully for CRUD operations and dashboards.
But energy operators don't think in CRUD. They ask questions: "Why did breaker B-412 trip three times last week?" "Which substations have transformer loading above nameplate during peak?" "Pull all NERC CIP-007 patch records for servers deployed after January 2023."
Building a traditional UI for these questions means anticipating every possible query, writing SQL for each, and maintaining hundreds of report templates. The explosion of permutations makes this approach unmaintainable. We tried it in 2019 for a 500-substation transmission operator—the backlog hit 247 report requests within six months.
The alternative is letting operators write SQL or learn a query DSL. That doesn't work either. I've never met a protection engineer who wanted to learn SQLAlchemy ORM syntax at 2am during a storm restoration.
Solution Architecture
The AI-native web pattern has four components:
LLM Inference Layer
Run your language models on-premises using Ollama. For energy applications, I recommend Llama 3.1 70B or Qwen 2.5 72B. These models handle structured data extraction and SQL generation reliably. Deploy on a dedicated inference server—we use 8x NVIDIA A100 40GB for production, but you can start with 4x RTX 4090 for development.
Key configuration: Set num_ctx to 32768 or higher. Energy data has long context windows—maintenance logs, alarm sequences, equipment specifications. A 4096 token limit will truncate critical information. In one deployment, extending context from 8192 to 32768 reduced hallucinations in root cause analysis by 63%.
Vector Knowledge Base
Use ChromaDB to store embeddings of your technical documentation, past incidents, and equipment specifications. When a query arrives, retrieve the top-k relevant chunks and inject them into the LLM prompt. This grounds the model in your specific equipment and procedures.
I embed everything: P&IDs, one-line diagrams, relay settings, maintenance procedures, past outage reports. For a 12GW generation fleet, we indexed 47,000 documents—1.2TB of raw PDFs converted to embeddings. Query time is 15-40ms for top-20 retrieval, which fits comfortably in our 2-second total response budget.
Critical detail: Use nomic-embed-text for embeddings, not OpenAI's API. Nomic runs locally through Ollama, which means your equipment data never leaves the OT network. This matters for NERC CIP compliance—embedding generation counts as data processing.
Content Ingestion Pipeline
You need fresh data. Energy systems change constantly—new equipment gets commissioned, procedures get updated, protection schemes get modified. A static knowledge base becomes stale in weeks.
Use Firecrawl to pull structured data from internal SharePoint sites, vendor portals, and equipment manufacturer websites. Firecrawl handles JavaScript rendering and returns clean markdown, which embeds much better than raw HTML. We run it nightly against 23 different content sources for our largest utility customer.
For browser automation—testing the LLM-generated SQL queries, validating report outputs, scraping equipment status from legacy SCADA HMIs—Playwright is the only tool I trust in production. The MCP integration means you can expose browser automation as a tool that the LLM can invoke directly. An operator asks "check current loading on all transformers in substation Alpha," and Playwright logs into the SCADA web interface, navigates to the right screens, extracts the data, and returns it to the LLM for summary.
Response Generation
The LLM receives: (1) the user's natural language query, (2) relevant context from the vector database, (3) available tools (database queries, API calls, browser automation), and (4) the conversation history. It generates either a direct answer, a tool invocation request, or a clarifying question.
For structured outputs—SQL queries, JSON API payloads, report templates—use grammar-constrained generation. Ollama supports this through llama.cpp's grammar system. Define a grammar for valid SQL or JSON, and the model physically cannot generate invalid syntax. This reduced SQL errors from 8% to 0.3% in our deployments.
Implementation Considerations
Latency Budget
Llama 3.1 70B generates 25-35 tokens/second on our A100 setup. For a 500-token response, that's 15-20 seconds of generation time. Add 2-3 seconds for vector retrieval, prompt construction, and tool execution. Total latency: 18-24 seconds.
This is acceptable for analytical queries and report generation. It's unacceptable for interactive dashboards or alarm handling. Know your use case. We use AI-native architecture for the "ask a question" interface, but real-time SCADA data still goes through traditional REST APIs with <100ms response times.
Cost Structure
On-premises LLM inference costs are fixed (hardware depreciation, power, cooling) rather than variable (per-token API charges). For our 500-user utility deployment, hardware cost $380,000 and draws 12kW continuously. That's $10,512/year in electricity at $0.10/kWh.
Compare to Claude 3.5 Sonnet API costs: 500 users × 50 queries/day × 1500 tokens/response × $0.015/1k tokens = $562,500/year. The break-even point is under nine months, and you keep full data sovereignty. Try the Sovereign Savings Calculator to model your specific usage patterns.
Security and Compliance
AI-native web applications blur the line between user input and system commands. An operator's natural language query becomes a SQL statement executed against your operational database. This is a massive injection risk.
Mitigation strategies I actually use:
- Run LLM-generated SQL in a read-only replica, never against the production database
- Use parameterized queries even for LLM-generated SQL—parse the model's output, extract parameters, build the query safely
- Log every prompt and response for audit trails (NERC CIP-007 R5.5.2 requires this anyway)
- Implement role-based context filtering so the LLM only sees data the user is authorized to access
Model Selection
I've tested 23 different open-source models for energy applications. Two consistently outperform:
Llama 3.1 70B: Best general-purpose model. Excellent at SQL generation, document summarization, and multi-step reasoning. Requires 80GB VRAM (two A100s or four RTX 4090s).
Qwen 2.5 72B: Better at technical writing and detailed explanations. Slightly worse at structured output. Same VRAM requirements.
Do not use 7B or 13B models for production energy applications. I've tried—Llama 3.1 8B hallucinates equipment specs, generates invalid SQL 15% of the time, and can't maintain context across multi-turn conversations. The cost savings aren't worth the reliability loss.
Failure Modes
AI-native web applications fail differently than traditional ones. When a database query fails, you get an error message. When an LLM fails, you get a plausible-sounding answer that's completely wrong.
Real failure I debugged last month: An operator asked "show all transformers due for oil testing." The LLM generated SQL that joined the equipment table to the maintenance schedule using transformer name instead of equipment_id. Worked fine until two transformers had the same name ("T1" appears in multiple substations). The query returned 47 transformers instead of 23. The operator scheduled unnecessary maintenance on 24 transformers, cost $67,000 in contractor time.
The fix: Validate LLM outputs programmatically before execution. For SQL queries, check for ambiguous joins, missing WHERE clauses on large tables, and cartesian products. For API calls, validate against OpenAPI schemas. For browser automation, verify expected elements exist before clicking.
Real-World Trade-offs
After deploying AI-native web applications in seven different energy organizations, here's what actually matters:
This pattern excels at: Document Q&A, root cause analysis, compliance reporting, anomaly explanation, natural language data access for non-technical users.
This pattern fails at: Real-time control, alarm processing, high-frequency data streams, anything requiring <1s latency, applications where wrong answers cause safety issues.
Hidden cost: Prompt engineering and maintenance. Your LLM's behavior depends entirely on system prompts, which need constant tuning as your data and use cases evolve. We have one full-time engineer doing nothing but prompt maintenance for our largest customer. Budget for this.
Hidden benefit: Reduced custom development. We killed 180 pending report requests by deploying an AI-native query interface. Operators can now ask for any report they want, formatted however they want, without waiting for dev cycles.
The Verdict
The AI-native web pattern is production-ready for analytical workloads in energy operations—if you run your own models. The latency is manageable, the cost structure favors high-volume use cases, and the flexibility eliminates report backlogs.
But keep it away from real-time systems. I would never put an LLM in the hot path for SCADA data acquisition, protection relay logic, or automatic generation control. The stakes are too high and the latency too variable.
Start with a single use case: document Q&A or natural language database access. Deploy Ollama on local hardware, index your technical documentation in ChromaDB, and give operators a chat interface. Measure actual usage patterns for three months before expanding. Ask our EthosAI Chat assistant for a personalized implementation roadmap based on your specific infrastructure.