# Code for IBM i Developer Detective * * * > This is the follow-up to my [IBM Dev Day: BOB in Action](https://sbm-tech.hashnode.dev/ibm-dev-day-bob-in-action-hackathon) post. That one covered the Dev Day talks. This one is about what I actually built during the hackathon. Three days. Forty Bob coins. One idea. One Developer. That's what the **IBM TechXchange 2026 Pre-conference Dev Day Hackathon** looked like from the inside. The objective was simple: pick a real problem, use IBM Bob to build something that solves it, and demo it to the judges. I picked a problem I'd been personally faced and I think I built something genuinely useful. * * * ## The Starting Point Let me be clear upfront: the **Code for IBM i VS Code extension** is genuinely excellent. It's the tool that brought IBM i development into the modern IDE world — and it keeps getting better. The maintainer team does great work. But I am trying to address 'if an issue ocurrs while working with Code for IBMi extension'. And the most common one here is: you upgrade to a new version, something unexpected happens during startup, and you're not sure why. The usual workflow: search GitHub issues manually, land in a thread, piece together a root cause from scattered comments, then write a support ticket hoping you've included everything the maintainers actually need. It works — it just takes 15 to 30 minutes that you didn't plan for. That's the gap Detective fills. More like a diagnostic layer that sits alongside it and saves you that time when you need it. * * * ## What I Built: Code for IBM i Developer Detective **One prompt. Bob does the rest.***→* [*GitHub repo: Bob-builder-sangamesh*](https://github.com/Sangameshsbm/Bob-builder-sangamesh) The first thing worth calling out: Detective is a **custom Bob mode**. One of the things Bob enables is the ability to create your own modes — purpose-built personalities with their own skills, workflows, and grounding rules. Detective is exactly that. You switch to the **Code for IBM i Detective** mode in Bob's mode picker, and from that point Bob knows exactly what to do, which scripts to run, and what rules to follow. No manual setup, no prompting from scratch every time. ![](https://cdn.hashnode.com/uploads/covers/6a4a5ff62ecc7309bd8b0e4f/4a4ecd83-f135-4d47-bd3c-7398dfc3616a.png align="left") **Step 1 — Collect the extension logs** A plain Node.js script reads the Code for IBM i extension output directly from VS Code's standard log directory — `AppData` on Windows, `Application Support` on macOS, `.config` on Linux. No VS Code API. No IBM i connection. Pure filesystem read. The developer never has to find the log manually. **Step 2 — Search the GitHub issue tracker** Another script takes keyword extractions from the log data and queries the `codefori/vscode-ibmi` GitHub issue tracker via the GitHub Search API. It returns structured JSON — issue number, title, state (open/closed), URL, comment count, body excerpt. **Step 3 — Score confidence** The search results and log data feed into a symptom matcher that calculates a confidence score (0–100) and assigns a band — high (75–100), medium (40–74), or low (below 40). High confidence means a known issue matched well. Low confidence means the AI does open-ended analysis on the raw log data instead. **Step 4 — Generate outputs** * A **resolution report**: problem summary, key log errors, matched GitHub issues, ordered fix steps, validation checklist * A **ready-to-raise GitHub issue**: pre-filled with your environment table, full log errors, connection trace, related issues already cross-referenced, and an AI-proposed root cause — clearly labelled so maintainers know exactly what needs their verification What would normally take 15–30 minutes of manual searching now takes under two minutes — and you get back to the actual work faster. * * * ## The Design Decision Here's the design principle I cared most about: the output has to be trustworthy. A developer acting on a diagnosis, and a maintainer triaging a GitHub issue, both need to know the facts came from real data — not from an AI filling in gaps. So I made a deliberate decision early on: **the AI touches nothing in the diagnostic path**. The pipeline has two layers, with a hard boundary between them: **Deterministic layer** (no AI): * `collect-diagnostics.js` — reads the actual log file. Returns structured JSON. Either the log line is there or it isn't. * `search-github-issues.js` — calls the GitHub Search API. Returns real issue data. Either the issue exists or it doesn't. * The symptom matcher calculates the confidence score mathematically. **AI layer** (Bob, only at the end): * Reads the structured JSON output from the deterministic layer * Writes the prose: the resolution summary, the root cause paragraph, the GitHub issue body * Has one hard rule built into the Bob skill: *"Never speculate. Never invent log lines, error messages, extension versions, or issue numbers."* The result is a system where every fact in the report came from a script, and the AI only does what AI is actually good at — synthesising and writing clearly from real data. The evidence is auditable. The confidence score is reproducible. The AI adds value without introducing hallucination risk into the diagnostic path. I called this the **two-adapter, one-core-engine** design. The core engine is IDE-agnostic plain JavaScript. Two thin orchestration layers sit on top — one for Bob (via a custom skill and mode), one for VS Code with Copilot or Claude (via `AGENT.md`). Same scripts. Same templates. Same outputs. No re-implementation. * * * ## The Architecture in Plain English * * * ![](https://cdn.hashnode.com/uploads/covers/6a4a5ff62ecc7309bd8b0e4f/4a87e7a1-9862-475c-ad51-f299b3fde23d.png align="center") ## The Demo: Two Scenarios, Both Offline I built the demo to run entirely offline — no live IBM i, no GitHub token required. ### Tier 1 — Port 449 warm-up (~30 seconds) The simplest possible IBM i error: the extension can't connect because port 449 is blocked. ```plaintext investigate Code for IBM i issue: cannot connect, port 449 connection refused ``` Bob reads the mock log, finds `ECONNREFUSED` on port 449 across four error lines, matches it at 89% confidence, and returns ordered fix steps (start host servers, check NETSTAT, check firewall) in under 30 seconds. ### Tier 2 — Issue #3239 main act (~90 seconds) A real GitHub issue. A real scenario that developers have actually hit after upgrading. ```plaintext investigate Code for IBM i issue: stuck at Starting Mapepire after upgrading to 3.x ``` The mock log contains exactly what a developer would see after upgrading from Code for IBM i 2.14.1 to 3.0.2: ```plaintext ERROR - Mapepire server did not respond within expected time WARN - Previous Mapepire job may still be active from version 2.x ERROR - MCH3601 received from NOXDBSRV: Pointer not set for location referenced ERROR - Connection timeout waiting for Mapepire. Status: Starting Mapepire ``` Bob reads this, extracts keywords, searches GitHub, finds [issue #3239](https://github.com/codefori/vscode-ibmi/issues/3239) (open, 14 comments) and #3201 (a related closed issue fixed in 2.14.1), calculates the confidence score, surfaces the root cause (stale `NOXDBSRV` job in MSGW from version 2.x — an `MCH3601` pointing at stale service program objects in QGPL), and generates a complete GitHub issue body. The before/after is the point of the demo. What a developer might send today — a short message, maybe a screenshot, doing their best — versus what Detective produces: a structured, complete report with environment details, log errors, connection trace, and related issues already cross-referenced. The maintainers get everything they need to act fast, and the developer doesn't have to spend time assembling it manually. * * * ## What Happens When There's No Known Issue This question came up during the demo and it's important. If the GitHub search returns no matches, the workflow doesn't stop. Bob switches to open-ended AI analysis of the raw log data — surfacing error codes, suspicious patterns, version mismatches — and still produces a structured GitHub issue body. The "Related Issues Found" section simply states: no matching issues found. The developer still gets a useful starting point. The maintainers still get complete diagnostic data. And the next person who hits the same scenario has a properly documented case to reference — which helps the whole community over time. * * * ## What Bob Actually Did in the Build People ask: how much did Bob help during the actual build? Honestly — a lot. The scripts (`collect-diagnostics.js`, `search-github-issues.js`), the mock data fixtures, the templates, the `AGENT.md` — Bob helped write all of it. I drove the design decisions and the architecture. I wrote the Bob skill and mode configuration. I made the call on the AI/deterministic split. But the implementation of the scripts themselves? Bob and I paired on those. That's the right way to use it. Bob as a pair programmer who executes your design, not as an autonomous builder who makes the design decisions for you. * * * ## The Measurable Improvement | Step | Manual | With Detective | | --- | --- | --- | | Find the extension log | Scroll VS Code Output panel manually | 1 script call, structured JSON | | Search for known issues | Open GitHub, search manually, read threads | Automated API search, structured result | | Understand root cause | Read a 40-comment thread | 1-paragraph summary grounded in log data | | Write a quality GitHub issue | From scratch, usually missing key data | Structured body in seconds, all data present | | Total time | 15–30 minutes | Under 2 minutes | * * * ## Try It Yourself The full project is on GitHub — [**Code for IBM i Developer Detective**](https://github.com/Sangameshsbm/Bob-builder-sangamesh). Everything is open, documented, and runnable with Node.js 18+. No IBM i required. You can run the pipeline manually in a terminal: ```powershell # Tier 1 — port 449 $env:IBM_I_MOCK='true'; $env:IBM_I_MOCK_SCENARIO='port449' node src/collect-diagnostics.js node src/search-github-issues.js "port 449 connection refused ECONNREFUSED" # Tier 2 — Mapepire hang $env:IBM_I_MOCK='true'; $env:IBM_I_MOCK_SCENARIO='mapepire-hang' node src/collect-diagnostics.js node src/search-github-issues.js "Starting Mapepire stuck upgrade" ``` Or just open it in VS Code with Bob active, switch to the **Code for IBM i Detective** mode, and type your symptom. Adding a new known issue to the registry is a single JSON object in `src/known-issues.json`. No code changes needed. * * * ## Closing Thought The Code for IBM i extension is a fantastic tool — and the maintainer team is a small group of people doing genuinely important work for the IBM i community. Detective is simply a time-saver: a diagnostic layer that helps developers get to the answer faster, and helps maintainers get the information they need without having to ask for it. Built with Bob, in three days, because the problem was real and the solution felt within reach.