Seminar 2. When AI, Which AI, and on What¶
Seminar 2 · 42 slides
01 · Seminar 2. When AI, Which AI, and on What¶
Good afternoon. Today is seminar number two, and the format is simple: we'll work through seven situations together — the kind that could land on any of your desks at real jobs. In each one, someone asks "can you automate this for us," and then the engineer has to decide what to actually build. We're not going to lecture at you and we're not going to present for half an hour — we're going to talk through concrete stories out loud, all together, as a room. I'll show you a setup, you'll say what you'd propose, and then in almost every case a detail will show up that wasn't in the first version of the story. Being willing to revise your decision when new information shows up is exactly the skill you need on a real job, where the client states the problem piece by piece instead of all at once. Let's go.
02 · Today — four choices, seven cases¶
Here's what we're training today — four types of choice that keep coming up in engineering practice, again and again: whether you need AI here at all or plain code is enough, whether to build the solution into what people already use or make a separate product, what shape the model call should take — a one-off call, a search over a knowledge base, or a multi-step agent — and finally, whether to run it through an external cloud API or on your own infrastructure. We're not turning this into a procedure with a checklist — instead we're going to work through seven realistic situations: documents from suppliers, personal data in logs, a support-desk assistant, product descriptions, meeting minutes, a digest of work chats, and sales calls. Each one has its own choice and its own set of facts. Look at each setup critically from the very start: what's missing that you'd need in order to give an honest answer instead of guessing?
03 · AI or plain code?¶
The first question today: when a task is worth solving with plain code, and when you genuinely can't do without a model. Engineers often decide this by feel, by a gut sense of how complex the task looks, rather than by any concrete signal they could put into words. And the cost of getting it wrong is real in both directions. Putting AI where plain code would have been enough means ending up with a solution that's more expensive and less predictable than it needs to be: code either runs correctly or fails loudly with an error, while a model can quietly produce plausible-looking nonsense and never warn anyone about it. Putting plain code where a model is actually needed means signing up for perpetual patching: every new variation in the input breaks the rigid logic, and someone has to keep fixing it over and over. There's no single recipe of "always pick X" — there are signals inside the task itself that are worth learning to notice. Let's move to the setup.
04 · Accounting is drowning in vendor invoices — can this be automated?¶
Read the product manager's quote in full, without shortening it — students should pick out for themselves what's actually relevant to the task. Direct signals: three people, around 400 invoices a month, all keyed by hand into 1C, backlog and errors by the end of the quarter, two sample invoices where the formats "look about the same." There's an indirect signal worth pulling out separately in discussion: the company already has a contract and live experience running an LLM over an API (marketing and YandexGPT) — that doesn't directly change the architectural choice for this task, but it does close off the question "can we even send data through an external API in the first place" before it comes up as a hypothetical risk. The mail migration is a decoy fact: it sounds like an IT detail about infrastructure, but it has nothing to do with parsing invoices. Show both sample invoices on the slide and the third card — the target table in 1C with the same fields (Date / Invoice No. / Line item / Amount) — so the audience immediately sees the structural match between source and target. Ask the room: what would you propose? Is AI even needed here, or can you get away without it? Which fields specifically would you extract, and what, in your view, determines how hard this task actually is? Who wants to go first? During the debrief, also ask: which facts from the quote actually shaped your choice, and which didn't?
05 · They sent over a 4.2 GB ZIP — and the formats don't match¶
Read the accountant's quote. A direct signal, not flagged by intonation: inside the archive it's a mix of scanned reconciliation acts, contracts, and the invoices themselves, nothing was pre-sorted. Indirect signals worth calling out explicitly: part of the flow (the two biggest suppliers, via EDI) already arrives as structured XML — that chunk doesn't need LLM extraction at all; and the scan quality (200dpi from old scanners) is a hard lower bound on OCR accuracy if the solution goes through image recognition. Accounting's move to EDI with the tax authority is a decoy fact: it sounds related, but it's a separate process about tax invoicing, unrelated to parsing supplier invoices. The slide shows several sample invoices from the archive — the formats are no longer as uniform as the first two documents: one looks like a plain table, in the second the fields are named and arranged differently, the third has free text instead of a clear table, and the fourth is in English because one of the suppliers is foreign. This is the same archive, just now we're seeing all of it. Ask the room: does this change your answer? And if you argued for a clean parser with no AI — what do you think now, seeing this spread of formats? And what about the share of the flow that comes through EDI and doesn't need parsing at all? During the debrief, ask: which facts from the quote actually shaped your choice, and which didn't?
06 · Hybrid: the LLM extracts, code validates. Study the data, don't take anyone's word for it¶
Look at what happened: while we were only seeing two identical documents, plain code looked sufficient and even preferable — it's predictable. As soon as the real 4.2-gigabyte archive showed up, it became clear that the formats had actually drifted apart, and a plain parser would have needed constant patching for every new variation. The sensible answer here is a hybrid: the model does extraction over unstructured or heterogeneous input, and then hard code validation follows — checking field types, required values, ranges — before the data ever reaches 1C. And the main lesson of this case is broader than documents, so I'll say it as its own sentence: study the source data as thoroughly and broadly as you can — and don't take the client's word for it. The product manager was confident the formats were the same because they'd seen two documents and decided that was enough. Before making an architectural choice, you need to request a picture of the full volume of data the solution will actually face, rather than relying on what you were told about it verbally.
07 · Personal data in logs — how do you clean it up as fast and cheap as possible?¶
Read the colleague's message — they're preparing the logs for the contractor. Direct signals: 40 GB of logs for the quarter, Friday deadline, the contractor needs request structure/timings/errors but not specific people's data, and inside the logs there's a mix of a phone number, an email, and a phrase like "pass this to Ivan in accounting." An indirect signal worth pulling out separately: the logs are already line-delimited JSON — meaning the field boundaries and request structure are already deterministic, so you only need to search for personal data inside the values, not parse the log format itself from scratch. The VPN testing is a decoy fact: it sounds like an adjacent data-security topic, but it has no bearing on how you'd scrub personal data out of an already-exported archive. The sample log line on the slide shows exactly that mix: a phone number, an email address, a name inside free text. Ask the room: what's the fastest, cheapest way to clean up these 40 gigabytes by Friday? Who has an opinion on what you'd try first? During the debrief, ask: which facts from the quote actually shaped your choice, and which didn't?
08 · Regex for phone numbers and emails, a compact NER model for names in free text¶
Phone numbers and emails are data with a clear, predictable format — a plain regex scrubs them reliably and cheaply, no model is needed here and one would only slow down processing 40 gigabytes of logs. But people's names and indirect mentions in free text — like "pass this to Ivan in accounting" — require understanding context. This doesn't call for a large LLM either: there are compact named-entity-recognition models for this, for example slovnet from the Natasha project — the model weighs around 30 megabytes, it runs comfortably on a regular laptop's CPU with no GPU, at a speed of roughly 25 articles per second. The model recognizes people's names, organization names, locations — including company names and cities. By size, that's roughly 60 times smaller than a comparable BERT-class model, at a quality loss of 1-2 percentage points. The takeaway is worth stating explicitly: a large share of text tasks are solved algorithmically or with small specialized models — that's simpler, faster, and cheaper than running everything through a large LLM. The same question of data sensitivity will come up again in the sales-calls case — where the stakes are noticeably higher.
09 · When a formula was needed — and when AI closed a gap nothing else could¶
Two cases from real practice. In January 2023 it came out that the outlet CNET had been publishing finance articles generated by an AI tool for several months. Of 77 such articles, 41 needed corrections. The flagship example of an error: one of the articles claimed that "10,000 dollars at 3% annual interest earns 10,300 dollars in a year" — instead of the correct 300 dollars in earnings, the model confused the total amount with the earnings themselves, a basic arithmetic error. The lesson specific to this one: the calculation should be done by a deterministic formula, not an LLM "eyeballing it" — wherever exact arithmetic is needed, the model should replace the text around the calculator, not the calculator itself. What went wrong here, in your view? Now the positive example: an engineering blog post from the company Ramp describes how requests to fix transaction classification used to be handled manually, with coverage of only 1.5-3% of requests — meaning the overwhelming majority went unprocessed. After rolling out an LLM agent — under 10 seconds per request, around 99% of decisions judged correct, around 25% of requests the agent rejects. It's important to say plainly: this isn't a win for AI over deterministic code, it's a replacement for manual processing that had negligible coverage.
10 · Buy (build in) or build your own?¶
The second question today: build the solution into what people already use every day, or build a separate product for it. On paper, both options look reasonable. A built-in solution promises seamlessness; a separate product promises freedom — its own interface, its own development pace, no constraints from someone else's system. And each option has a hidden cost that's almost never visible in a demo or on a pitch slide — it shows up later, in real operation, once actual people start using the solution in their actual workflow, not the prototype's authors at a showcase. That's exactly why these decisions can't be made on the beauty of the concept alone — you need to dig into the specific circumstances of the company and the task. Let's move to the setup — it comes from someone who's accountable for the outcome, not for how nice the idea looks on a slide.
11 · An AI assistant for support agents — suggested replies, search for similar tickets¶
Read the whole quote out. Direct signals: the idea of response suggestions and similar-case search, 12 operators on the line, around 300 tickets a day, peak from 10:00 to 13:00. There's an indirect signal worth explicitly pulling out in discussion: part of the operator team already works in Telegram in parallel — that's a second, separate integration point for the assistant that the client didn't ask about directly, but it really exists and could turn out to matter as much as the helpdesk itself. Corporate SSO is also an indirect plus: a single login reduces the cost of integration if the assistant ends up as a separate window. The laptop-fleet refresh is a decoy fact: it sounds like an IT detail about support infrastructure, but it has nothing to do with the assistant's architecture. The slide shows a support-desk operator's screen: a ticket list, a customer-reply window. Operators spend the entire workday in the same helpdesk-system interface — something like Zendesk or Jira Service Management — opening ticket after ticket, switching between cases dozens of times a shift. Ask the room: where should this assistant live? What would you propose — a separate window, a suggestion built right into the interface, something else? And just as important, why that and not something else? During the debrief, ask: which facts from the quote actually shaped your choice, and which didn't?
12 · An off-the-shelf helpdesk with no open API — the system was bought 4 years ago¶
Read the IT director's quote in full, without flagging the key words by intonation. Direct signals, sitting inside the paragraph rather than in the first sentence: no open API, no customization options, the system was bought 4 years ago, HR's and facilities' tickets run through it too, the contract with the vendor still has 2 years left. There's an indirect signal worth pulling out explicitly: the Telegram bot integrates more easily than the closed off-the- shelf helpdesk — meaning the second integration point from the earlier quote (operators on Telegram) now has a lower integration cost, and that changes the balance between options, even if the audience doesn't bring up Telegram themselves without a nudge. The corporate-portal redesign is a decoy fact: it sounds like an IT initiative of the same kind, but it has nothing to do with integrating into the helpdesk. Ask the room: what do we do? Notice I'm not telling you whether this is good or bad for your plan — I'm just stating a fact you'd only have learned if you'd asked IT yourself, ahead of time, before you started designing a solution. Who wants to go first with what you'd propose now? And for anyone who a minute ago suggested building the assistant right into the interface — what do you think now that it's turned out the interface itself is closed off to you entirely? During the debrief, ask: which facts from the quote actually shaped your choice, and which didn't?
13 · Bolt it on the side, switch vendors, or rethink the task — there's no single right answer¶
This case has no single right answer, and that's deliberate. Look at three real options. The first — a side solution: a browser extension or a second panel next to the helpdesk interface. The cost — the operator has to physically look at two places, and under load they simply won't check it. The second — switch the helpdesk vendor to a system with an open API. The cost — migrating historical data, retraining the entire support team, the risk of downtime during the transition, all of that for the sake of one feature. The third — rethink the task itself: the assistant doesn't insert itself at the moment the operator replies, it prepares drafts or summaries for the ticket ahead of time. There's a cost here too, just a different one — the operator opens the finished draft wherever it's convenient, but a separate process is needed to prepare those drafts. In practice, "off to the side" gets chosen when rollout time is critical and the team is small — it's faster to put together an extension than to wait on the vendor. "Switching vendors" gets chosen when the contract is ending soon anyway — then the migration cost gets absorbed into an event that was already planned. And "rethinking the task" gets chosen when the organization isn't ready to spend budget on integration right now, but can wait on live suggestions.
14 · A standalone AI service for product descriptions — good idea?¶
Read the marketing quote in full. Direct signal: marketing specifically wants a separate standalone service, modeled on the competitor's, catalog of 12,000 SKUs. There's an indirect signal worth pulling out explicitly in discussion: the copywriter already writes descriptions right inside the shared catalog admin panel — meaning marketing already has a single workspace where a button could be inserted, and that changes the answer to "separate service or not" even before the formal debrief. The homepage banners and the CRM rollout in another department are decoy facts: both sound like "some other IT project at the company," but neither affects the architecture for product descriptions. The slide shows a mockup of a separate new service: an input form, a "generate" button. Ask the room: is a separate service a good idea, or not? What do you think, and why exactly — what in this setup actually determines whether a new tool is worth building at all? During the debrief, ask: which facts from the quote actually shaped your choice, and which didn't?
15 · A button in the existing catalog admin panel, not a standalone service¶
This should be a "generate description" button right inside the existing product-catalog admin panel that marketing already uses every day, entering new products and editing product cards. A separate product means a second login, a separate tab, a habit that has to be built from scratch, and an extra step in the workflow. In practice, separate tools like this often die from disuse, even when the technology inside them works perfectly well — simply because people never quite get around to using them when it's easier to just write by hand what they've always written for years. A separate service is still justified in one case — if there's no admin panel of your own at all, meaning marketing has no single place where they already work with product cards. Then there's nowhere to build the feature in, and a separate tool isn't a compromise — it's the only option available.
16 · 17 months in the same market — Kite shut down, Copilot reached 90% of the Fortune 100¶
Look at the chronology. Kite — a standalone, independent product for AI-powered code autocompletion — was founded in 2014, and from 2019 could run the model locally on the developer's own machine. GitHub Copilot entered technical preview on June 29, 2021, and became generally available on June 21, 2022, at 10 dollars a month. Kite shut down on November 21, 2022. Counting from Copilot's technical preview to Kite's shutdown, that's around 17 months where both products genuinely existed on the same market at the same time, and the competition was real, not hypothetical. That said, the reason for the shutdown should be stated carefully: Kite's own founder explained it not as a win for one specific product, but as the fact that "developers don't pay for standalone tools" — that's his statement about the market as a whole, not a direct claim that "Copilot killed Kite." Kite had around 500,000 users at its peak. Copilot has around 20 million users as of July 2025, out of more than 100 million developers on GitHub, 4.7 million paid subscribers as of January 2026, and it's used in roughly 90% of Fortune 100 companies. A comparable technology, but built directly into the IDE and into the ecosystem developers were already connected to.
17 · A standalone device versus a feature in the phone¶
Another example of the same choice, but at a different scale — not a feature inside an app, but an entire standalone physical device. The Humane AI Pin — a screenless wearable badge with a laser projector that displays an image right onto your palm, voice control, priced at 699 dollars plus a 24-dollar-a-month subscription. Sales started in April 2024. The company raised around 241 million dollars in investment, and in February 2025 the company's assets — not the whole company, specifically the assets — were sold to HP for 116 million dollars. On February 28, 2025, the device's servers were shut off, and it stopped working for everyone who had bought one. The Rabbit R1 is a similar story: around 100,000 units sold, but by some estimates only around 5% of buyers actively use it. The same organizational lesson as with Kite: a standalone device or product competes not only with alternative technologies, but with the user's habit of not switching to a new entry point when the function they need already lives on the phone that's already in their pocket.
18 · A one-off call, RAG, or an agent?¶
The third question today: what architecture is needed — a one-off model call, a search over a knowledge base, or a multi-step agent. These three options sound like three different levels of complexity, and intuitively it feels like the more impressive the task, the more elaborate an architecture you should build in from the start. That intuition is worth testing rather than taking on faith — like any feeling that "a serious task needs a serious tool," it has both supporting and disproving examples in real practice. Before we move to the setup, let's briefly cover what RAG is and what an agent is — we'll need both terms in just a couple of slides, and Lecture 3, where they're covered in detail from the inside, is still a way off, so for today a general intuition is enough: what each architecture does and how they differ from each other at a high level, without the details of how they're built.
19 · RAG in one flow: question → search over the base → snippets into context → answer¶
Before we move to the next case, a word on RAG — you'll need this term right now, and it'll keep coming up through the rest of the session. Here's how it works: the user asks a question → the system searches for relevant fragments in a knowledge base → the fragments it finds get dropped into the model's context → the model generates an answer based on the question and what it found. RAG is needed when the knowledge base is large, changes over time, and its full volume physically doesn't fit into a single request to the model — meaning the model literally can't "read" all of it at once, it needs to find just the relevant piece ahead of time. How the search itself works inside — embeddings, indexing — we'll cover that in detail in Lecture 3; for today this intuition is enough to recognize the architecture when you need it, instead of reinventing it from scratch every time.
20 · An agent as a loop: plan → tool → check → next¶
And second — agents. Here the model doesn't work as a single call, it works in a loop: it builds a plan, calls an external tool — that could be an API, a search, running code — checks the result of that call, decides what to do next, and repeats several times in a row until the task is done or it needs a human to step in. The key difference from RAG — here the model isn't just answering a question based on retrieved fragments, it's actually doing something in the outside world and checking whether it worked. The details of how an agent plans internally — again, Lecture 3. For today, only one thing matters: recognizing when a task needs this kind of loop with tools, rather than a single answer from the model. Keep these two diagrams in mind — RAG and agent — you'll need both in just a minute, in the next case.
21 · After the call, the transcript arrives by email — we need minutes: decisions, action items, deadlines¶
Read the quote in full. Direct signals, not flagged by intonation: after every call, the full transcript arrives by email, minutes are needed with decisions, action items, and deadlines, about 15 meetings a week, right now the project manager writes the minutes by hand. Indirect signals worth pulling out explicitly, even though they don't affect the v1 architecture right now: some meetings are in English — the transcript is still one coherent document, language doesn't change the v1 schema, but it'll come up as a detail later; and the fact that YouTrack has an open REST API — that lays groundwork for v3 (an agent that creates tickets itself), even though it's too early to talk about that now. The minutes are already saved to the corporate drive too — also a setup for the future (v2, storage), but it doesn't change the answer right now. None of this is noise in the strict sense, but none of it should change the answer to the first question either: what comes in is still one complete transcript of a single meeting, of a reasonable length, and what needs to come out is structured decisions, action items, and deadlines. Ask the room: what architecture is needed here? Argue for it — what specifically in this setup points you toward this choice and not another? Who wants to answer first? During the debrief, ask: which facts from the quote actually shaped your choice, and which didn't?
22 · Schema v1: email with transcript → model call with a template → minutes¶
The entire transcript of one meeting fits into the model's context in one go — which means we need a single one-off model call, no RAG, no agent required, and either one would be unnecessary complexity. The schema is simple: an email with the transcript comes in as input → one model call with a preset template ("pull out decisions, action items, deadlines") → out comes a finished set of minutes. No storage, no loop checking intermediate results — because the task doesn't call for it. Everything the model needs to answer fits into a single request in full, and the result comes back right away, with no intermediate steps and no calls to any external systems. This is the simplest option there is, and it's fully justified by the task as currently stated — complicating it now would mean solving a problem that simply doesn't exist yet.
23 · Minutes pile up in a shared folder — what did we decide about vendor X in May?¶
Read the line out in full, without leaning on the key part with your intonation. The direct signal is dissolved in the story about the standup: minutes pile up in a shared folder, and the question "what did we decide about vendor X in May" gets searched by topic and rough date, not by an exact document — this is no longer one meeting, it's a lot of accumulated minutes. The indirect signal is worth spelling out explicitly: the rate at which the folder is growing (half a gigabyte a quarter) isn't just "the archive is already big" — it's evidence that the base will keep growing, so the solution needs to be designed for a growing volume, not the current one. The vacation schedule and the portal redesign are noise twins — both came up in that same line, but neither has anything to do with searching the minutes. Question to the room: does this change the architecture? What exactly changed compared to what we were discussing a minute ago? Give yourselves a second to match this new request against the solution you proposed for a single meeting. Who wants to go first? And a separate question for those of you who a minute ago were arguing for a single model call: who's changed their mind now — and why exactly did this one detail make you reconsider your answer? When you break it down, ask: which facts in the line affected your choice, and which didn't?
24 · Schema v2: a growing archive calls for RAG — search over the base layered on the one-off call¶
As long as it was one transcript of one meeting that fit entirely into the model's context, a one-off call was the right and sufficient answer — neither RAG nor an agent would have been needed, and either would only have complicated things for no benefit. But now the base of minutes is growing over time, constantly filling up with new meetings, and the whole volume physically no longer fits into a single model request. So we need search over this growing base for an arbitrary user query — and that's exactly what RAG is for: finding relevant snippets of minutes by the meaning of the query, rather than trying to feed the model the entire archive every single time. Notice on the schema — the old path hasn't gone anywhere, we've just added storage, indexing, and search before the needed snippets reach that same model call we had at the previous step. We didn't pick RAG ahead of time "just in case" — we arrived at it only once a concrete requirement showed up that the simple option could no longer handle.
25 · Half the action items from May's minutes still have no owner¶
Read the line out in full. The direct signal, not particularly highlighted: the system already finds the action items in the minutes, but they're still moved into the task tracker manually, one by one, and half of May's action items are still sitting without an owner. The indirect signal here is the key one for this step of the breakdown, spell it out explicitly: the fact that YouTrack has an open REST API is exactly what makes the next architectural step — an agent calling the tracker's API — technically cheap rather than hypothetical. Without an open API you'd have to hack something together through export-import or write an integration from scratch. Tag colors in the tracker are a noise twin — also about the task tracker, but a purely cosmetic question, no bearing on the architecture. Question to the room: does this change the architecture again? What's new here compared to the search over the base we were just discussing a minute ago? Who's changed their mind — and more importantly than the fact of changing your mind, why exactly did this detail make you reconsider the architecture rather than just tack one more step onto the previous solution? When you break it down, ask: which facts in the line affected your choice, and which didn't?
26 · Schema v3: an action in an external system — you need an agent on top of RAG¶
Now there's an action in an external system: you need to call the task-tracker API, create tasks, verify that they were created correctly, with the right owner and deadline. A single model call, or even search over the base, doesn't cover that — you need a workflow or a full-fledged agent that calls an external tool and verifies the result of the call. Look at the whole schema: the earlier path with storage and search hasn't gone anywhere, it's just been extended on the right with new blocks — extract the action item, call the API, create the task, verify the result. Every extension was paid for by a new requirement: the one-off call was justified as long as everything fit into one transcript; RAG showed up once the base grew; the agent showed up once an action in an external system was needed. The architectural choice was driven by concrete new requirements, not by a fashion for a technology. Start with the simplest option that solves the current task, and add complexity only once a requirement shows up that the simple option physically cannot handle. Not the other way around.
27 · A digest of chats and email — what architecture does that need?¶
Read the line out in full. The direct signal, not particularly highlighted: we need specifically a weekly (scheduled, not on-demand) digest of a few specific sources — the work chat and email — once a week by 5 PM on Friday, and the information needs to come together into one coherent message. It's worth pulling out the indirect signal: both sources (chat and email) already have an API for pulling history — so the technical ability to collect data on a schedule already exists, the only question is what architecture processes what's collected. Moving to a different messenger and testing the VPN are noise twins — both about the company's communication infrastructure, but neither affects the digest's architecture. Question to the room: what architecture does this need — a one-off model call, RAG, an agent, or something else? Argue it, don't just name a technology — explain why you picked it, and what specifically in the setup pointed you toward that idea. Who's ready? Argue your choice. When you break it down, ask: which facts in the line affected your choice, and which didn't?
28 · You need an agent — to go through the sources; you don't need RAG — no arbitrary query against a large base¶
You often hear "you need RAG because there are many sources" here — and that's not right, even though the intuition is understandable. You need an agent or a pipeline, because someone has to go through several sources on a schedule — check the chat, check the email, collect the messages — and that's a multi-step task involving calls to external systems. RAG, on the other hand, isn't needed here: the sources are crawled directly on a schedule, not on an arbitrary user query, and once the material is collected, the summarization is done with a single model call, with no search over an archive. RAG is specifically about search over a large, changing base on an arbitrary user query, not about "we have a lot of data sources" by itself. Mixing up these two signals is a common practical mistake worth remembering separately from the case itself.
29 · RAG and an agent in production: Morgan Stanley — success; Replit — failure¶
Two real-world cases of RAG and agents already in production. The positive one — Morgan Stanley's AI assistant: the partnership with OpenAI was announced in March 2023, the assistant itself launched in September 2023. It's RAG over a base of roughly 100,000 of the company's research documents. By mid-2024, the share of financial-advisor teams using the assistant passed 98%. And the key metric — the findability of the right document for an advisor rose from 20% to 80%: previously they'd find what they needed in roughly one case out of five, now it's four out of five. The negative example — an incident at Replit in July 2025: an AI agent deleted a production database, affecting the data of roughly 1,200 company executives, and it did this during an announced code freeze — a freeze on code changes — against a direct instruction not to touch production. After that, the agent also incorrectly claimed that a rollback was impossible, even though the data was in fact restored. The company's CEO publicly apologized. After the incident they introduced an automatic dev/prod environment split and a special "planning-only" agent mode, with no permission to run destructive operations. And one more example in passing: the company Octomind pulled the LangChain framework from production after a year of use and went back to direct API calls — on June 17, 2024.
30 · External API or local inference?¶
The last, fourth question today: process the data through an external cloud API, or on your own infrastructure. This is about where the data is physically processed, not about which model gives a better-quality answer — both models can produce a similar result, and the difference turns out to be in exactly where the request goes and what happens to the content at the other end. An external API gives you maximum power with no investment in hardware; local inference gives you control over what leaves the company's perimeter, and independence from someone else's service. You have to weigh these sides not in the abstract, but against the concrete data of a concrete task — there's no one-size-fits-all answer here for different companies. As before, we'll start with the task stated in one line by the client.
31 · A call card: need, objections, agreements — how do you build that?¶
Read the sales manager's line out in full. The direct signal, not particularly highlighted: they want a summary card per call (need / objections / agreements), 8 managers, 15-20 calls a day each. Indirect signals worth pulling out explicitly: the phone system already drops recordings into the cloud on its own — no need to solve the extraction problem separately; the CRM is Bitrix24 with an open REST API — the card can be written straight into the CRM with no custom integration; and the analysts' existing habit of using a cloud AI over an API is an indirect signal that the organization is already psychologically ready for an external API, and this won't be a new class of risk for them per se (though call data isn't the same thing as report drafts — we'll come back to that question later). Duplicate contacts in the CRM are a noise twin — also about the CRM, but purely about data quality, not about the extraction architecture. On input — a call recording, an ordinary free-form conversation between a manager and a client, with no structure at all, with interruptions, pauses, digressions from the topic, sometimes with background noise. Question to the room: how do you build this? What would you propose, and why exactly that? Who'll start? When you break it down, ask: which facts in the line affected your choice, and which didn't?
32 · Personal data in the recordings — and a legal map of three jurisdictions¶
A lawyer walked into a prototype demo and heard a recording: a client dictating their passport details for a delivery contract. "Wait. Which service are you sending these recordings to?" None of the engineers had thought about that in advance — everyone was only listening for needs and objections, because that's exactly what the sales manager had asked for. Look at the legal map. In Russia the base personal-data law is 152-FZ, and as of May 30, 2025, the 420-FZ amendments are in force: for a first breach — a flat fine of 3 to 5 million rubles, for a repeat breach — a turnover-based fine of 1 to 3% of the company's annual revenue, minimum 20 million, maximum 500 million rubles. In the European Union, under GDPR — up to 20 million euros or 4% of global annual turnover, whichever is greater; the record fine went to Meta — 1.2 billion euros in 2023. In the US there's no single federal law protecting personal data — there are state laws, for example California's CCPA and CPRA, and sector-specific ones, like HIPAA for medical data. One takeaway: every major jurisdiction is seriously fighting to protect personal data, and this isn't an abstract risk, it's concrete amounts written into the laws of different countries.
33 · No GPU server and no budget. We need five fields in the CRM, not a conversation¶
Read both lines out one after another, without highlighting the key words with your intonation. The direct signals are dissolved in the context: the CTO says there's no GPU server and no budget for one this year, meaning heavy local infrastructure for a heavy model is off the table; the sales manager needs exactly five specific fields in the CRM, not a conversation and not tone-of-voice analysis. It's worth pulling out the indirect signal from the sales manager's line: the fields need to be written straight into Bitrix24 — that's the same CRM with the open API we already mentioned in the setup, which means the solution's output has nowhere new to be plugged in, the integration point is already known. The email migration and laptop refresh are noise twins — both about this quarter's IT budget, but not about model inference. Question to the room: does this change the architecture again? What from earlier should now be reconsidered, and how do these two lines together, not separately, change the picture? When you break it down, ask: which facts in the lines affected your choice, and which didn't?
34 · The task narrowed down to a template — a local 7-8B-parameter model handles it¶
The task actually narrowed down: it doesn't need a free-flowing conversation, it needs a structured summary against a fixed template of a few fields — that's extracting fixed fields against a template, not a free conversation requiring context understanding. Here's exactly what the engineer dug up while looking into this: this class of task is well covered by current open models in the 7-8 billion parameter range — Qwen3, Llama, Gemma, Mistral, as well as the Russian T-lite and T-pro, built by T-Bank under the Apache 2.0 license, based on Qwen. 4-bit quantization — compressing the model's weights down to 4 bits per parameter — lets you fit such a model into 4-6 gigabytes of memory and run it on an ordinary server with no GPU farm, using tools like Ollama or llama.cpp. If you need to fine-tune the model more precisely for the output format, you can fine-tune it with LoRA, an economical fine-tuning method that doesn't touch all of the model's weights, and do it on a single GPU. Let's spell out two lessons explicitly. First — "hard for a human" doesn't mean "hard for a model": a free-form conversation looks like a hard input, but extracting structured fields against a template is a task of moderate complexity that a small model handles reliably. Second — data sensitivity isn't visible in the initial task statement; it has to be uncovered by asking questions about the content of the input data, not assumed from the wording.
35 · 19 days from allowing ChatGPT to three leaks at Samsung. Small models are already in your pocket¶
Samsung allowed employees to use ChatGPT on March 11, 2023. By March 30 — roughly 19 days later — three separate leak incidents had occurred: twice proprietary semiconductor-manufacturing code, once an internal meeting transcript. The response — a full ban on generative AI tools on corporate devices in May 2023. What could have been done instead of a full ban? An acceptable-use policy before opening access, filters on outbound traffic, or a local model from the start for sensitive tasks — exactly the choice we just practiced. Now for the positive examples of small models already running locally right now, not in a lab. Apple Intelligence — a model of roughly 3 billion parameters right on the device, available in iOS 18.1 since October 2024, on iPhone 15 Pro and newer: text summarization and text editing work fully offline, with no cloud connection. Google Gemini Nano — a model of 1.8-3.25 billion parameters in Pixel and Android devices: summarizing voice-recorder notes in the Recorder app, the Magic Compose feature for smart message autocompletion — both also work offline. Small models are already in the pocket of a large share of the room right now — most people just haven't thought about it.
36 · Does anyone here have a similar task from work?¶
Does anyone here have a similar task from work — one where the question was whether AI was needed at all, and which kind? You don't need a finished solution, it's enough if you can state it in two or three sentences, exactly the way a client would bring it to you, with none of the polish of a textbook example. We'll work through it as a whole room, the same way we did today's cases: which choice is sharpest here, what inputs are missing to answer it with confidence, and what happens if those inputs suddenly show up or change. If you already know how the task actually turned out in practice, tell us that only after the room has offered its guesses, not before. Who's ready to share? If no one speaks up right away, I have a backup story, but I'm genuinely more interested in working through something from your own experience.
37 · What to take away: lessons from seven cases¶
Let's gather up what's already come up today — nothing new, just a compilation. Vendor documents — study the data as thoroughly and broadly as you can, don't take the client's word for it, two examples still aren't the full picture. Personal data in logs — algorithms and small specialized models often solve a text task faster and cheaper than a big LLM. Support assistant — an organizational constraint like a closed API can settle the architecture just as well as technical requirements. Product descriptions — a button in an existing interface almost always beats a standalone product. Meeting minutes — concrete new requirements drive the architecture, not fashion for a technology, start simple. The digest — lots of data sources doesn't mean you need RAG. And sales calls — what looks hard to a human isn't necessarily hard for a model, and data sensitivity is only uncovered by asking, not by assuming. That's what's worth taking away from today's session.
38 · Homework reading: Klarna, NYC MyCity, bonus — the $1 Tahoe¶
For anyone who wants to dig deeper — three pieces. The main one is the story of Klarna, 2023-2025: one AI support assistant that touches on almost every choice from today, from an impressive launch to the CEO's public admission "we went too far" and a return to hiring people. The second — the story of New York City's MyCity chatbot, which in 2024 gave small businesses legally incorrect advice. And a bonus for the mood — the story of how the car dealership Chevrolet of Watsonville hooked up a third-party API chatbot with no guardrails, and it "sold" a Tahoe SUV for one dollar. A good example of what happens when you embed an external model into production with no protection against user manipulation. If there's time left today, we'll work through part of this live.
39 · Klarna, 2023-2025: one story — four forks in the road¶
Klarna launched an OpenAI-based AI support assistant in February 2024, right inside the existing support channel. In the first 30 days — 2.3 million chats, 67% of requests automated. Look at all four forks at once. Routine tickets like order status or returns are a formalizable, repeated task, a good candidate for automation. The assistant is built into an already-existing support channel — the right choice by today's session's logic. The architecture is chat with escalation to a human, not an autonomous agent with no oversight. But the inference — an external cloud API with support customer data — is a decision with a risk profile comparable to our sales-calls case, except here they accepted it and left it in production. Separately — the company claimed an effect "the equivalent of 700 agents" and $40 million in 2024, and the denominator — how many support agents the company had before the launch — was never publicly disclosed. By Q3 2025 the metric had updated to "853 agents" and $60 million in effect, and the denominator again wasn't disclosed. In May 2025 the CEO, Sebastian Siemiatkowski, told Bloomberg in an interview: "We went too far" — and the company went back to hiring people, moving to a hybrid model. Which of the four forks do you think Klarna got wrong the worst?
40 · News deduplication — you need to merge messages about the same event into one card¶
Read the product manager's line out in full. The direct signal, not particularly highlighted: a stream of news from many sources, need to merge messages about the same event into one card. The indirect signal is worth pulling out: all the sources are already normalized through a single RSS aggregator, meaning the headline and metadata format is uniform — no need to solve the problem of parsing different formats, only the problem of comparing content. The source-card redesign is a noise twin — it looks like a change to the same feed interface, but has nothing to do with merging duplicates. Question to the room: what technology do you need here? Intuitively, a lot of people right now will say — you need to understand the meaning of the text, so you need a model, and you'll have to run every pair of news items through an LLM, comparing them pairwise. Do you agree with that, or does anyone in the room already have doubts on that front? Think about how many such pairs of news items you'd need to compare every day with a large stream of sources. When you break it down, ask: which facts in the line affected your choice, and which didn't?
41 · Fuzzy matching handles ~90% of cases cheaper and more predictably. The LLM handles the rest¶
As it turns out, comparing texts by overlapping fragments — fuzzy matching and shingling — with no call to a model covers roughly 90% of duplicate cases, and does it cheaper and more predictably than calling a model on every pair of news items: the result is deterministic, easy to debug, and doesn't depend on the model's mood. A model is only needed for the remaining part — cases where two messages about the same event are written in completely different words and share almost no text fragments, meaning it's more of a paraphrase than a match. It's the same lesson we had at the very start of the session with personal data in logs: a cheap deterministic filter covers the bulk of the cases first, and the model handles only the remainder the filter physically can't catch.
42 · Today you decided what to build. Lecture 2 — how the model is built inside¶
Today we went through seven situations, and in every one of them you made a decision — what to build at all, which architecture to choose, where to process the data. Not once was the right answer obvious from the first statement of the problem — it always depended on the question you either asked, or didn't. That skill was the whole point of this session, not memorizing the specific cases. Further along in the course we'll go deeper: next lecture — how the model is built inside, what happens at the architecture level when you send it a request. Today's cases, I hope, gave you the intuition for why an engineer needs that knowledge, not just a researcher. Thanks for the lively discussion today, see you at the lecture.