Artificial Intelligence

Chapter learning objectives

1. Distinguish models, training, inference, prediction, and classification in a business workflow.
2. Evaluate a model using appropriate data and the consequences of different errors.
3. Explain prompts, context, retrieval, generation, and tool-using agents.
4. Design a governance rationale covering bias, privacy, security, oversight, and accountability.

Goals for this part

• Explain Models and Data.
• Explain Training and Inference.
• Explain Prediction and Classification.

Models and Data

Artificial intelligence (AI) refers to computational approaches that perform tasks associated with capabilities such as prediction, perception, reasoning, or language use. A model is a representation used to produce an output from inputs. Machine learning develops model behavior from data rather than specifying every relationship through hand-written rules.

Models and Data

A late-response model might use request type, arrival time, and queue length. It does not observe every condition affecting service. Missing staffing information or inconsistent time records can limit what it learns. The model's output is therefore about the task as represented by its inputs and training, not an independent account of organizational reality.

Models and Data

Features are input characteristics used by a model. Labels, when available, describe outcomes used in supervised learning. Defining “late” requires a service rule and reliable timestamps. That definition is a business decision that shapes the technical problem. Data quality from {doc}Data and Analytics <../06_data_and_analytics/0600_data_and_analytics> remains relevant when the analysis uses AI.

Training and Inference

Training adjusts a model using data and an objective. Inference applies the resulting model to new inputs. Evaluation needs data that were not used to fit the model, so that apparent success is not simply a reflection of examples it has already encountered.

Training and Inference

For a time-dependent service process, testing on a later period can help expose changes in demand or operating conditions. Evaluation must also avoid data leakage, where information unavailable at the decision time enters the training or test inputs. Using a final resolution timestamp to predict lateness at arrival would give the model an unrealistic advantage.

Training and Inference: comparison

Training and Inference

A successful test supports a decision about this use; it does not establish competence on every future task.

Prediction and Classification

A prediction estimates an unknown value or outcome. Classification assigns a case to a category, often using estimated probabilities or scores. A manager might classify a support request as requiring early review when an estimated delay risk exceeds a chosen threshold.

Prediction and Classification

Changing the threshold changes the balance between missed cases and unnecessary reviews. A false positive is a case flagged as positive that does not meet the target condition; a false negative is a missed positive case. The operational cost of each depends on the setting. Flagging every request can find many delays while overwhelming reviewers.

Prediction and Classification

Accuracy alone can conceal this problem. In a hypothetical set of 100 requests with five late cases, a model that always predicts “on time” is 95% accurate but identifies none of the late requests. Managers need measures tied to the purpose, the capacity to act, and the consequences of mistakes.

Prediction and Classification

The model supports a workflow; its score is not the final management outcome.

Goals for this part

• Explain Prompts and Context.
• Explain Content Generation.
• Explain Agents and Workflows.

Prompts and Context

Generative AI produces content such as text, images, or code from patterns learned during training and information supplied during use. A prompt specifies a request or instruction. Context is information available to the system when generating a response, including the current conversation or retrieved documents.

Prompts and Context

A clearer prompt can improve the specification of a task, but it cannot guarantee that the system has correct information or will follow every constraint. “Answer accurately” does not supply an updated policy.

Prompts and Context

A useful support request identifies the customer question, provides the approved policy text, and specifies what to do when that text does not answer the question.

Prompts and Context

Retrieval-augmented generation adds retrieved material to the context used for generation. It can make relevant organizational information available without retraining the model. Its reliability still depends on retrieving the right documents, keeping them current, and checking whether the answer follows from them. A displayed citation is a claim about evidence that a reviewer may still need to inspect.

Prompts and Context

A large language model (LLM) is a model trained on substantial language-related data to perform tasks involving sequences of text and, in some systems, other modalities. A token is a unit of representation used by a model; it need not correspond to a whole word. Token limits can affect how much material is available in one context.

Prompts and Context

An embedding represents an item, such as a text passage, as numerical values that support tasks such as similarity search. Similarity can help retrieve a relevant policy passage, but it does not establish that the passage answers the question or that it is current.

Content Generation

Generated content is not automatically a verified statement about the world. Confabulation, also commonly called hallucination in AI discussions, describes unsupported or false generated content presented in a plausible form. NIST's Generative AI Profile identifies this as a risk that organizations need to evaluate for their use.

Content Generation

For the support assistant, quality involves more than grammar. A reply must reflect the correct policy, address the customer's actual circumstances, avoid exposing information, and identify when staff review is necessary. A draft can be useful even when it is not safe to send without review.

Content Generation: comparison

Content Generation

The table turns “review the answer” into observable tasks. Evaluation should use representative cases, including questions the policy cannot answer.

Agents and Workflows

An AI agent is a system that uses a model within a workflow to select or carry out actions toward a goal, often by using tools. The surrounding application determines which tools exist, what permissions they have, and whether an action requires approval. A language model does not independently acquire authority to change business records.

Agents and Workflows

A support agent might retrieve a policy, prepare a draft, request approval, and then send an authorized reply. Issuing a refund adds a different capability with financial and customer consequences. The organization should evaluate and authorize that action separately from drafting text.

Agents and Workflows

Controls include limiting tool access, distinguishing external content from trusted instructions, checking action inputs, logging what happened, and providing a way to stop or reverse work when possible. Prompt injection occurs when untrusted content attempts to redirect the system's behavior. A message or retrieved document should be treated as task data rather than a source of permission.

Agents and Workflows

The practical unit of evaluation is the complete workflow, including people and tools.

Agents and Workflows

Automation assigns specified work to a machine or system. An assistant that prepares a draft automates part of content preparation; a system that sends the reply automates an additional action. These are different scopes of automation and need separate evaluation. The more consequential action should not inherit permission simply because the drafting task was approved.

Goals for this part

• Explain Bias and Data Quality.
• Explain Privacy, Security, and Controls.
• Explain Judgment and Accountability.

Bias and Data Quality

Bias in an AI setting can arise through the problem definition, available data, labels, model choices, and the process in which outputs are used. A dataset can accurately describe historical decisions while reproducing past disadvantages. Removing an obvious sensitive field does not necessarily remove patterns associated with it.

Bias and Data Quality

For a support-priority model, historical resolution times may reflect which customers previously received more attention. Training on those outcomes can reproduce an operating pattern rather than identify an impartial measure of need. Managers should examine the meaning of the label as well as the model's technical fit.

Bias and Data Quality

Evaluation should consider relevant groups, difficult cases, and people omitted from the data. Differences in outcomes need interpretation in the context of the service and its purpose. A single aggregate accuracy score can hide a serious failure affecting a small group. Appropriate review therefore requires knowledge of the work and the people affected, not only a model benchmark.

Privacy, Security, and Controls

AI systems inherit ordinary information-system risks and add risks associated with model inputs, outputs, and tool use. Information sent to a service may include customer records or confidential documents. Retrieved content may be outdated or adversarial. Generated output may reveal information or lead users to take an inappropriate action.

Privacy, Security, and Controls

Controls should follow the information and the action through the workflow. Limit access to relevant records, define permitted inputs, check the provider's handling arrangements, test representative failures, and record important model and configuration changes. A policy statement without an operating check is weak evidence that the risk is managed.

Privacy, Security, and Controls: comparison

Privacy, Security, and Controls

These are examples for analysis, not a complete security design. Controls need to match the consequence and context of the use.

Judgment and Accountability

A governance arrangement identifies the purpose of a use, who owns its risks, how performance is evaluated, and who may change or stop it. NIST's AI Risk Management Framework organizes risk-management activities under Govern, Map, Measure, and Manage. The framework supports contextual decisions rather than certifying that every system with a checklist is safe.

Judgment and Accountability

A meaningful human review needs time, relevant information, authority, and the ability to disagree. Requiring an employee to approve hundreds of unfamiliar outputs under severe time pressure can produce a formal approval step without dependable oversight.

Judgment and Accountability

For the assistant, managers should name the policy owner, the person responsible for evaluation, the reviewer for consequential actions, and the service owner who handles incidents. They should also define changes that trigger reevaluation, such as a new model, different records, or expanded action permissions. Accountability continues after deployment because the system's use and operating conditions can change.

Judgment and Accountability

Explainability concerns providing an account that helps the relevant audience understand an output or system behavior. A generated explanation can itself be inaccurate, so it should not be treated automatically as a faithful record of the model's internal process. Responsible AI refers to organizational practices that seek appropriate, accountable uses and responses to risks.

Judgment and Accountability

Model risk is the possibility of adverse consequences from a model's limitations, misuse, or operation in a changed context. A complete response needs more than an explanatory label on a screen.

Reading and assignments

• Chapter reading
• Curated references
• Preview, Case Study, and Homework