Data and Analytics

Chapter 6 · Strategic Information Systems

Lecture draft

From the content sections and aligned assignments.

Chapter learning objectives

  1. Describe entities, records, identifiers, and relationships in an organizational dataset.
  2. Distinguish warehouses, lakes, and operational storage by their intended uses.
  3. Interpret dashboard measures and distinguish descriptive evidence from predictive claims.
  4. Evaluate accuracy, completeness, consistency, timeliness, and data-quality controls.

Data

Chapter 6 · Data and Analytics

Read: Data

Goals for this part

  • Explain Data Resources and Storage.
  • Explain Data Models and Relationships.
  • Explain Ownership and Governance.

Data Resources and Storage

An entity is something an organization needs to describe, such as a customer, product, or order. A record describes an instance, and a field holds one attribute, such as an order date. A database organizes related records so that applications can store, retrieve, and maintain them under shared rules.

Data Resources and Storage

Separate files can be useful for a small task, but repeated copies create coordination problems. If a customer's address changes in one file and remains unchanged in another, different teams can act on conflicting information. Data redundancy is repeated storage of the same fact; inconsistency occurs when copies disagree.

Data Resources and Storage

A database management system provides mechanisms for defining, querying, and controlling database content. It does not decide what “customer” should mean for the organization. That definition remains a management and data-design responsibility. The retailer needs to distinguish an individual, an account, and a transaction before a storage choice can solve its reporting problem.

Data Models and Relationships

A data model describes the records, attributes, and relationships the organization needs. In a relational model, tables organize records into rows and attributes into columns. A primary key identifies a row, while a foreign key connects a record to a related record in another table.

Data Models and Relationships

For example, one customer can place many orders, so a customer identifier appears in the order records. Each order can contain several line items, so the line-item table needs an order identifier. Treating each line item as a separate customer would inflate the customer count.

Data Models and Relationships: comparison

Table Example identifier Meaning of one row
Customer Customer ID One customer according to the agreed definition.
Order Order ID One order placed by a customer.
Order line Order ID and line number One product line within an order.

Data Models and Relationships

Read the final column before combining tables or counting rows. A relationship can legitimately produce several rows per customer. Good modeling separates facts that belong to different entities, reducing contradictory updates and making later analysis easier to interpret.

Data Models and Relationships

Structured Query Language (SQL) is a language used with relational databases to define or work with data, including selecting records and combining related tables.

Data Models and Relationships

A manager need not write a query to specify its business meaning: “count distinct customer identifiers with fulfilled orders during the month” states a different request from “count order lines.” Master data are shared reference records for important entities, such as products or customers. Their consistent maintenance helps multiple processes refer to the same entities.

Ownership and Governance

Data governance establishes who may make decisions about data definitions, access, quality, use, and retention. A data owner is accountable for decisions within a defined area. A data steward often coordinates the practical work of maintaining definitions, resolving issues, and checking quality. Titles vary, so the assigned responsibilities matter more than the label.

Ownership and Governance

The retailer might make a customer-service manager accountable for the operational definition of a customer account, with a steward maintaining the data dictionary. Finance may require a different measure for paying customers. Governance should make those distinctions explicit rather than forcing every team to use one ambiguous count.

Ownership and Governance

A data dictionary records field meanings, valid values, sources, and relevant rules. Metadata describe the data, including their origin and update time. These records make data usable beyond the team that first collected them. Access should also match a defined purpose: availability to one department does not automatically justify reuse for every analysis.

Quick check

Name the concept that completes this statement.

An ________ is something an organization needs to describe, such as a customer, product, or order.

Quick check

Name the concept that completes this statement.

A ________ describes an instance, and a field holds one attribute, such as an order date.

Your turn

Use the chapter case: A dashboard with conflicting totals.

Create a data dictionary for five fields, including meaning, source, owner, identifiers, and update timing.

Prepare a brief explanation with a stated assumption and an outcome measure.

Recap

Define what records mean before selecting storage or counting results. Relationships, identifiers, ownership, and metadata connect data management to dependable organizational work.

Analytics

Chapter 6 · Data and Analytics

Read: Analytics

Goals for this part

  • Explain Warehouses and Data Lakes.
  • Explain Business Intelligence and Dashboards.
  • Explain Descriptive and Predictive Analysis.

Warehouses and Data Lakes

A data warehouse brings selected data from multiple sources into a governed structure for reporting and analysis. Preparation commonly reconciles identifiers, definitions, formats, and time periods. A data mart serves a more focused audience or subject area. These arrangements help analysts compare information that originated in different operational systems.

Warehouses and Data Lakes

A data lake can retain varied data, including material that is raw or only partly processed. Flexibility can support new questions, but usefulness still depends on metadata, access controls, ownership, and preparation. A collection of files without those arrangements can become difficult to interpret or trust.

Warehouses and Data Lakes

AWS's data-lake explanation supports this distinction; it is a service-provider perspective rather than a reason to adopt a particular product.

Warehouses and Data Lakes

For the retailer, a warehouse might contain agreed daily sales and returns by store. A lake might also retain service messages and device logs for future analysis. Neither automatically makes the resulting measures accurate. The preparation and governance work determines whether the data answer the intended question.

Business Intelligence and Dashboards

Business intelligence (BI) combines information preparation, analysis, and presentation to support organizational understanding and decisions. A dashboard presents selected measures so users can monitor conditions. A useful dashboard connects measures to a purpose, defines them consistently, and makes time periods and limits visible.

Business Intelligence and Dashboards

The retailer's sales dashboard needs to distinguish orders placed, orders fulfilled, and net sales after returns. A large total can conceal late data or a change in what is counted. Showing the refresh time and measure definition can be more useful than adding another chart.

Business Intelligence and Dashboards: comparison

Measure Definition to settle Interpretation risk
Customer count Distinct customers or distinct accounts? Duplicate accounts inflate the apparent reach.
Sales Orders placed, fulfilled, or paid? A canceled order may remain in the total.
Return rate Returns divided by which eligible sales? Mismatched periods distort comparisons.

Business Intelligence and Dashboards

Use the middle column before interpreting the right-hand problem. The visual display is the final layer of a chain that begins with the records and their preparation. {doc}Decision Making <../12_decision_making/1200_decision_making> examines how managers use such evidence.

Business Intelligence and Dashboards

Data visualization represents information through graphical forms such as lines, bars, or spatial layouts. The form should match the question. A trend line can reveal changes over time, while a comparison of categories may be clearer as aligned bars. Labels, scales, missing observations, and aggregation affect interpretation.

Business Intelligence and Dashboards

A visually striking chart with an unclear definition does not compensate for a weak measure or poor source data.

Descriptive and Predictive Analysis

Descriptive analysis summarizes observed conditions, such as weekly sales or the distribution of delivery times. Predictive analysis estimates an unknown outcome from patterns in available data. A forecast can support planning, but it remains conditional on the data, method, and setting in which it will be used.

Descriptive and Predictive Analysis

Suppose the retailer predicts next week's pickup demand from prior orders. A recent promotion or a store closure may change the relationship between past and future activity. Managers need to understand those differences rather than treating the prediction as a guaranteed count.

Descriptive and Predictive Analysis

Association also differs from causation. Stores with more staff may report more sales because staffing responds to demand, because staffing affects service, or because another condition influences both. A descriptive relationship alone cannot distinguish those explanations. Before recommending a change, identify the business question, the decision it supports, and what additional evidence would be needed.

Descriptive and Predictive Analysis

This keeps analysis tied to judgment rather than equating a more complex model with a better decision.

Quick check

Name the concept that completes this statement.

A ________ brings selected data from multiple sources into a governed structure for reporting and analysis.

Quick check

Name the concept that completes this statement.

A ________ serves a more focused audience or subject area.

Your turn

Use the chapter case: A dashboard with conflicting totals.

Select a public dashboard or an invented organizational dataset and state the decision it supports.

Prepare a brief explanation with a stated assumption and an outcome measure.

Recap

Analytics depends on preparation and interpretation as well as tools. Distinguish storage arrangements, define dashboard measures, and treat predictions as conditional evidence.

Data Quality

Chapter 6 · Data and Analytics

Read: Data Quality

Goals for this part

  • Explain Accuracy, Completeness, and Consistency.
  • Explain Timeliness and Validation.
  • Explain Controls and Decision Consequences.

Accuracy, Completeness, and Consistency

Accuracy concerns whether a recorded value represents the relevant reality correctly. Completeness concerns whether required observations or fields are present. Consistency concerns agreement among related representations. The dimensions answer different questions, and improvement in one does not automatically establish the others.

Accuracy, Completeness, and Consistency

A customer's recorded postal code may be accurate while the delivery address is incomplete. Two systems may consistently repeat the same wrong address. A sales file can contain every order but omit cancellations, making it incomplete for a net-sales calculation. Evaluate quality against a defined use rather than an abstract claim that a dataset is “clean.”

Accuracy, Completeness, and Consistency

For the retailer, duplicate customer accounts create another issue. Two valid accounts can represent the same person. Deleting one without understanding its transaction links may lose information. A correction needs rules for identifying duplicates, preserving histories, and documenting uncertainty. The business meaning of the record guides the technical treatment.

Timeliness and Validation

Timeliness concerns whether information is available and current enough for the task. A daily inventory snapshot may be appropriate for a trend report and unsuitable for a live pickup promise. The same dataset can therefore have adequate quality for one decision and poor quality for another.

Timeliness and Validation

Validation checks data against stated rules. A check can identify an impossible date or missing identifier, but passing the rule does not prove that a value is true. An incorrect address can have a valid format. Stronger checks may compare related records or reconcile an observation with an independent source.

Timeliness and Validation: comparison

Rule Problem it can detect Problem it may miss
Order date is required. An absent date. A plausible but incorrect date.
Product ID must exist. An unknown code. Selection of the wrong valid product.
Fulfilled quantity cannot exceed ordered quantity. A contradictory transaction. An unrecorded physical shortage.

Timeliness and Validation

The final column explains why validation needs an operating process for investigating exceptions. Rules create evidence for review; they do not remove the need to understand the work.

Controls and Decision Consequences

A data-quality control is an arrangement intended to prevent, detect, or correct a relevant data problem. Preventive controls might define required fields or standard identifiers. Detective controls might compare order and shipment totals. Corrective controls assign ownership for resolving mismatches and informing affected users.

Controls and Decision Consequences

Suppose returns reach the retailer's dashboard two days late. A useful response could show provisional totals, disclose the delay, and reconcile the final period after returns arrive. Hiding the gap creates an appearance of precision while giving managers less usable information.

Controls and Decision Consequences

Quality monitoring needs priorities. A misspelled internal label and an incorrect delivery address have different consequences. Managers should evaluate how an error affects the decision, how often it occurs, how quickly it is detected, and who must respond. Measuring the percentage of completed fields alone may encourage staff to enter placeholder values rather than obtain the required information.

Controls and Decision Consequences

The measure should support the quality objective instead of becoming a substitute for it.

Quick check

Name the concept that completes this statement.

________ concerns whether a recorded value represents the relevant reality correctly.

Quick check

Name the concept that completes this statement.

________ concerns whether required observations or fields are present.

Your turn

Use the chapter case: A dashboard with conflicting totals.

Create a data dictionary for five fields, including meaning, source, owner, identifiers, and update timing.

Prepare a brief explanation with a stated assumption and an outcome measure.

Recap

Evaluate quality against the intended decision. Combine clear definitions, validation, independent checks, visible limitations, and accountable correction processes.

Reading and assignments