What is Knowledge Operations?
Overview · 3 min read
Knowledge Operations is the discipline of treating organisational knowledge as a maintained product rather than an accumulating archive. It defines how knowledge is captured, structured, versioned and served to both people and AI systems, and it owns the quality of what those systems retrieve. Where knowledge quality is unmanaged, every downstream decision inherits the defect.
Retrieval is not record-keeping
A system of record and a retrieval layer are built for different jobs. Records exist to preserve the state of a transaction or a document; retrieval exists to answer a question with the smallest sufficient, most current excerpt. Confusing the two is the most common architectural mistake in knowledge work. A folder full of signed documents is a record system. It will not reliably answer “what is the current inspection interval for this asset type?” unless someone has also built the metadata, chunking and precedence rules that let a system find and rank the answer.
Good Knowledge Operations therefore begins with a controlled vocabulary and a metadata schema, not with a search engine. Terms need owners, definitions need to be agreed, and documents need stable identifiers that survive editing. Without this layer, retrieval returns plausible text that cannot be verified against an authoritative source. The test of the layer is precedence: when two documents disagree, the system must know which one governs, and why.
The operating loop
Knowledge Operations runs as a loop: capture from the point of work, structure against the schema, publish with a named owner and an effective date, then measure whether the content is being found and used. Measurement is what separates a maintained corpus from a decaying one. Usage tells you which documents matter; correction rates tell you which are wrong; unanswered questions tell you what is missing.
- Capture — pull knowledge in at the moment it is created, not in a periodic clean-up.
- Structure — apply the controlled vocabulary, metadata and versioning rules consistently.
- Serve — expose content to people and systems through the same governed retrieval layer.
- Govern — assign owners, review dates and a retirement path for every item.
Ownership is the scarce resource
Most knowledge programmes fail for organisational reasons rather than technical ones. No one is accountable for whether a document is still true, so the corpus drifts while the tooling is perfected. Fixing this is unglamorous: name an owner for each knowledge domain, give them a review cadence and the authority to retire content, and report on coverage and usage rather than on how many documents were ingested. Volume is a vanity metric; currency and findability are the ones that matter.
Rule of thumb: if a question cannot be answered consistently by two people and an assistant from the same source, that source is not yet operationally ready — it is still just stored.
Questions to ask
- Which knowledge is authoritative, and who signs off when it changes?
- How is content retired or superseded when it becomes wrong?
- Can a retrieval system cite the exact version and source it used?
- Who owns the metadata that makes content findable in the first place?
Data Architecture for AI
Architecture · 2 min read
An AI-ready data architecture separates the storage of truth from the storage of speed, and makes both addressable through shared identity and metadata. The aim is not one monolithic platform but a set of layers whose boundaries are explicit and testable. These decisions constrain every model, agent and report built on top, so they are worth making deliberately rather than by accretion.
Separate the record layer from the retrieval layer
Systems of record — transactional databases, historians, document repositories — are optimised for correctness and durability. Analytical and retrieval workloads need different access patterns: wide scans, embeddings, low-latency lookups, joins across domains. Attempting to serve both from one store usually degrades both. A common pattern is a lakehouse or layered warehouse in which raw data is landed immutably, transformed into curated domain datasets, and then exposed through semantic views that define agreed business meaning.
Streaming deserves its own lane. Operational decisions — anomaly alerts, routing, pricing — need data within seconds, while reporting tolerates hours. Mixing these cadences in one pipeline obscures which latency the consumer actually depends on. Architects should name, per dataset, whether it is batch, micro-batch or streaming, and why. Freshness, like resolution elsewhere, is a requirement to be chosen rather than a default to be inherited.
Metadata is the load-bearing layer
A catalogue turns stored data into discoverable data. For AI it must carry more than table names: it should record ownership, sensitivity, refresh cadence, quality status and lineage. Models and agents then inherit the same definitions humans use, which is what prevents a dashboard and an assistant from disagreeing about the same metric. Feature reuse belongs here too — a shared feature definition avoids every team recomputing a subtly different version of the same signal.
- Ingestion — land raw data unchanged, with source, timestamp and batch identity preserved.
- Curation — transform into domain datasets with agreed schema and quality thresholds.
- Semantics — publish metrics, entities and vocabularies as governed views.
- Service — expose the same definitions to reporting, models and retrieval alike.
Fit the pattern to the organisation
Patterns such as lakehouse and data mesh are not ideologies; they are trade-offs. A lakehouse centralises storage and compute beneath a single governance layer, which suits organisations that need consistency and can sustain a strong platform team. A mesh federates ownership to domains, which suits large, diverse organisations willing to invest in shared standards and self-service tooling. The wrong choice is usually made by copying a peer rather than by asking where meaning and quality can realistically be enforced, and by whom.
Rule of thumb: if two consumers of the same metric would get different numbers, the problem is architecture, not reporting.
Questions to ask
- Which datasets are systems of record, and which are derived?
- Where are business definitions enforced, and who approves a change?
- What latency does each operational consumer actually require?
- Can a model input be traced back to a source record and a transformation?
SOP Digitisation & Knowledge Capture
Guide · 2 min read
Digitising a standard operating procedure is not scanning it. Digitisation means turning a procedure into structured, versioned and addressable content that a person can follow and a system can retrieve. It is often the first Knowledge Operations project because the material already exists and its value is legible to the business.
From document to structured procedure
A digitised procedure has a stable identifier, a version history, an owner, an effective date and a defined scope. Its steps are discrete and ordered, each with the tools, inputs and acceptance criteria attached. Where a step depends on a value — a temperature, a torque setting, a tolerance — that value should reference the governing parameter rather than repeat a number that will drift out of date. This structure is what allows a query such as “show me every procedure that touches this asset class” to work.
Metadata is what makes procedures findable and safe. Tagging by asset type, site, discipline, regulatory scope and revision status turns a flat library into something an assistant can navigate and cite. The same tags support review: procedures past their review date, or tied to a decommissioned asset, surface automatically instead of quietly persisting.
Capture the knowledge that is not written down
Much operational knowledge lives in the heads of experienced staff and in informal notes. Structured capture — short interviews against the schema, task observation, or short recorded walkthroughs transcribed into the same template — converts that tacit knowledge into content that survives turnover. Capture should be scheduled around risk and retirement, not left until someone leaves. A short interview held deliberately is worth more than a reconstruction attempted in the final week of a notice period.
- Stable identity — every procedure has a permanent identifier independent of its file name.
- Parameterised values — reference governing settings rather than hard-coding numbers.
- Discrete steps — ordered, verifiable steps with inputs and acceptance criteria.
- Review metadata — owner, scope, revision status and a scheduled review date.
Choose the first tranche deliberately
Digitisation is worth sequencing rather than attempting in bulk. The strongest first tranche combines high operational risk with stable content: procedures that are frequently consulted, expensive to get wrong, and unlikely to change next month. That combination produces early value and builds the schema against real material. Attempting every procedure at once spreads ownership thin and produces a large library whose currency no one can realistically maintain.
Rule of thumb: if you cannot route a procedure to a specific owner and a specific asset class, it is a document, not a digitised procedure.
Questions to ask
- Which procedures carry the highest operational or compliance risk if they are wrong?
- Who is accountable for each procedure, and when was it last reviewed?
- Are values in the document linked to their governing parameters, or copied by hand?
- What knowledge would be lost if a single experienced person left this week?
Time-Series Data for Industrial AI
Technical · 3 min read
Industrial AI depends on time-series data whose meaning is carried by context as much as by the reading. A sensor value without its unit, sampling rate, asset, calibration state and surrounding events is ambiguous, and ambiguous inputs make models that cannot be trusted on the plant floor. Treating time-series as a governed asset — with defined resolution, retention and semantics — is what makes the difference.
Sampling, resolution and aggregation
Not every signal needs the same cadence. Vibration monitoring may require high-frequency capture, while a tank level might be polled once a minute or less. Storing everything at the finest resolution is expensive and rarely justified; the design question is what resolution each downstream use actually needs. Aggregation choices matter too: averages hide spikes that matter to an anomaly model, so the aggregation window and statistic should be chosen against the use, and both should be recorded alongside the data.
Alignment across signals is a persistent source of error. Clocks drift, tags are polled on different cycles, and events arrive with their own timestamps. A defined time base, consistent time-zone handling and an explicit policy for interpolating or holding gaps keep a multi-signal model from training on misaligned inputs.
Context, quality and missing data
Every stream needs an asset hierarchy and a unit registry so that a reading can be tied to a physical thing and a physical quantity. Quality flags — sensor fault, out-of-range, imputed — belong with the data, not buried in a separate spreadsheet. Gaps are normal and should be handled by a stated policy: discard, interpolate with a marked method, or hold the last value, each with different consequences for a model.
- Resolution per use — store at the finest rate the use case requires, and no finer.
- Asset and unit context — tie every tag to a registered asset and a defined unit.
- Quality flags — carry fault and imputation status with each value.
- Event annotation — record maintenance, changeovers and faults as timestamped events.
Storage and access patterns
Storage and access patterns follow from the semantics. A time-series store optimised for append and range scans serves dashboards, models and alerts differently from a warehouse optimised for joins. Real-time alerting needs the most recent window at low latency; training needs long history with consistent quality. Some organisations keep a hot copy near the edge for immediate decisions and a governed copy in the central platform for analysis and audit. Whichever arrangement is chosen, both copies should share the same asset and unit definitions so they cannot quietly diverge.
Rule of thumb: if an engineer cannot tell what a sensor reading means, how often it was sampled and what is missing, a model cannot be trusted to act on it either.
Questions to ask
- What resolution does each use case genuinely require, and why?
- Is every tag mapped to a registered asset and an agreed unit?
- How are gaps and faults represented, and is that policy documented?
- Are operational events captured alongside the signals they explain?
Data Governance for AI Pipelines
Framework · 3 min read
Governance for AI pipelines is the set of controls that keep data fit for automated use: lineage, quality gates, access rules and retention. Framed correctly, it is what lets teams move quickly without silently degrading the models that depend on shared data. It is an engineering discipline with policy attached, not a compliance formality bolted on at the end.
Lineage and quality gates
Lineage answers where a value came from and how it was transformed. For AI it must be continuous from source record to model input, because that chain is what allows a surprising output to be explained or a defect to be traced. Lineage is only useful when it is enforced by the pipeline rather than documented by hand; a diagram that drifts from reality is worse than none at all.
Quality gates turn governance into something operational. Contracts between producing and consuming teams define expected schema, freshness, completeness and range. When a gate fails, the pipeline should quarantine or halt rather than pass bad data downstream, and the failure should be visible to the owner of the producing dataset. Gates set thresholds for the uses that exist, and those thresholds are reviewed as uses change.
Access, retention and defensible deletion
Access controls should be defined at the dataset and field level, with sensitivity classification driving who can read, transform and export. For AI this includes the training and evaluation sets, which are often forgotten precisely because they are copies. Retention rules state how long each class of data is kept and, just as importantly, how it is removed when the retention period ends or a deletion request arrives. A pipeline that cannot delete a record from its derived datasets and embeddings does not have governance — it has hope.
- Continuous lineage — trace every model input back to a source and a transformation.
- Data contracts — agreed schema, freshness and quality expectations per dataset.
- Quality gates — quarantine or halt on failure, and notify the producing owner.
- Retention and deletion — defined periods that propagate into derived data and indexes.
Governance that speeds work up
Good governance is measured by what it enables, not by how many policies it publishes. When definitions, quality expectations and access rules are clear, teams integrate datasets without repeated round-trips and analysts trust shared metrics. When they are not, every consumer rebuilds a local copy and the same reconciliation work is paid for again and again. The practical test is whether a new pipeline can be stood up against a governed dataset quickly, with lineage and controls inherited rather than re-argued from scratch.
Rule of thumb: if you cannot trace an AI output to its source and delete that source everywhere it was copied, you do not yet govern the pipeline.
Questions to ask
- Is lineage enforced by the pipeline or maintained by hand?
- Who owns each dataset, and what contract does it publish?
- What happens automatically when a quality gate fails?
- Can a deletion request reach every derived copy and index?
← Back to Knowledge Bank