AI Engineer
Design, build, evaluate, and operate production AI systems
An engineering path from measurable problem framing and data contracts through classical and deep learning, LLM applications, agents, evaluation, security, and production operations. The work centers on reviewable implementations, test evidence, and explicit operating limits.
Prerequisites
- No prior machine-learning study is required; complete AI Engineering Foundations before Classical Machine Learning
- Working Python knowledge: functions, classes, packages, virtual environments, testing, and debugging
- Command-line and Git proficiency, plus working knowledge of HTTP and JSON
- Working familiarity with pandas or another tabular-data stack, practical SQL, web APIs, and asynchronous Python; complete the course-provided bridge before the second lab for any missing area
- Foundational algebra, probability, statistics, and linear algebra; the track applies these foundations rather than replacing a mathematics course
AI Engineering Foundations
Start with plain-language AI concepts, then translate an ambiguous public-service problem into a measurable ML task, establish a baseline, and build a leakage-safe data workflow.
- Produce an Arabic-English vocabulary guide that distinguishes AI, machine learning, deep learning, and generative AI with one decision example and one limit for each
- Write a problem brief that names the decision, owner, prediction target and timing, available data, baseline, and measurable success criteria
- Validate the dataset against a documented contract and run a baseline whose features and split exclude target and future-data leakage
Start Here: AI Engineering for Public Services
Build the mental model and vocabulary you need before touching machine-learning code.
Problem Framing and Decision Design
Define the decision, label, users, and constraints before writing a line of model code.
Data Contracts and Quality Gates
Make data assumptions explicit with schemas, validation, profiling, and versioned quality rules.
Lab assessment
Complete the four-part artifact for one bounded prediction decision. Use a reviewable dataset, record every validation check, and keep the final test partition untouched.
Submit the bilingual vocabulary guide, problem brief, data contract and validation report, runnable baseline, and a short README with reproduction steps.
The brief defines the unit, prediction time, target, action, owner, baseline, and measurable release criteria without substituting a model score for the service outcome.
Schema, types, ranges, missingness, provenance, label timing, and failed checks are documented with reproducible evidence.
The baseline runs from a clean environment, fits only on training data, uses a decision-relevant metric, and reports deterministic results.
Feature availability and split logic are justified, at least two leakage paths are tested, and data or deployment limitations are explicit.
Arabic and English terms are accurate, paired consistently, and understandable to a reviewer without machine-learning experience.
Classical Machine Learning
Learn what classical ML means in everyday terms, then build strong tabular models with the right features, objectives, thresholds, and error analysis before reaching for deep learning.
- Produce a plain-language comparison that selects classification for the stated decision and rules out regression, clustering, and anomaly detection with evidence
- Implement a versioned scikit-learn pipeline that fits preprocessing and features inside leakage-safe training and validation folds
- Select an operating threshold and publish an error report covering calibration, confusion costs, important slices, and actionable failure patterns
Classical Machine Learning: The Big Picture
Learn what classical ML is, how it learns from structured data, and why it is often the right first choice.
Supervised Learning in Practice
Frame prediction problems, choose objectives, establish baselines, and reason about generalization.
Feature Engineering Without Leakage
Transform raw records into stable signals while keeping training and serving behavior identical.
Imbalanced Classification and Calibration
Choose metrics, thresholds, weights, and probability calibration for rare-event decisions.
Validation Strategy and Error Analysis
Build trustworthy experiments with time-aware splits, confidence intervals, slices, and actionable errors.
Lab assessment
Train and compare a defensible classifier against the established baseline. Freeze the test set until model, preprocessing, metric, and threshold choices are complete.
Submit the method comparison, source and environment lockfile, runnable pipeline, experiment table, threshold analysis, and error report.
The selected task, objective, baseline, and model family follow from the decision and data, and rejected alternatives have technically sound reasons.
Preprocessing, encoding, feature generation, fitting, and inference are one reproducible pipeline with fixed seeds and no fit-time access to validation or test data.
The report compares the baseline and candidate on appropriate metrics, justifies the threshold from error costs, and inspects calibration and meaningful slices.
The work tests leakage and imbalance risks, avoids unsupported causal claims, and identifies conditions under which the classifier must not be used.
The comparison and error findings are concise, traceable to results, and understandable to a non-technical decision owner.
Deep Learning
Understand tensors, automatic differentiation, training loops, regularization, embeddings, and the transformer architecture.
- Implement and test a PyTorch model whose tensor shapes, dtypes, outputs, loss, and target contract are explicit
- Run a reproducible training experiment that records configuration, seed, learning curves, validation metrics, and the selected checkpoint
- Diagnose underfitting, overfitting, or unstable gradients from evidence and justify whether the trained model improves on a simpler alternative
Neural Networks from Tensors to Training
Understand tensors, forward passes, losses, gradients, architecture choices, and practical debugging.
Training Loops and Regularization
Write a correct training loop, tune optimization, and control overfitting deliberately.
Embeddings and Transformer Fundamentals
Understand representation learning, attention, transformer blocks, and when to use pretrained models.
Lab assessment
Train one compact model on the supplied task, beginning with shape assertions and a tiny-batch overfit check. Compare it with the simplest credible baseline.
Submit model and training code, pinned environment, configuration, checkpoints or hashes, learning curves, metric table, and experiment note.
Automated checks cover input, intermediate and output shapes, dtypes, label encoding, loss compatibility, and inference mode.
The loop handles batches, optimizer steps, gradient reset, validation, checkpoint selection, and device placement correctly.
Configuration and seeds reproduce the reported curves and metrics, and the selected checkpoint is supported by held-out validation evidence.
The note identifies observed failure signals, applies proportionate regularization or debugging, and records data, compute, and misuse limits.
The conclusion states whether deep learning is warranted relative to the baseline, using quality and operational evidence rather than architecture popularity.
MLOps and Model Serving
Make training reproducible, package models behind typed interfaces, and operate them with monitoring, safe releases, and rollback plans.
- Package a versioned model and preprocessing pipeline behind a typed inference contract with validated requests and traceable responses
- Demonstrate repeatable build, startup, health, readiness, prediction, and overload tests from a clean environment
- Define service, data, and model monitors with owners, alert thresholds, release gates, and a tested rollback procedure
Reproducible Experiments and Artifacts
Make model development auditable by versioning data, code, configuration, metrics, and artifacts.
Model Serving and Inference APIs
Design typed inference APIs, choose serving modes, control concurrency, and release models safely.
Monitoring, Drift, and Safe Operations
Observe model services across health, data, quality, cost, and user outcomes.
Lab assessment
Serve the previous model as an immutable versioned artifact. Exercise normal, invalid, concurrent, dependency-failure, and rollback paths and retain the resulting evidence.
Submit the service repository, build manifest, API contract, automated test output, monitoring specification, release checklist, and rollback record.
A clean build produces a named model, preprocessing, dependency, configuration, and schema version whose provenance and hash are recorded.
The interface rejects invalid inputs, preserves training-time feature semantics, returns model metadata, and passes golden prediction tests.
Readiness, timeout, concurrency, payload, dependency-failure, and overload behavior are tested against explicit service objectives.
Metrics distinguish service health, input drift, output behavior, and delayed outcomes, with thresholds, owners, and response actions.
The release gate names approval and stop criteria, protects sensitive telemetry, and includes evidence that the previous version can be restored.
LLM Application Foundations
Work with language models as probabilistic APIs: manage context, request structured outputs, and design for latency, cost, and failure.
- Implement an inspectable LLM request and typed response boundary that validates structured output before application use
- Enforce explicit context, output-token, latency, cost, retry, and concurrency budgets and report their measured behavior
- Test malformed output, refusal, timeout, rate limit, provider error, and exhausted-budget paths with deterministic application fallbacks
How LLMs Behave at Runtime
Understand tokens, context windows, generation controls, uncertainty, latency, and cost.
Native LLM APIs and Structured Outputs
Build a provider call with explicit roles, schemas, tools, bounded retries, and normalized errors.
Prompt Design, Context, and Cost Control
Design layered prompts, select useful context, and make token budgets observable.
Lab assessment
Build one bounded native-SDK operation that turns validated input into a typed result. Capture sanitized request metadata and exercise each defined failure path.
Submit runnable service code, schemas, configuration, focused tests, sanitized traces, budget measurements, and a failure-behavior note.
Input and output schemas reject unknown or invalid values, and no generated structure reaches business logic before parsing and validation.
The complete model request is explicit and testable, including messages, tools, tool choice, response format, model configuration, and structured-output settings.
Measured tests demonstrate hard bounds for tokens, retries, deadlines, concurrency, and estimated cost under normal and adverse inputs.
Provider and validation failures map to safe application outcomes without unbounded retries, leaked internals, or silent partial results.
The note states assumptions, unsupported tasks, configuration ownership, privacy limits, and the deterministic fallback in reviewable language.
Retrieval-Augmented Generation
Build a grounded knowledge system from ingestion to retrieval, reranking, citations, and evaluation.
- Build a versioned ingestion path that preserves source identity, effective date, language, access scope, content hash, and deletion behavior
- Implement access-filtered retrieval, reranking, bounded context selection, abstention, and citation validation in the question-answering service
- Evaluate retrieval relevance, answerability, groundedness, citation correctness, and conflicting or unanswerable cases on a labeled set
RAG System Architecture
Design a grounded retrieval-augmented generation system from source documents to cited answers.
Retrieval, Hybrid Search, and Reranking
Improve RAG evidence with dense, sparse, hybrid, filtered, and reranked retrieval.
RAG Evaluation and Citations
Evaluate evidence, groundedness, answerability, and citation quality with a repeatable test set.
Lab assessment
Use an approved course corpus with stable source identifiers. Include answerable, unanswerable, stale, conflicting, permission-restricted, and prompt-injection cases.
Submit ingestion and service code, corpus manifest, retrieval configuration, labeled evaluation set, result report, sample cited answers, and threat notes.
Every indexed chunk is traceable to a versioned source and access scope, with repeatable updates, deletion, and duplicate handling.
Permission filters precede retrieval, context remains within budget, answers follow supplied evidence, and unsupported questions trigger abstention.
The labeled set covers stated risk cases and reports retrieval, answer, abstention, groundedness, and citation measures with inspected failures.
Displayed citations resolve to the exact supporting passage and effective source, while invalid or invented identifiers are rejected.
The service resists cross-scope retrieval and document instructions, protects sensitive logs, and states corpus, freshness, and answerability limits.
Agents and Tool Use
Turn model output into controlled actions with typed tools, explicit state machines, budgets, timeouts, and approval boundaries.
- Define an explicit state machine that separates deterministic workflow steps from model decisions and terminal states
- Implement typed tools with server-side identity, authorization, idempotency, validation, minimal permissions, and mocked side effects
- Demonstrate traces for success, refusal, invalid arguments, tool failure, repeated calls, exhausted budgets, timeout, and required human approval
Tool Calling and Action Contracts
Expose typed, least-privilege tools and validate every model-proposed action.
Agent Orchestration and State
Build bounded workflows with explicit state, budgets, timeouts, retries, and human handoffs.
Agent Safety and Reliability
Defend tool-using workflows against injection, runaway execution, unsafe side effects, and ambiguous failures.
Lab assessment
Implement one bounded agent in a sandbox with no irreversible external side effects. Run the prescribed scenario suite and preserve redacted traces.
Submit the state diagram, tool schemas and authorization rules, runnable agent, scenario tests, redacted traces, and approval and recovery policy.
States, transitions, stop conditions, retries, deadlines, budgets, and human gates are explicit, finite, and consistent with the implemented flow.
Schemas are precise, arguments are validated, authorization is enforced server-side, repeated requests are safe, and outputs are treated as untrusted.
Automated scenarios prove expected state transitions and final outcomes, and each trace links model choice, tool request, approval, result, and error.
Irreversible or high-impact actions are excluded or require informed approval, and cancellation, timeout, and partial failure leave a known recoverable state.
Trace fields support review without exposing secrets or unnecessary personal data, and retention and ownership are documented.
Production AI and Capstone
Evaluate AI systems like software, observe them in production, protect data and users, and assemble the complete capstone architecture.
- Produce a bilingual service design that traces user journeys to data, model, tool, human-review, fallback, and tenant-isolation boundaries
- Define offline and online quality gates, traces, service and model metrics, feedback handling, alert ownership, and recovery objectives
- Demonstrate a delivery plan with privacy and security controls, staged release, incident exercises, rollback evidence, and explicit no-launch criteria
AI Evaluation and Observability
Combine offline evaluations, traces, metrics, feedback, and release gates for AI systems.
Security, Privacy, and AI Governance
Protect tenants, users, secrets, and sensitive data across the AI lifecycle.
Design and Ship the AI Capstone
Assemble the complete architecture, contracts, evaluation plan, and delivery sequence for a production AI service.
Lab assessment
Synthesize the preceding work into one implementation-ready design for a bounded bilingual service. Resolve contradictions between quality, security, privacy, cost, and operations before recommending release.
Submit the architecture and data-flow diagrams, bilingual service contract, evaluation and observability plan, threat and privacy review, runbooks, staged delivery plan, and decision record.
The design specifies identities, trust boundaries, data flows, model and tool contracts, human decisions, dependencies, failure paths, and ownership without hidden components.
Quality gates and observability connect representative offline cases to live service, model, safety, cost, and outcome measures with owners and thresholds.
Controls cover least privilege, tenant isolation, secret handling, data minimization, retention, audit, abuse, approval, and incident responsibility.
The staged plan has measurable entry, stop, rollback, and no-launch criteria, and incident exercises demonstrate recoverable service states.
Arabic and English journeys preserve equivalent decisions, evidence, errors, fallback, and escalation, with RTL and accessibility requirements ready for verification.