AI Data Scientist
Turn reliable data into measurable AI and analytical decisions
An applied Python data-science path from analytical framing and quality-controlled data through statistics, feature engineering, scikit-learn modeling, evaluation, responsible analysis, reproducible delivery, and a stakeholder decision brief.
Prerequisites
- Working Python 3.12+ knowledge, including pandas, NumPy, scikit-learn, plotting, functions, environments, and debugging
- Practical SQL for filtering, joins, grouping, aggregation, and checking tabular data
- Foundational descriptive and inferential statistics, probability, sampling, uncertainty, and experiment concepts
- Ability to work in notebooks and the command line, manage environments, and use Git for versioned analytical work
- Ability to explain analytical findings, uncertainty, and limitations to a non-technical stakeholder
Analytical Problem Framing and EDA
Frame an analytical question, understand a dataset, and tell an evidence-based story before modeling.
- Write an analysis brief that defines the decision, population, unit of analysis, outcome, timeframe, comparison, stakeholders, and pre-model success criteria
- Build a reproducible pandas profile from a practical SQL extract covering schema, distributions, missingness, duplicates, outliers, time coverage, and important segments
- Publish an initial insight narrative whose tables and visualizations support bounded findings, alternative explanations, limitations, and the next analysis question
Problem Framing and Exploratory Data Analysis
Translate a service question into a population, outcome, timeframe, and dataset before exploring patterns.
Data Profiling and Quality
Inspect types, completeness, ranges, duplicates, distributions, and data-quality assumptions systematically.
Visualization and Analytical Storytelling
Choose visuals that answer a decision question and communicate patterns, uncertainty, and limits without distortion.
Lab assessment
Use Python 3.12+, pandas, a versioned notebook or script, and the supplied database or extract. Complete the profile before choosing a model or presenting a recommendation.
Submit the analysis brief, SQL query, environment lock, reproducible notebook or script, generated profile, figure files, insight narrative, and data-question log.
The brief defines a decision-relevant question, population, unit, outcome, period, comparison, exclusions, owner, and measurable completion criteria.
SQL grain, joins, filters, dates, and row counts are verified, and the pandas profile computes schema and quality statistics correctly and reproducibly.
Tables and visualizations inspect distributions, relationships, time, missingness, outliers, and important segments with appropriate scales and denominators.
The narrative separates observation from explanation, avoids causal claims, documents selection and measurement limits, and names unanswered questions.
The decision summary uses accurate labels, readable figures, sourced values, and plain Arabic and English terminology where stakeholder-facing text is required.
Data Quality and Preparation
Prepare trustworthy analysis data through explicit contracts, cleaning rules, and reproducible transformations.
- Define and test a data contract for grain, keys, fields, types, ranges, units, nullability, allowed values, freshness, lineage, and target timing
- Implement deterministic pandas transformations for missing values, duplicates, outliers, normalization, joins, and derived fields without overwriting raw data
- Reconcile source and output counts and publish a quality report with failed checks, exclusions, before-and-after measures, lineage, and unresolved fitness risks
Data Contracts and Lineage
Define what each field means, where it came from, when it is available, and how transformations affect it.
Cleaning and Missing Data
Choose transparent rules for invalid, missing, duplicated, and outlying values without silently changing meaning.
Join and Transform Data Safely
Combine sources with explicit keys, time boundaries, row-count checks, and reproducible transformation steps.
Lab assessment
Create the analysis-ready dataset as a derived, versioned output. Preserve raw inputs, encode every cleaning choice, and fail the pipeline on critical contract violations.
Submit the contract, transformation code, automated data tests, source and output manifests, analysis-ready dataset or reproducible build reference, lineage diagram, and quality report.
The contract captures grain, keys, schema, validity, timing, provenance, ownership, and critical versus warning thresholds for the analytical purpose.
Cleaning and joins preserve intended grain, avoid silent row loss or multiplication, use defensible missing and outlier rules, and produce deterministic output.
Automated tests and reconciliations quantify validity, completeness, uniqueness, consistency, freshness, and before-and-after effects, including failures.
A clean run from versioned inputs and pinned environment recreates the output and links every derived field and exclusion to code and source.
The report states which analyses the data supports, which it does not, unresolved bias or quality risks, and clear use instructions for the next analyst.
Statistics and Experiment Design
Use statistical reasoning to separate signal from noise and design fair comparisons.
- Estimate a decision-relevant quantity with a justified sample, point estimate, uncertainty interval, distribution checks, and practical interpretation
- Run and verify an appropriate statistical comparison with explicit null and alternative hypotheses, assumptions, effect size, and multiple-testing treatment where needed
- Design an experiment or justified quasi-experiment with assignment, unit, outcome, guardrails, sample-size rationale, analysis plan, and validity threats
Sampling and Distributions
Understand populations, samples, variation, distribution shape, and why one observed average is not the whole story.
Hypotheses, Confidence, and Uncertainty
Use confidence intervals and hypothesis tests appropriately, including their assumptions and limits.
Experiment Design and Causality
Design comparisons that reduce bias and distinguish correlation from a credible causal claim.
Lab assessment
Answer one decision question with the supplied analysis-ready data, then design the next credible comparison. Pre-specify the method before inspecting the final outcome split.
Submit the statistical note, executable analysis, assumption diagnostics, effect and interval results, experiment protocol, sample-size calculation, pre-analysis plan, and limitation register.
Population, estimand, sampling process, exclusions, weights if used, missing data, and representativeness limits match the decision question.
The chosen estimate and test fit the data and design, assumptions are checked, calculations are reproducible, and multiplicity is handled when material.
Results include effect size and uncertainty, distinguish statistical from practical importance, and connect estimates to an explicit decision threshold.
Assignment, units, comparison, outcome timing, guardrails, sample size, interference, attrition, analysis, and internal and external validity threats are addressed.
The note states only conclusions supported by the design, clearly labels exploratory work, and communicates assumptions and limitations without hiding null results.
Feature Engineering and Validation
Create useful model inputs and evaluation splits without leaking future or protected information.
- Create a feature catalog with definition, source, owner, type, transformation, availability time, refresh, missingness, allowed use, and expected relationship for each candidate
- Implement a scikit-learn preprocessing pipeline that learns encoding, scaling, imputation, and feature selection from training folds only
- Select and test time, group, stratified, or nested validation splits that mirror deployment and block target, future, duplicate, and entity leakage
Define Features and Availability
Document a feature’s meaning, source, availability time, missing-value rule, and risk before using it.
Encoding, Scaling, and Pipelines
Apply preprocessing through fitted pipelines so training, validation, and serving use the same transformations.
Validation Splits and Leakage
Choose time, group, or stratified splits that reflect deployment and prevent answer information entering a model.
Lab assessment
Design features and validation for the stated prediction time and deployment population. Treat unavailable, prohibited, or ambiguously timed fields as excluded until proven safe.
Submit the feature catalog, availability diagram, fitted-pipeline source, split implementation, automated leakage tests, fold summary, and validation rationale.
Every candidate is traceable, computable at prediction time, semantically clear, allowed for the purpose, and supported by a stated hypothesis rather than convenience alone.
Imputation, encoding, scaling, transformation, and selection fit only on training data and preserve stable column semantics at inference.
Split boundaries reflect time, entities, prevalence, tuning, and final evaluation needs, with no overlap that invalidates the performance estimate.
Automated and manual checks cover target proxies, future values, post-outcome operations, duplicate entities, preprocessing fit, and protected or prohibited information.
The plan identifies freshness, missing-at-inference, distribution-shift, subgroup, maintenance, and feature-removal risks and assigns follow-up checks.
Machine Learning Modeling
Build baselines, select appropriate model families, and train models with purposeful hypotheses.
- Implement decision, heuristic, and statistical baselines that establish the minimum useful performance for the validated task
- Run versioned scikit-learn experiments comparing justified regression or classification families under the fixed pipeline and validation plan
- Select a candidate from held-out evidence that balances task metrics, stability, calibration, interpretability, latency, maintenance, and error cost
Baselines and Model Selection
Compare simple baselines with candidate models using task constraints, interpretability, latency, and error costs.
Regression and Classification in Practice
Choose regression or classification from the decision and target, then match outputs to the right loss and metric.
Hyperparameters and Training Discipline
Tune learning settings through documented experiments rather than repeated test-set trial and error.
Lab assessment
Freeze the data contract, feature pipeline, splits, and primary metric before comparison. Tune only inside training data and reserve final test evidence for the selected candidate.
Submit experiment code and configuration, pinned environment, baseline results, run ledger, comparison table, selected serialized pipeline and hash, test report, and model-selection note.
Baselines reflect the current decision and simple credible alternatives, use the same held-out protocol, and make the required incremental value explicit.
Model families and search spaces follow hypotheses, preprocessing and folds are fixed, tuning is nested correctly, and seeds and configurations are recorded.
Results include primary, guardrail, calibration, variability, subgroup, latency, and resource measures, with uncertainty and no selective run omission.
The selected model is the simplest candidate meeting stated criteria; the note identifies failure modes, unsupported populations, and reasons not to deploy.
A clean command recreates the selected pipeline and reported test result from versioned inputs, with artifact identity and provenance recorded.
Evaluation, Fairness, and Communication
Evaluate model behavior, inspect errors across groups, and communicate uncertainty to decision-makers.
- Evaluate the frozen candidate on untouched data using decision-linked metrics, confidence intervals, calibration, threshold trade-offs, and comparison with baselines
- Perform error and slice analysis across pre-specified operational and affected groups with sample sizes, uncertainty, error costs, and investigated causes
- Issue a stakeholder recommendation that states supported use, threshold, human review, limits, monitoring needs, and deploy, revise, or stop decision
Metrics and Decision Thresholds
Select metrics and thresholds that reflect the real cost of missed, incorrect, and unnecessary actions.
Error Analysis and Fairness
Inspect failure patterns and group-level outcomes before concluding that an aggregate score is acceptable.
Communicate Uncertainty and Limits
Explain confidence, assumptions, limitations, and appropriate actions without overstating a model’s certainty.
Lab assessment
Evaluate the frozen model once on the untouched test set. Pre-register primary metrics and slices, report unfavorable findings, and avoid fairness claims unsupported by sample size or context.
Submit the evaluation protocol and code, locked result tables, calibration and threshold plots, error taxonomy, slice report, example review, stakeholder narrative, and recommendation record.
Untouched data, pre-specified metrics and slices, correct denominators, uncertainty, baseline comparison, and reproducible code support an unbiased final estimate.
Metrics match the decision and imbalance, calibration is assessed, and the selected threshold follows explicit error costs, capacity, and guardrails.
The report quantifies errors by meaningful slices with sample size and uncertainty, reviews representative cases, investigates mechanisms, and avoids unsupported conclusions.
Supported and prohibited uses, affected groups, residual risks, human review, data limits, monitoring, and revise or stop triggers are explicit.
The narrative translates metrics into service consequences, shows uncertainty and unfavorable evidence, and uses clear bilingual labels where the report is stakeholder-facing.
Deployment, Monitoring, and Reproducibility
Prepare analysis and models for repeatable delivery, controlled use, and monitored change.
- Package versioned data references, code, environment, configuration, model, metrics, and documentation behind one repeatable build and scoring command
- Design a controlled batch or service delivery path with schema validation, access limits, idempotency, failure handling, release approval, and rollback
- Define monitors for data quality and drift, service health, prediction behavior, delayed outcomes, fairness slices, and retraining decisions with owners and thresholds
Reproducible Data Workflows
Record code, data version, configuration, assumptions, environment, and outputs so an analysis can be repeated.
Batch Delivery, Serving, and Monitoring
Choose a delivery pattern and define owners, latency, failures, outputs, and monitoring before operational use.
Drift and Retraining Decisions
Detect changes in data or performance and decide whether to investigate, recalibrate, retrain, roll back, or stop.
Lab assessment
Create a delivery design for the evaluated model without implying production approval. Demonstrate reproducibility and monitoring on a controlled replay or synthetic feed.
Submit the delivery manifest, build and scoring commands, validation and replay evidence, architecture and access design, metric dictionary, dashboards or specifications, alert runbook, rollback and retraining decision record.
Versioned inputs, code, environment, configuration, schema, model, and metrics recreate the evaluated artifact and predictions with recorded identity and provenance.
The batch or service design validates schema, preserves feature semantics, controls access and duplication, handles partial failure, and supports traceable release and rollback.
Measures cover input quality and drift, prediction and calibration behavior, service health, delayed outcomes, slices, feedback, cost, and data freshness with valid windows.
Thresholds have owners and runbooks, distinguish noise from action, define pause and rollback, and require evidence and approval before retraining or redeployment.
Logs and monitoring minimize sensitive data, retention and access are defined, unsupported uses are stated, and the plan avoids claiming readiness beyond tested evidence.
Data Science Capstone
Deliver an evidence-backed analytical or modeling recommendation with reproducible work and clear limitations.
- Execute a versioned Python 3.12+ analysis from approved source data through validation, statistics or modeling, final evaluation, and generated decision evidence
- Write a decision brief that answers the original question with quantified evidence, uncertainty, alternatives, affected-group findings, limitations, and a recommendation
- Hand over data contracts, lineage, code, environment, outputs, model or report, use constraints, monitoring needs, owners, and next decision so another analyst can reproduce the work
Write the Data Science Capstone Brief
Define the decision, data boundary, methods, evaluation, risks, and stakeholder outcome for one focused capstone.
Run and Review the Capstone Analysis
Execute the analysis reproducibly, review code and results, and test normal and failure cases before presenting conclusions.
Hand Over the Data Science Work
Package the data assumptions, code, results, model or report, owners, limitations, and next decisions for another team.
Lab assessment
Complete one analytical or modeling decision end to end. Reuse the course methods and artifacts, resolve inconsistencies, and make a no-model or stop recommendation when the evidence requires it.
Submit the repository, pinned environment, data and model manifests, automated checks, reproducible run command, result package, decision brief, limitation and risk register, and signed handover checklist.
The question, population, data contract, methods, splits or comparisons, metrics, and conclusion remain aligned, with no leakage or unsupported change in scope.
Python, pandas, SQL, statistical, and scikit-learn work used in the project is tested, technically correct, appropriately simple, and reproducible from a clean environment.
The recommendation follows sourced results, compares a baseline and alternatives, quantifies uncertainty and practical effect, and exposes unfavorable evidence.
Data, method, subgroup, deployment, privacy, misuse, and monitoring limits are explicit, with supported and prohibited use and revise or stop triggers.
An independent reviewer can reproduce the stated outputs and understand ownership, versions, assumptions, use constraints, and next action from the handover alone.