Evaluating whether your proposed ML use case is technically feasible given your current data -- and whether ML is the right approach at all. We start by defining the problem formulation precisely: is this a classification task (binary or multi-class), a regression problem, an anomaly detection use case, or an NLP or computer vision problem? The formulation determines the data requirements, the evaluation metric, and the expected accuracy range. A binary classification problem requires a different label distribution, baseline model, and success metric than a multi-class or sequence labelling problem -- getting this wrong at the start leads to a model that is evaluated against the wrong target. Baseline model test: before recommending a full build, we train a simple baseline using scikit-learn (logistic regression, decision tree, or a gradient boosting model via XGBoost or LightGBM depending on the data type) on a sample of your actual data. A baseline that fails to beat a naive majority-class predictor on your sample disproves the feasibility assumption in hours rather than weeks. Cross-validation strategy for the baseline depends on the data structure: stratified k-fold for classification with class imbalance, time-series split for sequential data where future leakage would inflate baseline accuracy, and group k-fold where data groups (customer IDs, session IDs) must not appear in both train and validation splits. Evaluation metric selection: for imbalanced classification, accuracy is misleading -- we evaluate precision, recall, F1, and AUC-ROC to characterise model behaviour across the operating range rather than at a single threshold. Hyperparameter tuning for baseline models uses Optuna with a study budget of 50-100 trials -- enough to establish whether the problem is learnable without spending days on exhaustive search. Most assessments find either that a rule-based or threshold-based system handles 80% of the use case at a fraction of the cost, or that the data volume and labelling quality are too low to support reliable predictions. Both findings are more valuable than a confident recommendation that turns out to be wrong.

A structured review of every data source relevant to the use case: volume (classification models typically need 1,000+ labelled examples per class before generalising reliably; deep learning requires an order of magnitude more), quality (completeness rate per field, consistency across source systems, outlier prevalence), class distribution (a 95/5 positive/negative split requires different handling -- SMOTE oversampling, class-weight adjustment, or threshold calibration -- than a balanced dataset), and temporal coverage (is the historical data long enough to capture seasonality? Does the training period reflect current conditions or a regime that no longer applies?). Data quality profiling is performed with pandas-profiling or Great Expectations: missing value rates per column, cardinality of categorical features, distribution skew, and cross-source consistency checks flag problems that would silently degrade model quality if not addressed before training begins. Feature availability at inference time is the most commonly overlooked readiness dimension: data that exists in your historical database but is computed or aggregated after the prediction event cannot be used as an input feature without leaking future information into the training set. We map each candidate feature to its availability timestamp relative to the prediction event. Concept drift risk assessment evaluates whether the statistical properties of the training data are likely to remain stable in production: population shift (the input distribution changes), label shift (the relationship between inputs and outputs changes), or covariate shift (feature distributions change while the conditional label distribution stays stable) each require different monitoring and retraining strategies. PSI (Population Stability Index) and KS test (Kolmogorov-Smirnov) are the standard statistical tests for detecting distribution drift in production -- we design the monitoring approach in the audit phase so it is built into the system from day one rather than retrofitted after a performance degradation is noticed. SHAP (SHapley Additive exPlanations) analysis on the baseline model reveals which features drive predictions, confirming that the model is learning from signal rather than from spurious correlations in historical data that will not generalise. The output is a data readiness scorecard by use case -- a clear statement of what you have, what you need, and the gap between them.

Production architecture for the full ML system lifecycle -- from data ingestion to prediction delivery. MLflow or DVC for experiment tracking and model versioning: every training run reproducible, every model version auditable, rollback possible without re-running an experiment. Feature store design using Feast or Tecton where multiple models share computed features -- centralising feature engineering prevents the same transformation being reimplemented three different ways in three different pipelines. Model serving infrastructure: FastAPI or BentoML for online inference with latency requirements under 100ms; Celery or Ray for batch inference at scale. Online vs. batch inference decision: online for user-facing predictions where latency matters, batch for operational scoring (churn risk, credit assessment) where predictions can be pre-computed. Model monitoring using Evidently AI or WhyLabs for data drift and prediction drift detection -- because a model trained on last year's data degrades silently without monitoring. Retraining trigger design: scheduled retraining vs. drift-triggered retraining vs. manual review gate for high-stakes predictions.

Independent evaluation of ML platforms, data infrastructure tools, and cloud AI services against your specific use case, team capability, and budget -- with no vendor relationships and no referral incentives. Cloud ML platform comparison: AWS SageMaker (managed training jobs, SageMaker Pipelines for orchestration, built-in algorithms for common use cases, tight integration with S3/Glue); GCP Vertex AI (AutoML for teams without modelling expertise, Vertex Pipelines built on Kubeflow, BigQuery ML for SQL-native model training); Azure ML (Designer for low-code workflows, native MLflow integration, strong enterprise compliance for regulated industries). Open-source evaluation: Ray for distributed training and serving, Kubeflow for Kubernetes-native pipelines, Databricks for unified analytics and ML in a single lakehouse platform. Evaluation criteria include total cost of ownership at production volume, data residency requirements, vendor lock-in risk (is the model artefact portable?), team familiarity cost, and SLA coverage for production inference. The output is a scored comparison with a clear recommendation and documented reasoning.

Assessment of your in-house team's ML capability against the specific requirements of your proposed project -- mapped to the five distinct skill areas that most organisations conflate as a single "ML skill." Data engineering (ETL pipeline construction, feature transformation, data quality tooling): the most common gap and the one that blocks models from reaching production. Model training and experimentation (framework proficiency in scikit-learn, PyTorch, or XGBoost, experiment design, hyperparameter tuning): typically present in teams that have done any ML work. MLOps and deployment (model packaging, CI/CD for ML pipelines, serving infrastructure, monitoring): the gap that causes 85% of ML models to never leave the notebook environment. Production software engineering (API integration, system reliability, observability): often missing from data science teams. ML evaluation and measurement (offline metric design, A/B test design, business metric mapping): needed to know whether a model is actually working. The output is a gap map with a specific recommendation for each role: hire externally, train internally, or embed with our team.

For organisations with multiple ML use cases competing for the same engineering budget and data infrastructure, a structured prioritisation framework across four dimensions: business value (annual cost of the manual process, revenue uplift potential, decision quality improvement quantified in dollar terms); data readiness (how much preparation work before the first model can be trained?); implementation complexity (new infrastructure required vs. builds on existing pipelines); and strategic sequencing (which use cases generate data or infrastructure that reduces the cost of subsequent use cases?). The sequencing logic is where most ML programmes are designed incorrectly -- use cases are evaluated in isolation rather than as a cumulative programme. A centralised feature store built for use case one reduces the data engineering effort for use cases two through five. A shared model serving layer built for the first production model reduces deployment overhead for every subsequent model. The roadmap output is a phased 12-24 month programme with investment levels per phase, success metrics per use case, and explicit dependencies between initiatives.