Shipment ETAs predicted to 21.4 h RMSE across 304K shipments and 60+ lanes on SageMaker

Client story

A customer of our client, one of the global leaders in agricultural equipment manufacturing, could not predict shipment delivery times reliably across its supply chain network. Traditional ETA estimates broke down under the number of variables involved: shipping lanes, carrier service types, geographic factors and transit status updates. The consequences were:

• Reduced supply chain visibility and planning efficiency
• Difficulty coordinating multi-stop international shipments across the US, Europe and Asia
• Manual processes for tracking shipments through transit statuses
• Inability to use historical shipment data for predictive insight
• Difficulty optimizing vendor service across multiple carriers

The company needed a scalable, data-driven way to predict shipment ETAs from historical patterns, geographic data and real-time transit information, to improve operational decisions and customer satisfaction.

We used Amazon SageMaker to build machine learning models (linear regression and XGBoost) that predict shipment estimated time of arrival from historical transit data.

Challenge

Accurate ETA prediction meant overcoming data inconsistencies, multi-stop complexity and the need to scale ML across international lanes. The key lessons:

Technical lessons

• Data quality is critical. Geocoding accuracy strongly affects model performance. We had to correct city and country mappings manually and filter out shipments with missing coordinates. In the first iteration this reduced the raw dataset to roughly 4,400 usable shipments; the production models were later trained on 304,441 unique shipments.
• Feature engineering matters more than model complexity. Turning raw transit data into meaningful features (shipment lanes, distance calculations, business-day indicators) had more impact than model choice. The 64-feature set, including categorical encodings for lanes and service codes, proved effective.
• Multi-stop shipments are hard. The initial implementation excluded multi-stop pickups and deliveries and modelled single PICKEDUP to DELIVERED flows. Future iterations should incorporate multi-leg shipment patterns.
• Simpler models can win. Linear regression and XGBoost achieved similar RMSE (about 21 to 24 hours). Linear models may be preferable for interpretability in production logistics environments.
• Endpoint costs need active management. Automatic shutdown of unused SageMaker endpoints prevented unnecessary charges during development.

Operational lessons

• Reusable pipelines. Jupyter notebooks for data cleaning, geocoding and feature preparation enabled rapid iteration and retraining.
• Account-based routing. A Lambda function that routes by account ID gives a flexible, multi-tenant architecture for scaling to additional customers.
• Validation rules. Transit status filtering (PICKEDUP, INTRANST, DELIVERED) and temporal consistency checks were essential for training-data quality.

Solution

We developed an ML-powered predictive ETA solution on AWS that forecasts delivery times from historical shipment data.

• Data layer. Amazon Redshift stores orders, transit updates, pickup and delivery events, and geo-data.
• Data processing. Python ETL in SageMaker notebooks handles extraction, geocoding, distance calculations and the 64 features (lanes, service codes, business days).
• ML training pipeline. SageMaker trains linear regression (distance and categoricals) and XGBoost (tuned hyperparameters); models and datasets are saved to S3.
• Inference layer. SageMaker endpoints serve real-time predictions; a Lambda function (predictShipmentETA) routes requests by account ID.

Team: Senior Python Engineer, two Python Engineers, Data Scientist, Software Architect.

Architecture and tech stack

Redshift feeds SageMaker notebooks for ETL and feature engineering; SageMaker trains the models and stores artefacts in S3; SageMaker endpoints serve predictions behind an account-routing Lambda that exposes a REST API to the shipment management system.

• Amazon SageMaker
• AWS Lambda
• Amazon Redshift
• Amazon S3
• Python
• XGBoost and linear regression
• Jupyter notebooks

Results

The solution deployed to production and cut costs by removing the need for on-premises GPUs (avoiding $10K to $50K of CapEx), upfront hardware and maintenance. Pay-as-you-go SageMaker with auto-shutdown endpoints, serverless Lambda routing and automated predictions reduce the operational overhead of manual ETA processes and scale elastically.

Model performance

• Linear regression achieved an RMSE of 21.4 hours on the test dataset of 80,889 shipments
• XGBoost achieved an RMSE of 23.9 hours with 50 training rounds
• Models were trained on 304,441 unique shipments across multiple international lanes
• The pipeline processes 64 engineered features, including distance, geographic data and service codes

Operational outcomes

• Automated ETA prediction
• Real-time predictions via Lambda routing to SageMaker endpoints
• Multi-region support covering US, European and international lanes (60+ lane combinations)
• Account-based model deployment, so additional customers can be added without a linear increase in cost
• Production-ready REST API integration with the existing shipment management system

Business impact

• Better supply chain visibility through data-driven ETA forecasting
• Better coordination of multi-stop international shipments
• A foundation for predictive analytics across the agricultural equipment logistics network
• Scalable ML infrastructure on AWS for future model improvements and additional use cases

The project validated ML-based shipment prediction and established a framework for continuous model refinement as more historical data accumulates.

What made it work

Erbis’s takeaways from this engagement:

• Spend the effort on data, not on models. Geocoding fixes, status filtering and lane features moved the error more than any algorithm change.
• Prefer the interpretable model when accuracy is close. Planners can act on a linear model they understand; a marginal gain from a black box is rarely worth it.
• Design for multiple tenants from the first deployment. Routing by account ID meant adding customers did not mean adding infrastructure.
• Control inference cost explicitly. Auto-shutdown of idle endpoints is a small feature that kept the pay-as-you-go promise true.