Timestral: Mistral release time-series foundation models

Timestral, Mistral’s archetypal family of time-series basis models, delivers state-of-the-art probabilistic, multivariate forecasts alongside fair 209M parameters, outperforming larger models on key benchmarks during supporting former and forthcoming covariates. Available in four sizes (5M to 209M parameters), it balances accuracy and efficiency, operating efficiently on CPUs and excelling in zero-shot forecasting for varied domains. The example combines synthetic pre-training, covariate-aware attention, and long-context learning, alongside fine-tuning options for domain adaptation via Mistral Forge.

Today, we’re releasing Timestral, our archetypal family of time-series basis models. Our largest model, at fair 209M parameters, ranks archetypal on all three of the field's chief community benchmarks, GIFT-Eval, TIME, and fev-bench, outperforming the former province of the art alongside lone two-thirds of the example parameters.

Timestral delivers probabilistic, multivariate forecasts and supports past-only and known-future covariates. Our family of models sits on the Pareto forefront of forecasting accuracy versus example size, delivering foremost achievement at all size. Results have been submitted to the GIFT-Eval and TIME leaderboards.

Time Series Foundation Models (TSFM)

Time sequence is among the most average data types in enterprise, from sensor telemetry to observability metrics. Forecasts built on it run decisions in capability planning, energy, provision chains, hazard and maintenance.

Industrial flora forecast procedure sensors against the manufacturing schedule, capture equipment drift before it fails, and scheme capability and spare parts throughout thousands of assets. Finance teams forecast flows, volumes, and volatility to communicate positions, hedging, and treasury throughout thousands of accounts.

Classical models specified as ARIMA and linear regression remain well-established tools for forecasting period series. These models are normally fitted to a sole sequence and study lone from that series' history.

Deep learning models specified as PatchTST add the capability to study nonlinear patterns throughout many connected series, but they are motionless typically trained from scratch on all dataset, so what they study doesn't transport complete to new domains.

Time sequence basis models bring the pre-training method to period series. Trained on a ample and varied corpus, our example learns patterns that transfer throughout series, forecasting zero-shot on data it has never seen, equal in domains anywhere data is scarce.

Benchmark Results

We evaluated Timestral on the three chief community forecasting benchmarks: GIFT-Eval, TIME and fev-bench. Among pre-trained, zero-shot models, it ranks archetypal throughout all five metrics. These results arrive from a sole checkpoint, without ensembling.

GIFT-Eval, top ten of 83 comparable pretrained and zero-shot entries. Lower is better.

TIME, top ten of the 35 leaderboard entries. Lower is improved on the two metrics.

fev-bench, top ten of the 37 leaderboard entries, by SQL accomplishment score. Higher is better.

Family of models optimised for all specification

Timestral has been optimised as a family of four models: 5M, 28M, 123M and 209M parameters. The larger models enhance forecast performance, during the smaller ones are faster and cheaper. All four portion the identical architecture and training recipe. Every size sits on the Pareto forefront of accuracy versus example size: for all example size, no community example forecasts better.

Score vs example size on GIFT-Eval, TIME and fev-bench (shown as 1 − accomplishment score). Lower is better. Our 28M example strikes Chronos-2, concerning four times larger, on GIFT-Eval and TIME. Our 209M example strikes TimesFM-3 and Falcon-2.0 alongside small parameters.

Timestral does not need a GPU at inference. On a norm server CPU alongside 8 threads, forecasting one day onward from three weeks of 15-minute former takes 10 ms alongside the 5M example and 140 ms alongside the 209M model. Smaller models commerce several accuracy for speed: the 28M example is five times faster than the 209M model, and the 123M example comes inside 0.2% of the 209M mark at three quarters of its latency.

An manufacturing example

Consider an manufacturing factory alongside thousands of sensors throughout its equipment. Timestral can enhance the forecasts on that data in multiple ways.

Multivariate forecasting. A pump's vibration, heat and force don't move independently. A increase in one frequently explains or causes a alter in another. Timestral models connected sequence jointly, so it can use those requirements alternatively of forecasting all sensor in isolation.

Probabilistic forecasting. A sole predicted value may not display the complete picture. Timestral predicts a complete set of quantiles, so teams can see how broad the doubt is, set vigilant thresholds that indicate it, and scheme care about hazard fairly than a sole number.

Categorical covariates. Some covariates obtain values from a set of categories fairly than uninterrupted numbers, specified as the day of the week, or a machine's province (running, idle, under maintenance). Encoding these as numbers would enforce an command or measure they don't have. Timestral takes them as categorical inputs immediately to enhance forecasts.

Known-future covariates. Some drivers of sensor behavior are known in advance, specified as manufacturing schedules, planned shutdowns, climate forecasts. Timestral can circumstance on them to enhance forecast quality, so a scheduled burden addition is expected fairly than flagged as a surprise.

From Model to System

Timestral is a numerical example that can run inner existing monitoring pipelines, or be paired alongside our tongue models in multi-modal agentic workflows.

In the plant, that could appearance akin this:

1. Timestral operating continuously on the plant's sensor streams, detecting anomalies as they occur.

2. When it flags an anomaly on a pump, an delegate embeds the signals and compares against embeddings of former sensor data to discover akin historic failures.

3. The delegate afterward starts sub-agents to run a base logic analysis, example on environment from that plant, specified as care records, change logs and controller notes, equipment manuals, former activity orders, and readings from connected equipment.

4. The delegate drafts a suggested assessment according to the findings, alongside supporting evidence and imaginable next steps.

The factory director gets a assessment and a scheme backed by the sensor data, fairly than fair an alert.

Key specialized contributions

Several ideas have been blended to create these results. Each volition be described in item in the specialized report.

1. Decoupled Horizon Forecasting. Causal decoders trained alongside adjacent place masking, as in Toto 2.0 and TimesFM-3, disguise out the values to be forecast, so they appearance akin missing values and are additionally misplaced as environment for afterward forecasts. Our approach, DHF, separates the two: the complete former stays in the sequence, and learned forecast tokens peruse the former before them. Every observed value thus serves as the two mark and context, true gaps remain flagged as missing, and training horizon lengths and missing-data augmentation can be tuned independently of all other.

2. Covariate-aware attention. Each input sequence is assigned a role: mark variate, past-only covariate or known-future covariate. Known-future covariates are apparent throughout the entire opening during targets remain causal, and data flows lone from covariates to targets. Timestral can hence use a holiday calendar or a climate forecast without leaking a target's own forthcoming into its forecast.

3. Synthetic-first data. The bulk of the training data was synthetic. We archetypal generated univariate processes alongside coherent temporal dynamics, including trends, seasonality and varied noise patterns. We afterward used a multivariate synthesizer to build multivariate sequence by introducing nonlinear requirements between these processes.

4. Two-stage training. Timestral is archetypal pre-trained at measure on purely synthetic data alongside environment size 8192, afterward post-trained on a mix of genuine and synthetic data alongside a 4x longer context. The goal is a pinball defeat complete 21 quantiles, a discrete approximation of CRPS.

A example that can be adapted

Public benchmarks measure broad performance, but manufacturing data can be considerably distinct in behavior and structure. It may additionally change complete period as markets alter regime, equipment degrades and petition patterns move. The enduring value lies in being capable to modify a example to in-domain data and keep retraining it as the allocation moves.

For organisations that desire to go further, Timestral is accessible through Mistral Forge, the identical training pipeline we use for our tongue models. It supports:

• Zero-shot inference out of the box.

• Architectural extensions, specified as domain-specific covariates.

• Fine-tuning on proprietary data, specialising to a domain during preserving broad performance.

• Control complete the training data, including adjusting the training mix, replaying pre-training data, and generating synthetic data samples to indicate the mark domain. Full authority complete the data ensures that nothing we backtest was seen during pre-training.

The complete pipeline can be deployed on our clients’ hardware, including air-gapped environments.

What’s next

We have published a GitHub repository so you can run Timestral on your own data. A specialized study alongside the complete results volition follow. For additional specialised forecasting problems, or to collaborate on research, delight attain out to us.