Skip to content
← All posts

AI Trading EAs

EA Overfitting Explained: Why a Perfect Backtest Can Fail Live

EABuild Team · · 9 min read

EA Overfitting Explained: Why a Perfect Backtest Can Fail Live – EABuild

EA overfitting happens when an expert advisor's settings are tuned so closely to a stretch of past prices that the EA describes that history instead of a repeatable market behaviour. The backtest looks smooth because the settings were chosen to make it look smooth. This post explains how overfitting arises, how to recognise it in a Strategy Tester report, and which simple tests reduce the chance that you rely on it.

What EA overfitting means in plain terms

Every EA has inputs: periods, distances, lot sizes, time filters. Change an input and the backtest changes. If you try enough combinations on the same price history, one of them will fit that history very well, partly because of real behaviour and partly because of pure chance. Overfitting, also called curve fitting, is choosing the combination that fits the noise.

A useful comparison is a student who memorises the answers to last year's exam. The score on that exam is excellent, but the student has not learned the subject, so a new exam with different questions goes badly. An overfitted EA has memorised a particular history. Live trading is the new exam.

Why it is easy to do by accident

Nobody needs bad intentions for this to happen. The Strategy Tester in MetaTrader lets you run an optimisation (a series of test passes that vary the inputs) and sort the results by profit. The top row is tempting. But the top row is simply the pass that happened to suit the sample best.

  • The more inputs you optimise, the more combinations exist, and the easier it is to find one that fits by luck.
  • The shorter the history, the fewer distinct market conditions the EA has been forced to handle.
  • The more rules an EA contains, the more room it has to bend around individual past events.

How an optimised EA ends up tuned to old prices

Consider a simple breakout EA with five inputs. You test it on one year of M15 data and run an optimisation. One pass shows a stop-loss of one exact size, a range length of one exact value and a session filter that happens to skip three bad weeks. The equity curve looks steady.

None of those exact values has to mean anything. A stop-loss slightly larger or smaller might have produced a much worse curve. That sensitivity is the key sign: the good result sits on a narrow spike, not on a broad plateau where neighbouring values also work reasonably.

Filters that remove past losses

Time filters and news filters can be sensible, but they can also be used to cut out exactly the days that hurt the backtest. A rule that simply skips a few named weeks is hard to justify with a market reason. A rule that avoids the first minutes around scheduled releases can be explained, tested on other periods and still make sense on data the EA has never seen.

Costs that the test left out

A backtest can also look better than reality because the modelled spread, commission or slippage was lighter than what your broker charges. This is not overfitting in the strict sense, but it has a similar effect: the test result describes a market that does not exist for your account. The guides on backtesting an EA in MT5 and backtesting an EA in MT4 cover how to set up the tester so costs are modelled as closely as the platform allows.

Warning signs of an overfitted EA in a backtest report

You cannot prove overfitting from one report, but several signs together should make you cautious.

  • A very smooth curve with very few losing periods. Real markets change character. A curve with no visible stress points over a long test may have been selected rather than earned.
  • Results that collapse when you nudge an input. Change a period by a small step. If the report changes completely, the settings were sitting on a spike.
  • Many inputs with exact, unusual values. Values such as a stop-loss of one very specific size, with no stated reason, suggest the numbers came from an optimisation run.
  • Good results on one symbol and timeframe only. Some EAs are designed for one symbol on one timeframe, and that is fine. But if the logic claims a general idea and only one symbol-timeframe pair works, ask why.
  • A test period that is short or covers only one kind of market. A trend-following EA tested only during a strong trend has not met its difficult conditions.
  • Few trades. A handful of trades can look impressive and still say very little.

In-sample, out-of-sample and walk-forward testing explained

The standard defence against overfitting is to keep some data hidden from the optimisation. These terms describe how.

In-sample and out-of-sample data

The in-sample period is the stretch of history you use to choose settings. The out-of-sample period is a later stretch you do not touch while choosing. You pick your settings on the first part, then run the EA once on the second part with those settings unchanged. If the out-of-sample behaviour resembles the in-sample behaviour in character, for example similar trade frequency and similar kinds of drawdown (the drop from the account's peak balance), the fit is less likely to be an accident. If it looks like a different EA, treat the in-sample result with suspicion.

Two rules make this useful. First, the out-of-sample period must come from after the in-sample one, because the EA will trade forward in time. Second, you get to look only once. If you look at the out-of-sample result, change the settings and look again, that data has become in-sample and you have lost the protection.

Walk-forward testing

Walk-forward testing repeats the idea in steps. You optimise on one window of history, test on the window right after it, then slide both windows forward and repeat. The result is a chain of out-of-sample segments, each tested with settings chosen only from earlier data. It tells you whether the process of optimising an EA tends to produce settings that keep working, not just whether one set of values did.

MetaTrader 5 includes a forward testing option in the Strategy Tester that holds back part of the period after an optimisation. A full walk-forward routine across many windows usually needs extra scripting or external tools, so many traders do a simplified version by hand: optimise, test on the next block, move on, and compare the blocks.

Other cross-checks

  • Run the same EA on a nearby symbol or a different timeframe where the idea should also apply, and see whether the behaviour is broadly similar.
  • Test with higher spreads and some slippage to see how fragile the edge is.
  • Test over a period that includes a different market regime, such as a quiet range followed by a strong move.

EA overfitting in AI-labelled and multi-set-file EAs

Overfitting is not limited to hand-built rules. The more adjustable parts a system has, the more it can be fitted to a past sample. That includes products described as neural network or AI based, because a model with many internal parameters can fit history very closely. The AI trading EA guide explains what the label does and does not tell you.

Product listings give you facts, not test evidence, so use them to decide what to examine rather than what to expect. Three examples from the shop show different things to check.

Several set files for one symbol

The Aurum AI EA for MT4 is an MT4 expert advisor for XAUUSD on M5, listed with grid, martingale and trend following logic and 15 set files. A set file is a saved list of input settings that you load in MetaTrader. When a download offers many set files, the useful question is why they differ and whether each was chosen on a different period than the one you will test on. The listing does not say how the 15 files differ, so open each one, read the values and compare their behaviour on a demo account.

Many symbols, one timeframe

The Perceptrader AI EA for MT5 is listed for NZDUSD, USDCAD, AUDNZD and AUDCAD on M5, with grid, scalping and neural network / AI logic and 12 set files. Several symbols give you a natural cross-check: you can see whether behaviour on one pair is roughly similar to behaviour on the others, which is a small test of whether the logic is general or tuned to one pair.

One symbol, strong strategy features

The Goldex AI EA for MT5 is listed for XAUUSD on M5 with hedging, martingale and news trading, and three set files. Martingale means the lot size grows after a losing trade, so a small change in entry timing can change how deep a losing run goes. That sensitivity makes it worth testing how the EA behaves when you nudge inputs and model wider spreads around news releases.

None of this says these EAs are or are not overfitted. It says what a careful buyer would test on any EA with the same features.

Checks before trading an EA live when overfitting is a concern

A short routine will not remove the uncertainty, but it does remove the most common mistakes.

  1. Write down which settings you chose and why, before you look at the next test.
  2. Backtest on one period and keep a later period untouched, as described above.
  3. Nudge each key input up and down by a small step and look for a plateau rather than a spike.
  4. Repeat the test with higher spread and slippage.
  5. Run the EA on a demo account for a period long enough to include several different market days. The article on demo and live account testing describes how to stage that.
  6. Compare demo behaviour with the backtest: trade frequency, typical trade length and the shape of drawdowns. The numbers will not match exactly, but the character should.
  7. If you move to a live account, start with the smallest size your broker allows and keep checking.

When live behaviour differs

If a live or demo account behaves very differently from the backtest, do not immediately re-optimise on the latest weeks. That is the same trap in a new form: fitting the newest sample. First check the boring causes, such as symbol suffix, spreads, time zone and whether algo trading is enabled. If those are fine, treat the difference as information about the EA.

Running an overfitting check on a demo account this week

Pick one EA you already own or plan to test. Choose a symbol and timeframe from its listing, run a backtest on one period, hold back a later period and compare. Then run it on a demo account while you keep a short log of what you see. Any EA in the membership catalogue can be tested this way, and each is a digital download you can load on a demo account straight away.

Trading foreign exchange and CFDs carries a high level of risk and may not be suitable for all investors. Past performance does not guarantee future results; only trade with money you can afford to lose.

Tags: AI Trading EAs

← All posts

Get every EA with membership

313 products · from $16.59/month

See plans

More in AI Trading EAs