Warmup and convergence

[DaveSkender]

Important

Warmup and convergence periods occur in most indicators in our library. Using the minimum amount of quote history as possible is NOT a good way to optimize your system.

If you were to put in the minimum amount of quote history data that you think is needed, it may not be enough to achieve desired precision -- this is not a bug or error in the formula, it's just part of the way the stochastic math works sometimes.

There are two reasons that initial indicator values may be missing or less accurate:

1. Missing: they simply can't be calculated. For example, a simple moving average of 10 periods can't be calculated until there are 10 periods of data to calculate; therefore, the first 9 periods return as null in our library.
2. Less accurate: initial values are less accurate for indicators that use prior values of itself (recursive). This is because they rely on an initial guess on the first calculable period, but will converge over time to be accurate.

See individual indicator documentation for information on what to expect and our pruning utilities to help you manage it.

more on convergence

This chart demonstrates. The center dark line is the precise indicator value, and the top and bottom dark lines reflect how an initial guess value for the indicator might begin and then become accurate.

Example: ADX(14)

• You think, I should provide 14 (N) periods of historical quotes to "optimize" for ADX(14), right?

• So you try that and you get nothing in return, it's only a null result. You're in the incalculable zone.

• You then read the formula and can see that it starts to calculate values at the 2×N mark and you get 58.53; great! but there is a hidden problem that you may not know ... this value is wildly inaccurate! You're in the convergence zone.

• If you then incrementally add more historical quotes to see this pattern you will see that precision to 2 decimal needs 150 historical quotes, and you'd need even more if you wanted even better precision.

  ADX(14) on 09/04/2020 with    5 historical quotes: 
  ADX(14) on 09/04/2020 with   14 historical quotes: 
  ADX(14) on 09/04/2020 with   28 historical quotes: 58.52588577
  ADX(14) on 09/04/2020 with   40 historical quotes: 45.90300880
  ADX(14) on 09/04/2020 with   50 historical quotes: 50.80068945
  ADX(14) on 09/04/2020 with   75 historical quotes: 40.83090019
  ADX(14) on 09/04/2020 with  100 historical quotes: 40.02238283
  ADX(14) on 09/04/2020 with  120 historical quotes: 40.15183256
  ADX(14) on 09/04/2020 with  150 historical quotes: 39.94376039
  ADX(14) on 09/04/2020 with  200 historical quotes: 39.94161678
  ADX(14) on 09/04/2020 with  250 historical quotes: 39.94163282
  ADX(14) on 09/04/2020 with  350 historical quotes: 39.94163320
  ADX(14) on 09/04/2020 with  500 historical quotes: 39.94163320

This is why we recommend for ADX that you use 2×N+100 periods of historical quotes before your intended usage period. Please keep this in mind as this library does not provide warnings or Exceptions when do not provide enough quotes to achieve moderate accuracy.

[DaveSkender]

I have done the work of evaluating all the indicators in the library and providing guidance in the documentation about how much is needed to provide moderate and high levels of precision, it's usually written in the following format, and will be unique for each indicator.

Historical quotes requirements

You must have at least 2×N+100 periods of quotes to allow for smoothing convergence. We generally recommend you use at least 2×N+250 data points prior to the intended usage date for better precision.

If you want to test these on your own, here's example code:

IEnumerable<Quote> quotes = [provide at least 1000 quote periods];

int[] quotesQuantities =
    new int[] { 5, 14, 28, 40, 50, 75, 100, 120, 150, 200, 250, 350, 500, 600, 700, 800, 900, 1000 };

foreach (int qty in quotesQuantities)
{
    IEnumerable<Quote> evalQuotes = quotes.TakeLast(qty);
    IEnumerable<AdxResult> results = evalQuotes.GetAdx(14);

    AdxResult last = results.LastOrDefault();
    Console.WriteLine(
        "ADX(14) on {0:d} with {1,4} historical quotes: {2:N8}",
        last.Date, evalQuotes.Count(), last.Adx);
}

We have done this evaluation on all relevant indicators. You can also see and run the convergence test code in our repo.

[DaveSkender]

If you wanted to exclude these warmup periods from your results, here's how we recommend doing it:

// fetch historical quotes from your feed (your method)
IEnumerable<Quote> quotes = GetHistoryFromFeed("SPY");

// calculate 14-period ADX
int n = 14;
IEnumerable<AdxResult> results = quotes
  .GetAdx(lookbackPeriods: n)
  .RemoveWarmupPeriods(2*n+100);

We use 2*n+100 here because the documentation recommends to not use the first 2×N+100 periods, where N = lookbackPeriods. You can specify any number you want here.

Without the removal of warmup periods, you will get a result set that is identical in size as the original quotes provided. When removing the warmup periods, it will be 2×N+100 smaller, in this case.

See .RemoveWarmupPeriods() docs in the Guide for more info.

[DaveSkender]

[ankurmalhotra89]

May I know then how charting Tools like Trading View would calculate. And specially for such stocks which dont have a huge history data available. Example,
I am checking the supertrend on a weekly chart. The first week data available on exchange is of 11 Oct 2021. symbol name is ABSLAMC from NSE India.
https://in.tradingview.com/chart/?symbol=NSE%3AABSLAMC

Now trading view calculates the supertrend and i dont know what smoothening it applies. But whatever formulae it may have hidden behind for smoothening, the data available is limited for both trading view and [Stock.Indicators] library

I am using the code below

 public static IEnumerable<SuperTrendResult> CheckSupertrend(List<StockData> quotes)
       {
           IEnumerable<SuperTrendResult> results = quotes.GetSuperTrend(10,3);

           return results;
       }

My result is off from what i see on Trading view. Attaching both raw weekly feed data of the same symbol and the export of the supertrend computation

Raw:
ABSLAMC.NS_1wk.csv

Library computed result :
ST_week.csv

Couldnt export data from Trading view as Essential plan doesn't support it. Please check it once there if needed
Let me know if any other info is needed

Thanks

[DaveSkender]

SuperTrend (ST) with a lookbackPeriod of 10 is calculated from an underlying Average True Range (ATR). ATR(10) is initialized with an approximate value, by averaging the first 10 True Range (TR) values. TR can’t be calculated on the first bar. As a result, the first possible ST value is on the 11th bar. Subsequent ATR values are calculated with a smoothed moving average, but since the first value was an approximation, it takes many more bars to diminish the approximation error.

[ankurmalhotra89]

Yeah i agree. I just want to know if I have the complete data that exists ( of say a particular stock like the one in my example) , is it possible to come very close to what TradingView is populating.
Why I am saying this ( and also emphasize) is because both we and Trading view have the exact same data range available as historical data, nothing more and nothing less.

Just to add , What I am noticing is that even after ignoring the first 15 Weekly rows of data, library computed data is different from the Trading View Data

[DaveSkender]

Regarding the difference between TradingView and this library, please review my prior analysis, demonstrating an error in the TradingView formula..

[ankurmalhotra89]

Thanks a lot Dave.