NinjaTrader Performance

I have used this platform for many years now and as well as having a background as a professional software developer (and general tech wrangler!). I am for this to be constructive and helpful, rather than critical. 

OK, where to start!  Perhaps a little about why NT8 performance can sometimes suck, and the reasons behind it, not all of which are NinjaTraders fault.

Firstly, and without trying to get too technical, a little about how I understand Ninja is constructed and why this affects performance for some of us. Whilst Ninja is billed as ‘multi threaded’, and it is, its design can introduce a bottleneck depending on how you have it set up. If you don’t know, your computer typically has a processor that can do multiple things at the same time, from a few to many. These are called worker threads, or just ‘threads’. Ninja IS multi-threaded, so, it will update the UI, load some data, and do other stuff in different threads at the same time. The gotcha? By design, it typically does all operations for a single instrument on a single thread. This means that if you have 6 charts open for a single instrument, the data will flow to each, in turn, one by one, in that single thread. Worse – If you have multiple indicators, your data will need to through each of those, too, one, by one. Think of this as going through doors. Each tick has to pass through each door before the next can be processed. This, or the doors that are slow to open and close on the way, is typically what causes the dreaded ‘chart lag’.

Now, there are reasons they built NT this way and they probably seemed like a good idea at the time, but if you are an order flow trader with all the widgets for example and have multiple charts open for the same instrument, like me, you can get into trouble during volatility.

So, let’s cover a few of the options and how they affect you, in terms of performance.

Each time you add an indicator to the chart, you have 1 very important option. The ‘Calculate’ option. This applies to all indicators. Every tick always flows through the bars and all the indicator logic. Add an indicator to the chart with calculate set to ‘OnEachTick’, and the indicator logic will be called for every single tick. Depending on what the indicator does, this can be quite intensive, especially if poorly coded and in volatility when you can have literally hundreds of them per second coming in. For some things, like counting volume, you need this, but for others, like a moving average, you do not. The ‘OnPriceChange’ option calls the indicator logic when the price actually trades at a different price from the last price. This can avoid a few (or a lot of) ticks in the middle and an MA only needs to be updated if the price changes, anyway, so this is more efficient. OnBarUpdate is the most efficient of them all. Perhaps your MA only needs to be updated on bar close, rather than intra-bar. If you run very short time frame charts like 2 tick range bars, for instance, updating the MA on each bar close is probably the best option. This calls the indicator logic only when a new bar is formed, with the values from the bar just closed. This is the most efficient setting of them all. 

On PJSIndies, I recently introduced my own intermediate settings. I have been trading instruments with 100 ticks per point, and even OnPriceChange is too inefficient. Blurts of HFT action can cause massive amounts of small price changes in a few ms.  I introduced a ‘MinPriceChange’ setting, so indicator logic is only run when the price moves a certain distance. This gives me much more control and increased platform performance through less resource usage. Maybe NinjaTrader will adopt this! It really is a huge improvement.

Hopefully, this gives you an idea of WHY you might get lag in volatility. But, what can you do about it!?  There is a semi-useful tool built into NinjaTrader. From the control centre, from the New menu, open the Output window. Once open, right-click it and choose ‘Ninjascript utilisation monitor’. This will give you a very approximate idea of what indicators are consuming most of your resources at that time. Bear in mind, if you have 5 charts with a single same indie on them, that indie will likely be at the top! It does not mean it is broken. Also, you can expect resource-intensive things like order flow to consume more resources than a bar type, for example. These numbers are a rough guide – I don’t find them that accurate. What you should look out for is anything that is massively out of whack with the other numbers. This may give you some clues as to what is bogging down your Ninja.

So, what are a good few things you can do to increase your performance!? These vary, depending on your technical ability and how far you want to go. 

The install itself writes a lot of data to log files in real-time as the platform is running. There is no option on Ninja to change these file locations, but I use Windows File junctions to re-locate the folders to a RamDrive, which is 50x faster than even my SSDs. The log and trace folders can safely be moved in this way. This will of course likely be entirely unsupported by NinjaTrader, but I have run this way for years. Some of my indicators write data to a database. This is also hosted on the ram drive. If your code does the same, maybe you can do this too. Running your platform data from an SSD over an HDD is probably going to make a big difference, given NT data is thousands and thousands of small files which are written in real-time.

Ninjatrader also has a single internal database which stores all your trade history etc. This can get quick large, especially over years! If you can stand to lose all your trade history, you can simply reset it, or delete it. NT will re-create it on startup. If not, I have a utility on my website called NT8 cleanup that will let you remove older data, and then compact the file (it is free, but use it at your own risk).

There is a ‘cache’ folder in the db folder (careful, there may be 2 cache folders!). This can get full of old data or even corrupted over time. Delete it occasionally (when your platform is shut down, of course!).

Review all your indicators and carefully check the ‘Calculate’ option on each of them. Perhaps set all the averages you can to OnBarClose only. RSI for example might be OnBarClose, rather than OnEachTick. If you have any indies set for OnEachTick, really think about if you need that. If you are only interested in price, then OnPriceChange is probably more suited. 

Whilst you are doing the above, check a few other things. For instance, if you have ‘AutoScale’ set on your MA, did you know this can cause an additional loop through every single bar on the display to find the min and max scale value? The only scale I typically care about is price. If you are the same, turn autoscale off for these indies. Do you need price markers? I personally hate them for most plots. Again, turn them off. Anything you do not need is a waste of processing time.

Drawing objects – Oh, man – These have cost me days and money! A few things. 1, having drawing objects on your charts typically slows down your charts. This is because, with every re-render of the screen, which typically happens several times a second by default, all those drawing objects have to be looped through (if using native NT drawing objects) and any ones in the display range drawn. The indicators that used the easy route, like Draw.ArrowUp, suffer the worse. Drawing these objects natively via the graphics card (in a function called OnRender) tends to be much more efficient if done correctly, but is actually far more complex and probably out of the scope of most tinkerers.  You can check the state of play on your chart. Right-click, go to Drawing Tools, then Drawing Objects – If you have 1000’s of them in there, try to do something about it. It will be hurting your performance.

The other thing with drawing objects – They can cause the lag in a massive way, if you get a corrupt one, even if it is off-screen. I believe the reason for this is the constant exceptions internally that can occur when the object is processed (I’ve done some debugging on this!). For instance, something you drew last week, but your chart is only showing the last few days. If you get unexplained lag you don’t normally experience, try going to each individual chart and right-click, drawing tools, and remove all drawing objects. Answer Yes! I personally have had instances where I forgot to do this and suffered for a couple of days with massive intermittent issues, which were all miraculously fixed, just by clearing the drawing objects. This has happened more than once to me, so, it is certainly something to remember, if you do run into issues, and perhaps especially if you have third-party drawing tools.

A trick – If you have a CPU with enough cores, and thus a few threads to spare, you might try this if you are trading the indexes, for example. If you were trading Nasdaq futures, there is the MNQ and the NQ. Now, prices on each are typically at most a tick or 2 different, and generally only in volatility? Do you get where I am going? Some of your price display charts can be NQ, and others, MNQ. Depending on which one you trade and which you track order flow on, if you do, will dictate the mix, but you might be able to get NT to spread the load in this manner and it is something I do.

A hardware consideration. Bearing in mind the threading model, if you have many charts open for a single instrument, and mainly trade one instrument at a time, a processor with fewer cores and more Ghz may provide your NT setup with a bit more go than a slower clock core with more cores. IMO, despite some comments I’ve seen, modern graphics cards appear to be barely taxed by NT, even with 6+ monitors (I typically get about 3% max usage on a modest card with that many)

Charts – A 60-minute bar chart with a few hundred bars will not consume much in the way of resources. A 2 tick unirenko chart, with thousands of bars and creating new ones at a great pace will consume quite a few resources!  Especially if you have indies on that chart too. (remember, each indie will have multiple buckets (series) for every single bar!) More bars, more data, more things to update and track on each tick. Try to limit the number of bars on your chart. For things like unirenko, you can set the numbers of bars back to the display in the data series settings, rather than a fixed number of days. For time charts, if you just need today, set this to 0. Do not load more data than you need.

Volume profiles, etc. You might need tick data for these in the short time frame, but for composite profiles, typically 1-minute data resolution is fine. You’ll save a ton of resources and your charts will load a lot quicker. Bear in mind these often still use tick data live, just 1 minute for the historical fill. If your volume profiles allow, merging ticks in high tick instruments can help with performance (if the indie is coded well) and your trading. On NQ, do you need to see .25 increments? Is .50, or even 1 point fine? On the composite, do you need to see every tick level? I don’t.  If your volume profile allows, setting the price resolution to a higher step can mean it has to work less in terms of the number of levels and lines it draws, the data it has to maintain, etc. It can also be less data for YOU to process!

Let me finish part 1 of this, by saying, Ninjatrader8 is the best platform available to me, but with great flexibility and a total lack of quality of code filters, you can produce indies that will grind NT to a halt, fairly quickly. It can take some time to figure out the pitfalls of coding for NT8.  However, if done correctly, most of the performance issues can be worked around (but it can be challenging!). I do hope they continue to develop and improve the platform. I honestly do not know of anything better for my purposes.

Now, if you get through this and DON’t have lag, you might notice one thing. The chart STILL lags! OK, maybe by a couple of hundred milliseconds, but this is, again, by design. I believe NT typically refreshes the chart at most, once every 250 milliseconds, even if the indies are running their logic much, much faster. This makes sense, to some degree. Now, the Ninja team believe this is perfectly adequate, but my personal preference is for a faster screen update. There have been studies that show the human eye/brain can react amazingly fast – much faster than you can actually ‘see’ something. I know that sounds weird, but it’s correct.  The way I scalp and look at very short-term price action, especially in this day and age of HFT etc., I want to see the wiggles. I’ve learned to interpret them, and I cannot see them if the chart only updates a few times a second. If you like, you can get my free price line indicator (on the website) to optionally FORCE an update more frequently. I typically go from 50-100 ms, on my entry chart ONLY. Now, bear in mind, that this is going to introduce even more strain on your workstation, forcing NT to redraw all the indicator plots and bars more times than it typically would out of the box. However, if you watch short-term price action, it can really bring your chart alive!

Good luck – If this helps, please let me know. I might do part 2 🙂