In this article, a mixed methodology that combines both the Autoregressive Moving Average Model (ARMA) and Neural Network Regression (NNR) models is proposed to take advantage of the unique strength of ARMA and NNR models in linear and nonlinear modelling. Experimental results with real data sets indicate that the combined model can be an effective way to improve forecasting accuracy achieved by either of the models used separately. The purpose for this article is to investigate the use of alternative novel neural network architectures when applied to the task of forecasting and trading the Athens Stock Exchange (ASE) 20 Greek Index using only autoregressive terms as inputs. This is done by benchmarking the forecasting performance of six different neural network designs representing a Higher Order Neural Network (HONN), a Recurrent Neural Network (RNN), a classic Multilayer Perceptron (MLP), a mixed-HONN, a mixed-RNN and a mixed-MP neural network with some traditional techniques, either statistical such as a an ARMA, or technical such as a Moving Average Convergence/Divergence (MACD) model, plus a naïve trading strategy. More specifically, the trading performance of all models is investigated in a forecast and trading simulation on ASE 20 time series over the period 2001 to 2008 using the last one and a half year for out-of-sample testing. We use the ASE 20 daily series as many financial institutions are ready to trade at this level and it is therefore possible to leave orders with a bank for business to be transacted on that basis. As it turns out, the mixed-HONNs do remarkably well and outperform all other models in a simple trading simulation exercise. However, when more sophisticated trading strategies using confirmation filters and leverage are applied, the mixed-MLP network produces better results and outperforms all other neural network and traditional statistical models in terms of annualized return. On the other hand, the Hybrid-HONNs shows a superiority after all sophisticated strategies, as filters and leverage, have been used in terms of annualized return as Dunis et al. (2010) mention in a recent paper.