Time series forecasting (eventually with python)

Question

• What algorithms exist for time series forecasting/regression ?
  • What about using neural networks ? (best docs about this topic ?)
  • Are there python libraries/code snippets that can help ?

Answer 1

The classical approaches to time series regression are:

• auto-regressive models (there are whole literatures about them)

• Gaussian Processes

• Fourier decomposition or similar to extract the periodic components of the signal (i.e., hidden oscillations in the data)

Other less common approaches that I know about are

• Slow Feature Analysis, an algorithm that extract the driving forces of a time series, e.g., the parameters behind a chaotic signal

• Neural Network (NN) approaches, either using recurrent NNs (i.e., built to process time signals) or classical feed-forward NNs that receive as input part of the past data and try to predict a point in the future; the advantage of the latter is that recurrent NNs are known to have a problem with taking into account the distant past

In my opinion for financial data analysis it is important to obtain not only a best-guess extrapolation of the time series, but also a reliable confidence interval, as the resulting investment strategy could be very different depending on that. Probabilistic methods, like Gaussian Processes, give you that "for free", as they return a probability distribution over possible future values. With classical statistical methods you'll have to rely on bootstrapping techniques.

There are many Python libraries that offer statistical and Machine Learning tools, here are the ones I'm most familiar with:

• NumPy and SciPy are a must for scientific programming in Python
• There is a Python interface to R, called RPy
• statsmodel contains classical statistical model techniques, including autoregressive models; it works well with Pandas, a popular data analysis package
• scikits.learn, MDP, MLPy, Orange are collections of machine learning algorithms
• PyMC A python module that implements Bayesian statistical models and fitting algorithms, including Markov chain Monte Carlo.
• PyBrain contains (among other things) implementations of feed-forward and recurrent neural networks
• at the Gaussian Process site there is a list of GP software, including two Python implementations
• mloss is a directory of open source machine learning software