Data Analysis for Stock Price Prediction

Abstract

In the field of applied mathematics, modeling random processes naturally arises in a variety of settings. Specifically within the context of financial markets, an intriguing pursuit is to model patterns of stock prices — which are characterized by high levels of noise and randomness. This thesis attempts to provide a sound methodology to forecast stock returns by examining a wide range of information and defining a systematic approach. In attempting to uncover patterns in movements of stock prices and forecast returns, I first engineer a variety of different input features which are derived from sources ranging from the historical time series of stock prices to alternative sources of information such as options data and analyst recommendations. In this thesis, I note the input features which provide the strongest insights in forecasting returns through several different methods. After analyzing the strongest input variables, I reassess if the obvious methodology of forecasting stock prices through a regression model is attainable. After this, an initial model is fit to the data. Consistent with the ”No Free Lunch” theorem — which states that there is no optimal algorithm for all problems — I test a variety of different machine learning algorithms to see which works best on my dataset, evaluating several different metrics. After identifying the best performing model, a variety of different tests are conducted to improve the performance of the model — including outlier detection and removal, feature elimination, and dimensionality reduction. Finally, I optimize the output of the model in a distinctive manner that is best suitable for future work.