Demystifying Volatility: A Brief Dive into Predicting Market Movements
As you know, I wrote a series about time series before, after these articles, I wanted to give an idea about how we can keep ourselves in a safe area in potentially risky situations with portfolio analysis. I wanted to tell you how we can guess in Python.
If you want to access previous articles, you can use the links below:
Then what is this Volatility?
Volatility sits at the heart of financial markets. It serves dual roles — as an informative beacon for investors and as a crucial component in numerous financial models. But why does volatility command such attention? The answer is rooted in one word: uncertainty. This element of unpredictability is fundamental to the mechanics of financial models.
With the global integration of financial markets, we’ve seen an amplification of uncertainty, underscoring the role of volatility. This term describes the intensity of fluctuations in the value of financial assets and often stands as a symbol of risk. It’s no exaggeration to say that volatility has taken center stage in realms like asset pricing and risk management.
Now, if you’ve delved into the finance world before, you might have come across terms like ‘volatility clustering’ or ‘information asymmetry’. Historically, we’ve leaned on traditional models like ARCH and GARCH to forecast volatility. However, they aren’t without their flaws. Given the recent market turbulences and the strides made in machine learning, there’s been a renewed interest in improving the way we predict volatility
Modeling Volatility with Python
This piece will delve into an exploration of both traditional and contemporary models — think SVMs — for volatility forecasting. To effectively model volatility, it’s imperative to identify the components or ‘features’ in our model. This essentially means modeling uncertainty itself to offer a clearer lens into real-world predictions.
So, how do we gauge the efficacy of these models? Enter ‘return volatility’ or ‘actual volatility ’, defined by the formula:
Where:
- ‘r’ stands for return
- ‘mean(r)’ represents the average return
- ’n’ is the total number of observations
Case Study: Impact of the Ukraine War on DAX
In a continuation of my previous article, I’ve chosen Germany’s most important stock index, DAX.
The aim? To gauge the reverberations of the Ukraine war on the economic landscape. For this purpose, I’ve gathered DAX data spanning from early 2012 to the start of 2023, analyzing the realized volatility throughout this period.
The Importance of Accurate Volatility Estimation
The methodology behind estimating volatility plays a pivotal role in the precision and reliability of subsequent analyses. In the ensuing sections, I will juxtapose classical forecasting techniques with cutting-edge, machine-learning-backed methods. The goal? To shed light on the prowess of machine learning models in predicting market shifts post-March 1, 2022.
0. Data Prep
We will collect our data again through yahoo finance, in this context, we provide the necessary inputs and access the data.
stocks = '^GDAXI'
start = datetime.datetime(2012,1,1)
end = datetime.datetime(2023,1,1)
dax = yf.download(stocks, start, end, interval='1d')and data seems like that:
Then we calculate the percentage change according to the daily closing values.
ret = 100 * (dax.pct_change()[1:]["Adj Close"])
realized_vol = ret.rolling(5).std()
1. Traditional Models
ARCH Model
The ARCH model, which stands for Autoregressive Conditional Heteroskedasticity, is a statistical model used to estimate variance. It was introduced in 1982 by economist Robert Engle. The ARCH model is particularly useful in financial econometrics for modeling financial time series that exhibit time-varying volatility, such as stock returns.
In many financial time series, large changes tend to be followed by large changes (of either sign), and small changes tend to be followed by small changes. This phenomenon is called “volatility clustering.” Traditional time series models like ARMA (Autoregressive Moving Average) are designed to capture patterns in the mean of a time series, but they do not handle volatility clustering well. This is where the ARCH model comes in.
ARCH Model Formulation:
The ARCH(q) model can be represented as:
Where:
- y_{t} is the return at time t
- \epsilon_{t} is the error term or white noise at time t
- \sigma _{t}^{2} is the conditional variance of t given the past values of the series
- z_{t} is a white noise error term with zero mean and unit variance
- q indicates the number of lags of the squared error term
The main idea is that the variance of the error term \epsilon_{t} is not constant, but rather a function of past squared errors.
n = (datetime.datetime.strptime('2023/1/1', "%Y/%m/%d") - datetime.datetime.strptime('2022/3/1', "%Y/%m/%d")).days
print(n)
split_date = ret.iloc[-n:].index
arch = arch_model(ret, mean='zero', vol='ARCH', p=1).fit(disp='off')
print(arch.summary())
bic_arch = []
for p in range(1, 5):
arch = arch_model(ret, mean='zero', vol='ARCH', p=p)\
.fit(disp='off')
bic_arch.append(arch.bic)
if arch.bic == np.min(bic_arch):
best_param = p
arch = arch_model(ret, mean='zero', vol='ARCH', p=best_param)\
.fit(disp='off')
print(arch.summary())
forecast = arch.forecast(start = split_date[0])
forecast_arch = forecast
rmse_arch = np.sqrt(mse(realized_vol[-n:]/100,
np.sqrt(forecast_arch.variance.iloc[-len(split_date):]/100)))
print("The RMSE value of ARCH model is {:.4f}".format(rmse_arch))
The result of volatility prediction based on our first model is like that.
GARCH Model
A natural extension of the ARCH model is the GARCH (Generalized ARCH) model. The GARCH(p, q) model combines the ARCH model with an autoregressive process, allowing for a more flexible structure to capture the time-varying volatility.
Here, p and q are the orders of the GARCH and ARCH processes, respectively.
Both ARCH and GARCH models have become foundational financial econometrics for modeling and forecasting volatility. They’re essential tools in risk management, derivatives pricing, and many other areas of finance.
garch = arch_model(ret, mean='zero', vol='GARCH', p=1, o=0, q=1).fit(disp='off')
print(garch.summary())
bic_garch = []
for p in range(1, 5):
for q in range(1, 5):
garch = arch_model(ret, mean='zero',vol='GARCH', p=p, o=0, q=q)\
.fit(disp='off')
bic_garch.append(garch.bic)
if garch.bic == np.min(bic_garch):
best_param = p, q
garch = arch_model(ret, mean='zero', vol='GARCH',
p=best_param[0], o=0, q=best_param[1])\
.fit(disp='off')
print(garch.summary())
forecast = garch.forecast(start=split_date[0])
forecast_garch = forecast
rmse_garch = np.sqrt(mse(realized_vol[-n:] / 100,
np.sqrt(forecast_garch \
.variance.iloc[-len(split_date):]
/ 100)))
print('The RMSE value of GARCH model is {:.6f}'.format(rmse_garch))
The volatility of returns is well-fitted by the GARCH model, partly because of volatility clustering and partly because GARCH does not assume that returns are independent, which allows it to account for the leptokurtic property of returns.
GJR-GARCH (Glosten-Jagannathan-Runkle GARCH):
The GJR-GARCH model was introduced by Glosten, Jagannathan, and Runkle in 1993. It modifies the GARCH model to allow for different reactions in volatility to positive and negative shocks.
The model can be represented as:
Where gamma controls for the asymmetry of the announcements and if:
- gamma = 0, the response to the past shock is the same
- gamma > 0, the response to the past negative shock is stronger than a positive one.
- gamma < 0, the response to the past positive shock is stronger than a negative one.
bic_gjr_garch = []
for p in range(1,5):
gjrgarch = arch_model(ret, mean='zero', p = p, o=1, q=q).fit(disp='off')
bic_gjr_garch.append(gjrgarch.bic)
if gjrgarch.bic == np.min(bic_gjr_garch):
best_param = p,q
gjrgarch = arch_model(ret, mean='zero', p=best_param[0], q=best_param[1], o=1).fit(disp='off')
print(gjrgarch.summary())
forecast = gjrgarch.forecast(start=split_date[0])
forecast_gjrgarch = forecast
rmse_gjr_garch = np.sqrt(mse(realized_vol[-n:]/100,
np.sqrt(forecast_garch.variance.iloc[-len(split_date):]/100)))
print("The RMSE value of GJR GARCH model is {:.6f}".format(rmse_gjr_garch))
EGARCH (Exponential GARCH)
The EGARCH model, proposed by Nelson in 1991, allows for asymmetry in the volatility response to shocks by modeling the log of the conditional variance rather than the variance itself. This ensures that the conditional variance remains positive.
The model can be represented as:
The main difference in the EGARCH equation is that the logarithm is taken of the variance on the left-hand side of the equation. This indicates the leverage effect, meaning that there exists a negative correlation between past asset returns and volatility.
bic_egarch = []
for p in range(1,5):
for q in range(1,5):
egarch = arch_model(ret, mean='zero', vol='EGARCH',
p=p, q=q).fit(disp='off')
bic_egarch.append(egarch.bic)
if egarch.bic == np.min(bic_egarch):
best_param = p,q
egarch = arch_model(ret, mean='zero', vol='EGARCH',
p=best_param[0], q=best_param[1]).fit(disp='off')
print(egarch.summary())
forecast = egarch.forecast(start=split_date[0])
forecast_egarch = forecast
rmse_egarch = np.sqrt(mse(realized_vol[-n:]/100,
np.sqrt(forecast_egarch.variance.iloc[-len(split_date):]/100)))
print("The RMSE value of EFARCH model is {:.6f}".format(rmse_egarch))
Successes of traditional models on our Case
+-----------+--------+
| MODEL | RMSE |
+-----------+--------+
| ARCH | 0,1207 |
| GARCH | 0,1228 |
| GJR-GARCH | 0,1228 |
| EGARCH | 0,1222 |
+-----------+--------+2. ML Model
Support Vector Machines (SVM)
Support Vector Machines (SVM) are a class of supervised learning algorithms originally designed for binary classification problems. However, their application has been extended to regression problems, termed Support Vector Regression (SVR), and they have found use in a wide variety of fields, including volatility forecasting.
In financial econometrics, accurate forecasts of volatility are essential for risk management, portfolio optimization, option pricing, and many other applications. Traditional methods for volatility forecasting include the GARCH family of models, as discussed earlier. More recently, machine learning techniques like SVM have been employed to improve the accuracy of forecasts.
Kernel Trick: SVMs, including SVR, use what’s known as the “kernel trick” to transform the data into a higher-dimensional space to find a hyperplane that best fits the data. Common kernels include linear, polynomial, and radial basis function (RBF). In the context of volatility forecasting, the choice of kernel can have a significant impact on the forecast accuracy.
Advantages of SVM in Volatility Analysis:
Non-Linearity: Financial time series data often have non-linear patterns that linear models may not capture effectively. SVMs can handle non-linear relationships due to their kernel trick.
Few Assumptions: Unlike some traditional time series models, SVM doesn’t make strong assumptions about the data distribution.
Flexibility: SVM can incorporate various features, not just past returns, providing flexibility in designing the input feature set.
from sklearn.svm import SVR
from scipy.stats import uniform as sp_rand
from sklearn.model_selection import RandomizedSearchCVrealized_vol = ret.rolling(5).std()
realized_vol = pd.DataFrame(realized_vol)
realized_vol.reset_index(drop=True, inplace=True)retruns_svm = ret ** 2
returns_svm = retruns_svm.reset_index()
del returns_svm["Date"]X = pd.concat([realized_vol, returns_svm], axis = 1, ignore_index=True)
X = X[4:].copy()
X = X.reset_index()
X.drop("index", axis = 1, inplace = True)realized_vol = realized_vol.dropna().reset_index()
realized_vol.drop('index', axis=1, inplace=True)svr_poly = SVR(kernel='poly',degree=2)
svr_lin = SVR(kernel='linear')
svr_rbf = SVR(kernel='rbf')
a. Linear SVM
Characteristic: Linear SVR tries to find a linear hyperplane that best separates the data in the feature space.
Use Case: Suitable for data that has a linear relationship between the features and the target variable.
para_grid = {'gamma':sp_rand(),
'C': sp_rand(),
'epsilon': sp_rand()}
clf =RandomizedSearchCV(svr_lin, para_grid)
clf.fit(X.iloc[:-n].values,
realized_vol.iloc[1:-(n-1)].values.reshape(-1,))
predict_svr_lin = clf.predict(X.iloc[-n:])
predict_svr_lin = pd.DataFrame(predict_svr_lin)
predict_svr_lin.index = ret.iloc[-n:].index
rmse_svr =np.sqrt(mse(realized_vol.iloc[-n:]/100,
predict_svr_lin/100))
print("The RMSE value of SVR with Linear Kernel is {:.4f}".format(rmse_svr))
b. RBF SVM
Characteristic: The RBF kernel can map the input data into an infinite-dimensional space. It can capture very complex relationships without needing to specify the exact transformation. Essentially, it considers the similarity between instances and can create non-linear decision boundaries.
Use Case: Suitable for data that doesn’t have a clear linear or polynomial relationship. It’s a popular choice because of its flexibility and capability to handle varied data shapes. However, careful tuning of the γ parameter is crucial, as it can greatly impact the performance of the SVR.
para_grid ={'gamma': sp_rand(),
'C': sp_rand(),
'epsilon': sp_rand()}
clf = RandomizedSearchCV(svr_rbf, para_grid)
clf.fit(X.iloc[:-n].values,
realized_vol.iloc[1:-(n-1)].values.reshape(-1,))
predict_svr_rbf = clf.predict(X.iloc[-n:])
predict_svr_rbf = pd.DataFrame(predict_svr_rbf)
predict_svr_rbf.index = ret.iloc[-n:].index
rmse_svr_rbf = np.sqrt(mse(realized_vol.iloc[-n:] / 100,
predict_svr_rbf / 100))
print('The RMSE value of SVR with RBF Kernel is {:.6f}'
.format(rmse_svr_rbf))
c. Polynomial SVM
Characteristic: It maps the input data into a higher-dimensional space using a polynomial function and then tries to find a hyperplane that best fits the data in this transformed space
Use Case: Suitable when the relationship between the features and the target variable is polynomial or has some sort of curved nature. The degree d of the polynomial determines the complexity of the curves
para_grid = {'gamma': sp_rand(),
'C': sp_rand(),
'epsilon': sp_rand()}
clf = RandomizedSearchCV(svr_poly, para_grid)
clf.fit(X.iloc[:-n].values,
realized_vol.iloc[1:-(n-1)].values.reshape(-1,))
predict_svr_poly = clf.predict(X.iloc[-n:])
predict_svr_poly = pd.DataFrame(predict_svr_poly)
predict_svr_poly.index = ret.iloc[-n:].index
rmse_svr_poly = np.sqrt(mse(realized_vol.iloc[-n:] / 100,
predict_svr_poly / 100))
print('The RMSE value of SVR with Polynomial Kernel is {:.6f}'\
.format(rmse_svr_poly))
Conclusion
Understanding volatility is crucial to grasping the intricacies of the financial market, primarily because it offers a measurable lens to uncertainty. This is not just a theoretical concept; It actively informs a large number of financial models, especially risk-centered ones.
While traditional models such as ARCH and GARCH attract attention, their rigid structure often limits their adaptability. It is heartening to observe that modern methodologies, such as the SVM techniques we reviewed, yield promising results that outperform even the best traditional models.
Stay tuned! In my next post, we’ll dive deeper into Neural Networks and MCMC models, juxtaposing their capabilities with SVM to discern the next frontier in volatility prediction.
References
- Andersen, Torben G., Tim Bollerslev, Francis X. Diebold, and Paul Labys. 2003. “Modeling and Forecasting Realized Volatility.” Econometrica 71 (2): 579–625.
- Bollerslev, T. 1986. “Generalized Autoregressive Conditional Heteroskedasticity.” Journal of Econometrics 31 (3): 307–327. 3): 542–547.
- Black, Fischer. 1976. “Studies of Stock Market Volatility Changes.” 1976 Proceedings of the American Statistical Association Business and Economic Statistics Section.
- Karasan, Abdullah, and Esma Gaygisiz. 2020. “Volatility Prediction and Risk Management: An SVR-GARCH Approach.” The Journal of Financial Data Science 2 (4): 85–104.
- Karasan, Abdullah (2021). Machine Learning for Financial Risk Management with Python Oreilly
- Taylor, S. 1986. Modeling Financial Time Series. Chichester: Wiley
Subscribe to DDIntel Here.
DDIntel captures the more notable pieces from our main site and our popular DDI Medium publication. Check us out for more insightful work from our community.
Register on AItoolverse (alpha) to get 50 DDINs
Support DDI AI Art Series: https://heartq.net/collections/ddi-ai-art-series
Join our network here: https://datadriveninvestor.com/collaborate
