Multimodal Transformer with Attention for High-Frequency Bitcoin Price Forecasting: Integrating Market, Sentiment, and Macroeconomic Signals

https://doi.org/10.5281/zenodo.18174404

Abstract

Forecasting Bitcoin prices for the near term is not easy because of the crypto market's rapid and complex nature. Therefore, I have created a Transformer model that can predict hourly Bitcoin prices with the help of attention. It gets into the price data as well as market, sentiment, and VIX (Volatility Index) factors. A comparison with classical methods such as ARIMA, XGBoost, and LSTM was performed in order to assess the model's performance. The outcome indicates that Transformer model outperformed all the others with a MAE of $3.26 and a RMSE of $4.57. Moreover, the model's attention mechanism reveals the factors it is most focused on, which allows better understanding of the predictions it makes. Thus, it seems that Transformers are "wizards" in this kind of rapid market guessing, offering not only accurate forecasts but also some insights into the underlying driver(s) of the predictions, which turns out to be beneficial for decision-making.