Derivative Lab Sentiment Analysis Blog

Bubble Trouble! Exploring an LPPL model for Crypto

CryptoTheorem group releases Bitcoin Bubble Index and Crypto Bubble Index.

bitcoin bubble index

This is an example of historical aggregated data between 2015 and 2018 for Bitcoin USD price was used from cryptocompare.com, giving a total of 1063 data points. To test the predictive power of the model, backtesting was done on 4 key peaks in 2017 corresponding to drawdowns of at least 20% or more. The LPPL model was then used to generate 30, 60 and 90 days forward estimates to examine the best time period window to predict near term crashes. The drawdowns (orange lines) are shown below among with the current time period represented by the green line at time index 1063 (ie 2017-11-28).

To learn more please their sites:

The Log-Periodic Power Law (LPPL) model is used to see whether the prica action for Bitcoin and for cryptocurrencies in general follows a log-periodic oscillation model for describing the characteristic behavior of a speculative bubble and predicting its subsequent crash. The new index from CryptoTheorem analyzes Bitcoin and cryptocurrency prices from 2015 to 2018 and generates LPPL simulations to compare various forward windows to estimate the critical time period indicative of a potential crash.

During the development phase, backtesting was done for four crashes in 2017 and the 60 days window was found to be a better balance of estimating a critical price peak versus a critical time window. The median critical time was found to be 26 days from the last price point (ie. Christmas Eve!) with an estimated peak price level.




Sentiment Analysis Trends

derivativelab | 2017-12-27 16:38

According to Google Trends, the word "sentiment analyis" has been gaining steady traction over the past 5 years. Sentiment refers to the attitude expressed by an individual regarding a certain topic.

When it is applied to trading, sentiment can simply be used as a directional signal to figure out whether you should go long or short within your portfolio. The most conventional way sentiment has been applied to trading is to long stocks when there are positive information on a company and short stocks when there are negative information on a company. A typical behavioral assumption is assuming that if there are positive information on a particualar company, it will draw other traders to purchase the stock and in return increase the stock pice. Whereas, if there are negative information, it will put fear into traders and tempt them to sell their stocks which in return will decrease the stock price of that particular company.

As human, we can easily interpret the attitude of an article. However, the problem lies when we have 10 or even 1000 companies in our portfolio, most of which are being mentioned frequently in the news or social media sties. The average adult reads 300 wpm and takes approximately 2 minutes to read an article with 600 words. Within 2 minutes, 10 of your portfolio companies may have generated over 100 articles or 100,000 mentions across social media sites. It is impossible for humans to keep up with the massive data overload which is where sentiment analysis comes into play.

Bag-of-words is one of the most conventional approaches to sentiment analysis. On a very basic level, it identifies positive and negative words within an article. The sum of the positive and negative words (assuming positive words holds a weight of 1.0 and negative words holds a weight of -1.0) is your sentiment score. This score is typically normalized on a scale from 1.0 to -1.0.

Many quantitative hedge funds have been incorporating the use of sentiment analysis in their trading strategies and it is a growing trend. The total time it takes a human to read 1 article and interpret its attutude towards a certain matter, a computer can read millions of articles and identify a specificsentiment score or attitude for each.

A common trend within the quantitative trading community is combining sentiment data with other types of data sources to bring a edge to a specific strategy.

Quantitative traders are becoming more receptive to incorporating sentiment indicators into their trading models and the trend will continue in the future as more traders are actively looking for an edge in the market. Many startups are competing in this space to provide sentiment and other types of innovative analytics for quantitative traders to utilized in their strategies.
And the growth of data is steadily increasing, making it more attractive to use innovative analytics such as sentiment to generate alpha.




Todays crypto-markets are driven by social chatter

Ever noticed how things in crypto get really exciting, people start making crazy predictions, and suddenly at its highest point, everything turns for the worst?

Then at the pit of despair, when the crowd thinks all is lost, fortunes turn again? We call this the "sentiment wave" and Derivative Lab is all about measuring social chatter so that you can make ideal trading decisions.

Hype, fear and greed dominate the cryptocurrency price action. Derivative Lab delivers clarity, with market sentiment content and data feeds that cannot be found anywhere else.

Trade with confidence

Stop trading in the dark. Get a regular stream of curated market news and insights, with updates on blockchain happenings, trends and sentiment. All avaialble in easy-to-grapsh newsletters, articles, and reports, and aggregated from across the world.

Never be blinden by the crypto-markets again

Derivative Lab gives you the daily pulse of cryptocurrency and blockchain markets. Get exclusive content streams from a network of crypto trading and investing experts

Minimize Risk, Maximize Profit

Think you can find market patterns, but just need the data to prove it? Are you a quantitative trader, looking to trade at speed and scale? Waith no longer. Derivative Lab brings you hard numbers as we build the world's largest library of datafeeds, specifically for cryptocurrencies and blockchain assets.




Why should you care about Market Sentiment?

A lot of investors mention the word "sentiment". But do you really know what is means, and how you can use it wisely as a trader? In short, Market sentiment is the feeling or tone of the market, or its crowd psychology, as revealed through the activity and price movement of the securities traded in that market.

Sentiment is an important feature of the financial trading system. Trading and pricing were always mostly depended on social interaction. Many times in recent history, it was the sentiment of the investing crowd that has driven prices up or down, rather than any other technical or fundamental event.

If you are new to sentiment, the basic theory is as follows. If we see excessive bullishness, it means most investors are probably close to fully invested. Conversely, if we see excessive bearishness, then most people probably sold already. If everyone has bought, then buying power becomes weak. If everyone has sold, then selling pressure drops.

Traditionally the sentiment has been though to predict. However, as a social trading network we make it easier to measure this sentiment very accurately simply by collecting the data from across our social trading network. This is the main idea in using wisdom of the crowds for trading.

How to read sentiment?

In our platform, the sentiment in a market is summed up as an index. It reflects what is going on right now in the cryptocurrency community. This allows our traders to get an immediate overview of the direction in which the crowd expect the market to head.

However, it is important to remember the sentiment does not predict the movement of the market accurately. Sometimes it might even point to the opposite. For example, sometimes the market price will clearly be on the rise, as the crowd sentiment will still be bearish rather than bullish.

What to do with this information?

The way in which you interpret and use the sentiment information is up to you and your willingness to take on risk exposure. Depending on your trading strategies, you can act in accordance with the crowd and sell in a bear market, or you can go against the masses and buy in the bear market.

If the sentiment feature tells you that many people selling this means the price will likely go down and you may be looking forward to a unique buying opportunity for a cryptocurrency you have been considering for a long time.

We hope you use this information wisely, and trade better with our sentiment indexes in our platform.




Sentiment Trading Strategy Examples

Simple Strategy

First, you can build a very basic strategy that simply buys when sentiment is superb, holds, and sells once sentiment gets into negatives. To mimic this, You can write a simple strategy, which just do this. If the cryptocurrency sentiment analysis signal is at 100, the strongest possible positive, then a buy is initiated, using 10% of the portfolio's available cash, and a stop-loss of 0.5% is set. This means that we would be invested in up to 10 assets at any one time. We could try to make the strategy a bit smarter by dynamically adjusting investment size according to available investments, but we'll keep it basic for now. After buying, the strategy holds the asset until sentiment hits -1 or lower, and then a sell is initiated.

Sentiment Threshold

Example 1: At the market open each day, assets with a sentiment score above the positive threshold are entered long and assets with a sentiment score below the negative threshold are entered short. Equal weight is applied to the long and short legs. These positions are assumed to be liquidated at the day's market close.

Exponential Moving Average Sentiment

Example 2: Using sentiment data from 9:10 EST which looks at an exponentially weighted sentiment aggregation over the last 24 hours. Each stock is separated into its respective quintile based on its sentiment index in relation to the universe. A long-short portfolio is constructed in a similar fashion as previously with long positions in the top quintile assets and short positions in the bottom quintile assets. Deutsche Bank's Quantitative Research Team recently released a paper about strategies that solely use SMA data which includes a long-term strategy.




Process of real-time social news analysis

Every day, millions of news are generated, and all these news are produced by humans in natural language. These news contain the information that can help on the field of decision making, algorithmic trading and risk management. Consider one piece of news may trigger a cryptocurrency price explosion. Or a hiding trend may be buried in a huge pile of news data. For now, these works are often done by professional analysts. But the task become harder and harder as soaring flows of news. That's why we need the automated and quantitative news analysis.

Let's first put aside how to do the automated analysis of news. Instead, think about what does a reader need to know from a piece of news. Normally, at the starting point, he or she may answer these two questions:

1. What's objective that the news is talking about? For example, Bitcoin or Ethereum?

2. In generatl, is it bad or good?

The technology of Named Entity Recognition is for answering the first question. More specifically, quickly determining which item in the text maps to proper names, such as people or places. For us, we need to go further to determine which cryptocurrency is involved in the news. We decouple the task into two parts:

1. Use the popular community package like NLTK and Stanford NER to narrow down the searching

2. Search for the cryptocurrency name using our crypto synonym database.

After the Named Entity Recognition process, the news will be documented under the identified cryptocurrency name for delivery or further analysis. Sometimes, on news mentioned several cryptocurrencies. In this scenario, relevance measure conducted. The relevance measure considers the location of a term in the text. For example, intuitively, one news may be more relevant to the cryptocurrency asset when the name of the token occurs in the title

To know whether a news is bad or good to a cryptocurrency, a common way is to search for the emotional states such as "angry", "sad", and "happy" and count on the occurrence of these states. In our case, we first collect a library of these emotional states specialized in the financial community. Next, we count on all the words that both in the library and text. Then normalize the counting result for both positive and negative words, where score -100 means that all words are negative and score 100 means that all are positive. These scores can be treated as quantitative measure of sentiment that can be used to compare between companies and time.

Finally, both Named Entity Recognition and sentiment scoring process is completed on the distributed computational clusters so that the analyzing result can be delivered and documented in real time.




Reading Bitcoin Sentiment | Long vs Short Positions




Sentiment Moving Average

Technical Analysis is the process of using charts and indicators based on the price and volume to predict the future movements of financial assets

It bears similarities with Sentiment Analysis as Technical Indicators are used to guess the opinions of other investors. For example, a large volume with increasing price could signify that investors are largely positive and bullish about the asset/

As such, Technical Indicator is a proxy for Sentiment Analysis.

This post will hope to show that incorporating Sentiment Analysis into a simple Technical Strategy can improve its performance

Strategies to be compared

Strategy A
If (5-day SMA > 10-day SMA): Long Asset
Else: Short Asset

Strategy B
If (5-day SMA > 10-day SMA) and (2-day Average Sentiment > A1): Long Asset
Else If (5-day SMA > 10-day SMA) and (2-day Average Sentiment < A2): Close Position
Else If (5-day SMA <= 10-day SMA) and (2-day Average Sentiment < B1): Short Asset
Else If (5-day SMA <= 10-day SMA) and (2-day Average Sentiment > B2): Close Position

The basis of both strategies is when the short-term moving average is above the long-term moving average, the asset price is on an upward path. Hence, we want to follow the trend and long the asset. Conversely, we want to short the asset in the other case.

The difference between the strategies is that B has additional entry and exit conditions for when the sentiment data does not agree with the moving average condition. A 2-day average is used here to smooth out noise which could cause the strategy to change positions constantly.

It is obvious that Strategy A will always have open positions whereas Strategy B will occasionally have no positions, instead of holding cash undil another opportunity arises.

Testing Methodology

Sentiment data from the Derivative Lab API will be used for the comparison of the 2 strategies. A1, A2, B1, B2 are selected to be 5, 4, 6, 7 respectively for Strategy B.

For the asset Bitcoin ($XBT) is used.

When either Strategy enters a position, the position size will be equal to 90% of the portfolios value. This is to simulate the leaving of some cash holdings to prevent for margin calls.




Using the Daily Sentiment Index




When Cryptocurrency Trading Meets Deep Learning

The automated analysis of textual data and its application in business analytics holds great promise for providing decision-makers with information from a sheer endless stream of news available online. Recent advances in computing have led to exciting new tools in the areas on Natural Language Processing. Sentiment Analysis and Machine Learning that can be used to make sense from an ever growing number of online sources. In a series of blog posts we will be looking at the basic concept of sentiment analysis in general and in the context of cryptocurrency trading, offer a behind-the-scenes view into Derivative Lab's very own efforts in building a sentiment engine, introduce a number of case studies and provide a glimpse into the future of machine learning and Artificial Intelligence (AI).

News Analytics and Sentiment Analytics

It is widely recognized that new information plays a key role in financial markets, impacting on volumes of trade, returns and volatility of prices, and consequently news has always been a key source of investment information.

While decision-makers have always considered a portfolio of varied news domains and sources, with the growth of the Internet, the amount of readily available information has grown exponentially. As major news outlets increasingly bolster their online portfolio, newspapers articles are increasingly published onle. Bloomberg alone adds an estimated 1 million news stories a day.

Apart from news produced by these reputable sources, an increasing number of opinionated documents of interest are published online asynchronously 24/7/365 on blogs, message boards and micro blogs (e.g. Twitter or Stocktwits) by large and varied user communities.

The enormity and high variance of this data presents an interesting opportunity for harnessing it into a foam that allows for specific market predictions, and in recent years this information has been repeatedly demonstrated to dramatically influence markets. A phenomenon recently termed "collective intelligence". For a single person (or even a group of people) harnessing this information successfully is increasingly impossible due its volume and asynchronous character. The necessity for automated collection, extraction, processing and aggregation of this hata has long been recognised and advances in machine learning techniques have led to exciting new tools for its analysis.

Challenges

It is widely accepted that the automated extraction of useful information from text is a complex challenge. Apart from technical constraints, word-sense disambiguation remains on of the major challenge in the computerised processing of textual unstructured data. Words frequently change their meaning depending on context, consequently changing the meaning of the surrounding body. In fact, functional structures are so complex that it builds the latest stage in infant learning acquisition, with most of us needing over 2.5 years to master even the simplest applications. For example, one would not consider the word "long" as eithere exceptionally positive or negative. However, most humans would rate "the laptop's start-up time was long" as negative, while "the laptop's battery life was long" would be considered positive.

Sentiment Analysis and Natural Language Processing

Netural Language Processing (NLP), Sentiment Analysis and Machine Learning are widely recognised as the key tools in transforming the plethora of available text into meaningful information.

Sentiment Analysis (also known as Opinion Mining), seeks to identify and categorise opinions expressed in a piece of text, especially in order to determine whether the writer's attitude towards a particular topic is positive, negative, or neutral. Sentiment Analysis builds on NLP, originally referred to as computational linguistics, and at its most basic refers to the use of algorithms that allow computers to process and understand human languages.

While being researched since the late 1940's, the field has been rapidly growing over the last decade. This can be attributed to:

  • the advent of the Internet and the increased availability of large amounts of electronic text; and
  • the availability of computers with increased speed and memory, and the consequent improvement in Machine Learning. Also, frequently referred as Artificial Intelligence (AI) or narrow AI, Machine Learning, describes the practice of using algorithms to parse data, learn from it, and then make a classification or prediction. So, rather than hard-coding a specific set of instructions for a particular task, the machine is "trained" using large amounts of data and algorithms that give it the ability to learn how to perform the task.

NLP Techniques

Over the last decade, various NLP algorithms have been developed, combined with Machine Learning techniques and applied in numerous (commercial) applications. For example many popular spam filters include Naive Bayes classifiers trained with NLP developed features to identify spam email. A similar approach can be used in Sentiment Analysis to classify text into positive, negative and neutral sentiment polarity. Although these models tend to treat words as atomic units - no notion of word similarity, or text structure - researchers and practitioners argue that they frequently outperform more complex models, at higher computational efficiency, and consequently are most applicable to.

As Machine Learning classification techniques require large quantities of relevant in-domain data for training the highly varied and specialized topics in market news present a unique challenge. A recent approach has benne provided by Google's Word2Vec algorithm, that has the potential to allow practitioners to overcome these limitations. It takes a text corpus as input and produces the word vectors as output, by constructing a vocabulary from a training data set and then learning vector representation of the contained words. This results in the capture of many syntactic and semantic regularitieis, represented by vectors. For example, 'melancholy' would be closest to 'bittersweet' (sentiment) as opposed to 'thoughtful' and 'warm' (semantic). When properly employed this approach can effectively aid in the capture of sentiment words not provided during training.

Future

NLP, Sentiment Analysis and Machine Learning are fields heavily investigated and substantial breakthroughs are to be expected over the next years. It should be noted that in academic settings, more complex algorithms, such as Recurrent Neural Network based language models, have already shown promising results. However, up-to-date computational complexity does not permit their use in robust applications relying on near-real time processing of information.




How important is toread market sentiment towards stocks?




Intraday Impuslive Sentiment Trading

Tick-by-tick Sentiment data can allow us to identify entry points throughout the day when there are spike in Sentiment. These spikes can be caused by the release of good or bad news that will drive trading and price movement.

This post will look to examine the performence of an impulsive sentiment strategy, trading when there are large spikes in sentiment, on the cryptocurrency pair BTCUSD.

How to measure spikes in Sentiment

A spike in sentiment simply means that there is a sudden and large change in sentiment. Hence, we will use the following measure:

Change in Sentiment = Current Sentiment - 10 tick average Sentiment

The average sentiment is a measure of the value that the sentiment should be at. If the last tick of sentiment is much larger or smaller than this average, it means that there is a large spike on the last tick

Another interpretation of the measure is that it is a measure of relative sentiment. The concept of Relative Sentiment means that when we look at current sentiment data, we need to also consider the sentiment data of the recent past before we can determine how good or bad current sentiment data is. For example, if the past sentiment has been extremely bad for successive days and the sentiment today is not as bad, we will view this as a good sign and cound the current sentiment as good even though the absolute value of the sentiment is not high.

When the current sentiment is above the average, the current sentiment is improving from the past. By the concept of Relative Sentiment, this means that the sentiment for the asset is good. Coversely, when the current sentiment is below the average, the current sentiment is poorer than the past and by the convept of Relative Sentiment, this means that the sentiment for the asset is bad.

Straightforward Trading Strategy

We construct the following trading strategy:

Whenever we receive a new tick of sentiment, we evaluate the change in sentiment.

If (Change in Sentiment) > 20: Long the Asset at end of current bar

If (Change in Sentiment) < -20: Short the Asset at end of current bar

Testing Methodology

We test using 15-minute bars of tick-by-tick price data of BTCUSD and tick-by-tick sentiment data from the Derivative Lab's API and backtest from 1/1/2017 to 12/1/2017

Performance

Unfortunately, it appears that our straightforward strategy does not seem to work well. At the same time, it appears to be extremely consistent. Therefore, a good change is to reverse our positions taken, taking on a contrarian approach instead.

Contrarian Trading Strategy

For our contrarian strategy, we look to take reverse positions to our initial strategy:

Whenever we receive a new tick of sentiment, we evaluate the change in sentiment.

If (Change in Sentiment) > 20: Short the Asset at end of current price bar

If (Change in Sentiment) < -20: Long the asset at end of current price bar

Performance

Unsurprisingly, simply reversing the trades taken managed to turn our previously terrible strategy around. A simple explanation for the success of the contrarian approach is overreaction to news. The market simply overreacts to both good and bad news leading to prices and either increase or devrease too much. These prices will return to fair value in time and this allows our contrarian strategy to profit from this inefficiency.

This also strong evidence of the speed that the market reacts to news. As we enter positions on the end of the current price bar, at the worst case, our strategy will enter positions 15 minutes after the release of news. Even being 15 minutes slower is too much to be able to catch the price movements.

Conclusion

While tick-by-tick sentiment trading can detect price movements, the market reacts to news extremely fast. Straightforward strategies of buying when sentiment is good and selling when sentiment is bad might not be able to catch price movements before they occur. For such an approach, speed of executions is extremely important and a high frequency setup will be required.

For lower frequency trading, consider a contrarian approach to news as there can and always will be overreaction of both good and bad news.




DISCLAIMER

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Derivative Lab. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Derivative Lab nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Derivative Lab about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Derivative Lab makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Bitcoin Sentiment Index
bitcoin sentiment
Ethereum Sentiment Index
ethereum sentiment
Litecoin Sentiment Index
litecoin sentiment
Bitcoin Cash Sentiment Index
bitcoin cash sentiment
Ripple Sentiment Index
ripple sentiment
IOTA Sentiment Index
iota sentiment