Federal Reserve Sentiment and Macro-Tracking ETFs


We try to find relationships between macro-tracking ETFs and the sentiment of the Federal Reserve Bank. We conclude that while these relationships seem to exist for certain ETFs, we need to incorporate higher-level macroeconomic variables to use these relationships as signals in practice.


Prattle provides sentiment data from central bank communications. We explore their Central Bank Sentiment Dataset: this dataset contains weekly sentiment data from many central banks around the globe (though we focus on the Federal Reserve Bank). The sentiment data wraps up all publications of the Fed from every week (between about 1998 and 2015) into a single real number representing the general sentiment of the Fed that week.

We try to find a relationship between sentiment at the Federal Reserve and various macro-tracking ETFs. We quickly find that the data itself is quite noisy. To mitigate the noise, we use Kalman-Filtered Moving Averages in our investigation. In this context, a Kalman Filter is a mathematical tool that allows us to construct moving averages that don't have the drawbacks of traditionally calculated moving averages.

We look at the following ETFs (relationship with the Fed Sentiment data included parenthetically):

  • SHY (no signal)
  • IEI (no signal)
  • IEF (possible signal)
  • TLT (probably a signal)
  • CWB (probably no signal)
  • DIA (probably a signal)
  • SPY (probably a signal)
  • GLD (probably no signal)


We conclude that finding relationships between Fed Sentiment Data and certain macro-tracking ETFs is possible, and in the cases of DIA and SPY looks quite promising, but can be problematic because of macroeconomic confounders. It seems that for the ETFs that have a reasonably close correspondence to Fed Sentiment, both Fed Sentiment and those ETFs really respond to the same general macroeconomic variables. However, they respond at different speeds -- so sometimes the Fed Sentiment responds first, at other times the ETF price responds first. Thus, when taking Fed Sentiment data as a signal, it is not easy to know whether you are moving to capture a future trend, or whether you are moving to capture a past trend. This makes it harder to trade on Fed Sentiment, though not at all impossible.


In [48]:
# Import libraries we will use
from matplotlib import pyplot
import datetime
import time
from pykalman import KalmanFilter
import numpy
import scipy
import pandas
In [49]:
# grab the prattle dataset
prattle_data = local_csv('prattle.csv')
# filter to data from the fed only
fed_data = prattle_data['frc']
In [50]:
# helper functions

def convert_date(mydate):
    return datetime.datetime.strptime(mydate, "%Y-%m-%d")

# for grabbing dates and prices for a relevant equity
def get_data(etf_name,trading_start,trading_end='2015-07-20'):
    # using today as a default arg: assuming most of the ETFs I want to inspect
    # are still trading.
    stock_data = get_pricing(etf_name,
                        start_date = trading_start,
                        end_date = trading_end,
                        fields = ['close_price'],
                        frequency = 'daily')
    stock_data['date'] = stock_data.index

    # drop nans. For whatever reason, nans were causing the kf to return a nan array.
    stock_data = stock_data.dropna()
    # the dates are just those on which the prices were recorded
    dates = stock_data['date']
    dates = [convert_date(str(x)[:10]) for x in dates]
    prices = stock_data['close_price']
    return dates, prices

# for grabbing the prattle data on which the etf has been trading.
def get_prattle(trading_start,trading_end='2015-07-20'):
    # filter down to the relevant time period
    data_prattle = fed_data[ > trading_start]
    data_prattle = data_prattle[trading_end >]
    dates_prattle = data_prattle['date']
    dates_prattle = [convert_date(str(x)[:10]) for x in dates_prattle]
    scores_prattle = data_prattle['score']
    return dates_prattle, scores_prattle


The first ETF we will test against the Prattle data is SHY. It's a 1-3 Year Treasury Bond ETF that has been trading since 2002-06-24. It's the most popular Treasury Bond ETF, storing nearly $11bn in assets.

In [51]:
# grab the SHY dataset.
dates_shy, prices_shy = get_data('SHY', '06-24-2002')
# grab the relevant prattle data.
dates_shy_prattle, scores_shy_prattle = get_prattle('2002-06-24')
In [52]:
# Lay a sentiment plot over a plot of SHY price

fig, ax1 = pyplot.subplots()
ax1.plot(dates_shy,prices_shy,c='green',label='SHY Price')
pyplot.ylabel('SHY Price')
# twinx to plot on the same graph
ax2 = ax1.twinx()
ax2.plot(dates_shy_prattle,scores_shy_prattle, label='Fed Sentiment')
pyplot.ylabel('Fed Sentiment Score')

The above graph is not particularly clear. There's a lot of noise. We'll construct some Kalman-filtered moving averages for both Fed Sentiment Score and SHY Price and overlay them to see if there are any patterns.

We'll also take this opportunity to explain the Fed Sentiment Score: Prattle collects all textual publications from a central bank, and uses natural language processing techniques to assign a sentiment score to each publication -- a positive score means a bullish market outlook, a negative score a bearish one. The further from zero, the more extreme the outlook: the sentiment scores are normally distributed about zero. The weekly sentiment score is effectively the average of all sentiment scores for publications that week.

In [53]:
# Initialize a Kalman Filter.
# Using kf to filter does not change the values of kf, so we don't need to ever reinitialize it.
kf = KalmanFilter(transition_matrices = [1],
                  observation_matrices = [1],
                  initial_state_mean = 0,
                  initial_state_covariance = 1,
In [54]:
# Filter Rolling Means
prices_shy_means, _ = kf.filter(prices_shy.values)
scores_shy_prattle_means, _ = kf.filter(scores_shy_prattle.values)

# Overlay Plots
fig, ax1 = pyplot.subplots()
ax1.plot(dates_shy,prices_shy_means,c='green',label='SHY Price MA')
pyplot.ylabel('SHY Price MA')
ax2 = ax1.twinx()
ax2.plot(dates_shy_prattle,scores_shy_prattle_means, label='Fed Sentiment MA')
pyplot.ylabel('Fed Sentiment Score MA')

At first glance, there appears to be a lagged relationship between the two moving averages. If we look at the rise, fall, and rise of the SHY price MA approximately between mid-2004 and mid-2009, we can see a correspondence to the rise, fall, and rise of the Fed Sentiment MA approximately between 2007 and 2012. However, there are two serious caveats to this observation:

  1. SHY is what we would potentially be investing in. The fact that this signal precedes the change in Fed Sentiment is not useful -- we would want it to be the other way around, i.e. for the change in Fed Sentiment to precede change in SHY.
  2. This may be a spurious correlation: looking at SHY from mid-2011 onwards, there's very little change in the MA. On the other hand, there are massive shifts in Fed Sentiment Score. This suggests that the two variables are not related.

We conclude that the Fed Sentiment Score MA does not provide an effective signal for trading SHY.


Next, we use the same procedure to test the IEI ETF against the Prattle data. Similar to SHY, IEI tracks 3-7 year Treasury Bonds. It's also quite popular, storing about $5bn in assets. IEI started trading on 11 Jan 2007.

In [55]:
# grab the IEI dataset and relevant prattle data.
dates_iei, prices_iei = get_data('IEI', '2007-01-11')
dates_iei_prattle, scores_iei_prattle = get_prattle('2007-01-11')
In [56]:
# Filter Rolling Means
prices_iei_means, _ = kf.filter(prices_iei.values)
scores_iei_prattle_means, _ = kf.filter(scores_iei_prattle.values)

# Overlay Plots
fig, ax1 = pyplot.subplots()
ax1.plot(dates_iei,prices_iei_means,c='green',label='IEI Price MA')
pyplot.ylabel('IEI Price MA')
ax2 = ax1.twinx()
ax2.plot(dates_iei_prattle,scores_iei_prattle_means, label='Fed Sentiment MA')
pyplot.ylabel('Fed Sentiment Score MA')

It is visually quite apparent that there most likely isn't a tradeable signal here. There are large trends in Fed Sentiment and IEI Price that do not match each other.

We also looked into IEF, which tracks 7-10 year treasury bonds, started trading on 2002-06-24, and stores about $6bn in assets. With IEF, we got results very similar to those for IEI. In the interest of brevity, we'll thus skip the IEF section.


Finally, we look at TLT, which tracks 20+ year treasury bonds. TLT stores about $6bn in assets. TLT started trading on 2002-06-24.

In [59]:
# grab the TLT dataset and relevant prattle data.
dates_tlt, prices_tlt = get_data('TLT', '2002-06-24')
dates_tlt_prattle, scores_tlt_prattle = get_prattle('2002-06-24')
In [60]:
# Filter Rolling Means
prices_tlt_means, _ = kf.filter(prices_tlt.values)
scores_tlt_prattle_means, _ = kf.filter(scores_tlt_prattle.values)

# Overlay Plots
fig, ax1 = pyplot.subplots()
ax1.plot(dates_tlt,prices_tlt_means,c='green',label='TLT Price MA')
pyplot.ylabel('TLT Price MA')
ax2 = ax1.twinx()
ax2.plot(dates_tlt_prattle,scores_tlt_prattle_means, label='Fed Sentiment MA')
pyplot.ylabel('Fed Sentiment Score MA')

This one is a little more promising: it looks like there is a very rough correspondence between the two time-series, though only if we shift the green one back by a few hundred days. We can think of the green peak at around 2009 to correspond to the less pronounced blue peak around mid-2007. With this shift, the following troughs and rises also correspond. We shift the TLT data back by 300 days to get a better look.

In [61]:
# Overlay Plots
fig, ax1 = pyplot.subplots()
ax1.plot(dates_tlt,prices_tlt_means,c='green',label='TLT Price MA')
pyplot.ylabel('TLT Price MA')
ax2 = ax1.twinx()
dates_tlt_prattle2 = map(lambda x: x+datetime.timedelta(days=300), dates_tlt_prattle)
ax2.plot(dates_tlt_prattle2,scores_tlt_prattle_means, label='Fed Sentiment MA')
pyplot.ylabel('Fed Sentiment Score MA')

Having overlaid the time-series, we can see some kind of rough correspondence between the two, given some time-shift. We want to figure out what time shift maximizes the correspondence. We cannot immediately apply a cross-correlation because the data from the Fed is weekly and the TLT price data is daily. We have to make some adjustments before we can come to any conclusions on TLT -- thus, we postpone this investigation to a later time (i.e. a separate notebook).

That being said, reflecting on IEI, IEF, and TLT, we may find ourselves somewhat surprised that there isn't a stronger relationship between Federal Reserve sentiment and treasury bond ETF price. We would expect that Fed sentiment would have a stronger impact on the shorter-term treasury bond ETFs.


Next, we look at CWB, which is the Barclays Capital Convertible Securities, which tracks the Barclays U.S. Convertible Bond \$500MM Index. CWB includes bonds with an outstanding issue size of more than \$500m and more than 31 days to maturity. CWB stores about \$3bn in assets. CWB started trading on 2009-04-16.

In [62]:
# grab the CWB dataset.
dates_cwb, prices_cwb = get_data('CWB', '2009-04-16')
# grab the relevant prattle data.
dates_cwb_prattle, scores_cwb_prattle = get_prattle('2009-04-16')
In [63]:
# Filter Rolling Means
prices_cwb_means, _ = kf.filter(prices_cwb.values)
scores_cwb_prattle_means, _ = kf.filter(scores_cwb_prattle.values)

# Overlay Plots
fig, ax1 = pyplot.subplots()
ax1.plot(dates_cwb,prices_cwb_means,c='green',label='CWB Price MA')
pyplot.ylabel('CWB Price MA')
ax2 = ax1.twinx()
ax2.plot(dates_cwb_prattle,scores_cwb_prattle_means, label='Fed Sentiment MA')
pyplot.ylabel('Fed Sentiment Score MA')

It's not clear whether there is a relationship here -- both moving averages follow a reasonably clear upward trajectory, and there appear to be some corresponding dips and rises. On the other hand, if we propose that the two variables are related, then CWB Price appears to be the causal variable, because wherever in this plot we see CWB and Fed Sentiment moving in some corresponding way, CWB will move first.

But that's barely plausible: a price change in CWB or TLT is unlikely to change Fed Sentiment, because these ETFs are far too small to be of economic concern. We'll discuss this in more detail in the next section.


Next, we look at DIA, which tracks the Dow Jones Industrial Average. DIA stores about \$11bn in assets. DIA started trading on 1998-01-20. Since we have data only from 2002 onwards, we'll start our investigation at 2002-03-03.

In [64]:
# grab the DIA dataset and relevant prattle data.
dates_dia, prices_dia = get_data('DIA', '2002-03-03')
dates_dia_prattle, scores_dia_prattle = get_prattle('2002-03-03')
In [65]:
# Filter Rolling Means
prices_dia_means, _ = kf.filter(prices_dia.values)
scores_dia_prattle_means, _ = kf.filter(scores_dia_prattle.values)

# Overlay Plots
fig, ax1 = pyplot.subplots()
ax1.plot(dates_dia,prices_dia_means,c='green',label='DIA Price MA')
pyplot.ylabel('DIA Price MA')
ax2 = ax1.twinx()
ax2.plot(dates_dia_prattle,scores_dia_prattle_means, label='Fed Sentiment MA')
pyplot.ylabel('Fed Sentiment Score MA')