100 Years of dow jones returns

A quick look at annual returns over the 100+ years of daily percent change (close to close) data that we have on dow jones

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import datetime

dj = local_csv("DjiaHist.csv", date_column = "Date", use_date_column_as_index = True)
dia = get_pricing("DIA", start_date = "2016-01-01", end_date = datetime.date.today(), frequency = "daily")

First cleaning up the data, especially the dates. Also adding day of the year into the df in order to sort all returns based on the day of the year and plot em all at once later

dj.sort_index(ascending=True, inplace=True)
dj.index = pd.to_datetime(dj.index)
dj.rename(columns={"Value" : "value"}, inplace=True)
dj["pct"] = np.log(dj["value"]).diff()
dj["year"] = dj.index.year
dj["day"] = dj.index.dayofyear

dia["day"] = dia.index.dayofyear
dia["pct"] = np.log(dia["price"]).diff()
dia = dia.drop(["open_price", "high", "low", "volume", "close_price"], axis=1)
dia.set_index(dia["day"], inplace = True)
dia.fillna(0, inplace=True)

Pivoting the returns table, so that we get returs for all years and all days of the year in separate columns

daily_rets = pd.pivot_table(dj, index=["day"], columns=["year"], values=["pct"])
daily_rets.convert_objects(convert_numeric = True)
daily_rets.fillna(0, inplace = True)
daily_rets.columns = daily_rets.columns.droplevel()
daily_rets.drop(2016, axis =1, inplace = True)
daily_rets.rename(columns = lambda x: str(x), inplace=True)

daily_rets.head(8)

screen-shot-2016-11-07-at-12-28-50

Heres how it looks with all years plotted along with 2016

f, ax = plt.subplots(figsize=(18, 12))
ax.plot(daily_rets.cumsum(), color="#333333", linewidth=1, alpha=0.1, label=None)
ax.plot(dia["pct"].cumsum(), linewidth=2, color="crimson", label="2016 returns")
plt.grid(False)
plt.ylabel("Annual return")
plt.xlabel("Day of the year")
plt.ylim(-0.7, 0.7)
plt.xlim(0, 365)
plt.axhline(0, linewidth= 1, color="#333333", linestyle="--")
plt.legend(loc="upper left")

unknown

Adding the mean returns of all years so one can compare with 2016.
Also added a daily returns histogram so the historical day to day fluctuatios are more clear and positive or negative periods are painted out clearly

daily_rets["mean"] = daily_rets.mean(axis=1)
daily_rets["2016"] = dia["pct"]

plt.figure(figsize=(18, 12))

ax1 = plt.subplot2grid((4,1), (0,0), rowspan=3)
ax1.plot(daily_rets.index, daily_rets.cumsum(), color="#333333", linewidth=1, alpha=0.06, label=None)
ax1.plot(daily_rets["mean"].cumsum(), color="#333333", linewidth=2, alpha=0.8, label="Mean returns since 1896")
ax1.plot(daily_rets["2016"].dropna().cumsum(), linewidth=2, color="crimson", label ="2016 returns")
plt.title("Cumulative 2016 Returns Vs Mean Historical Returns Since 1896")
plt.axhline(0, linewidth= 1, color="#333333", linestyle="--")
plt.ylim(-0.15, 0.15)
plt.grid(False)
plt.legend(loc="upper left")

ax2 = plt.subplot2grid((4,1), (3,0), rowspan=3, sharex=ax1)
ax2.fill_between(daily_rets.index, 0, daily_rets["mean"], where= daily_rets["mean"]<0, color="crimson")
ax2.fill_between(daily_rets.index, daily_rets["mean"], 0, where= daily_rets["mean"]>0, color="forestgreen")
plt.title("Mean Daily Returns")
ax2.grid(False)
plt.xlim(1, 365)

unknown-1

Now that the show is over, its time to look at returns around elections. Up to 1936 the votes were cast in early january, from there on the vote has been in early november, so i used returns from 1936 onward in the calc. Also plotted the mean return of post-election year

daily_rets["el_year"] = daily_rets.loc[:, "1936"::4].mean(axis=1)
daily_rets["post_el"] = daily_rets.loc[:, "1937"::4].mean(axis=1)

f, ax = plt.subplots(figsize=(18, 12))

ax.plot(daily_rets.index, daily_rets.cumsum(), color="#333333", linewidth=1, alpha=0.06, label=None)
ax.plot(daily_rets["mean"].cumsum(), color="#333333", linewidth=2, alpha=0.8, label="Mean returns since 1896")
ax.plot(daily_rets["el_year"].cumsum(), color="darksage", linewidth=2, alpha=0.8, label="Election year mean returns since 1936")
ax.plot(daily_rets["post_el"].cumsum(), color="steelblue", linewidth=2, alpha=0.8, label="Post election year mean returns since 1936")
ax.plot(rets_df["pct"].dropna().cumsum(), linewidth=2, color="crimson", label ="2016 returns")
plt.grid(False)
plt.ylabel("Annual return")
plt.xlabel("Day of the year")
plt.ylim(-0.15, 0.15)
plt.xlim(1, 365)
plt.axhline(0, linewidth= 1, color="#333333", linestyle="--")
plt.legend(loc="upper left")

unknown-2

We can also pull up decade returns. 80’s and 90’s were good times indeed. Applied a 21 day mean to the returns to the trends would be more clear

def decadeMean(start, end):
return daily_rets.loc[:, start : end].cumsum().mean(axis=1)

decade_rets = pd.DataFrame({#"1900’s" : decadeMean("1900", "1909"),
#"1910’s" : decadeMean("1910", "1919"),
#"1920’s" : decadeMean("1920", "1929"),
#"1930’s" : decadeMean("1930", "1939"),
#"1940’s" : decadeMean("1940", "1949"),
#"1950’s" : decadeMean("1950", "1959"),
#"1960’s" : decadeMean("1960", "1969"),
"1970’s" : decadeMean("1970", "1979"),
"1980’s" : decadeMean("1980", "1989"),
"1990’s" : decadeMean("1990", "1999"),
"2000’s" : decadeMean("2000", "2009"),
"2010’s" : decadeMean("2010", "2015")
}, index= daily_rets.index)

mean_rets = decade_rets.rolling(21).mean()

plt.figure(figsize=(18, 12))
mean_rets.plot(linewidth=1)
rets_df["pct"].dropna().cumsum().rolling(21).mean().plot(color="crimson", linewidth=2, label="2016")
plt.legend(loc="upper left")
plt.ylabel("Annual return")
plt.xlabel("Day of the year")
plt.grid(False)

unknown-3

The most revealing thing about this to me, is that the day to day fluctuations havent really changed over 100+ years – market still behaves the same.

For example, if we randomly reshuffle the order of daily returns of 1910 and compare it to 2015 reshuffled daily returns, its impossible to say which one is which. The nature and behaviour of day to day fluctuations is still the same.

unknown-4

2 thoughts on “100 Years of dow jones returns

  1. Awesome Job, thanks for sharing! Eye opening that day to day movements are still the same as 100 years ago! Slap in the face to all of those saying that markets changed somehow!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s