MischaFisher.com | Quantitative Economicshttp://mischafisher.com/2018-07-03T12:00:00-07:00Refreshing Pelican2018-07-03T12:00:00-07:002018-07-03T12:00:00-07:00Mischa Fishertag:mischafisher.com,2018-07-03:/refreshing-pelican.html<p>Two years after rebuilding this site, Pelican - the python based static site generator I use - was in need of an update. Here's a brief summary of a few meaningful changes to the build:</p>
<p>(and kudos to my good friend and former Google/FB engineer <a href="https://sahandsaba.com/">Sahand Saba</a> for leading the way on all of these changes)</p>
<h3>Updating Math Rendering</h3>
<p>The original version of this site used Mathjax to render LaTeX within Markdown, which worked well for displaying equations, but there was room for speed and bandwidth improvements. Enter <a href="https://katex.org/">Katex</a>, "the fastest math typesetting library for the web". </p>
<p>It has a simple installation, no dependencies, and it's own CDN. The trick is removing the need to have in-browser javascript doing the rendering. </p>
<p>The github project file is here: <a href="https://github.com/Khan/KaTeX">https://github.com/Khan/KaTeX</a></p>
<p>Installation is as simple as a pelican_katex.py file referenced in the pelicanconf.py file:</p>
<div class="highlight"><pre><span></span>```` PLUGINS = ['pelican_katex'] ````
</pre></div>
<p>And the repo here: <a href="https://github.com/sahands/pelican_katex">Sahand Saba Katex Pelican repo</a></p>
<h3>Configuring HTTPS for Amazon S3</h3>
<p>Looking up at the URL bar, you'll notice everything is now being served in <em>https</em>. This is thanks to a link between Amazon Cloudfront and the S3 bucket that hosts this site. </p>
<p>Quick links to get started are here: </p>
<ul>
<li><a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html">Getting Started with Cloud Front</a></li>
<li><a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-https-cloudfront-to-custom-origin.html">Requiring HTTPS Communication Between Cloudfront and Origin</a></li>
</ul>
<h3>Versioning Using Bitbucket</h3>
<p>Version control is a critical but often overlooked step not only embarrassingly frequently on the enterprise level, but also for many personal projects. Rather than running iterations of the site through copied folders hosted on my personal Dropbox account, I finally got around to version control using bitbucket. Git is of course more standard, but bitbucket provides free private repositories, which is a necessity for the backend python code being used to generate the site. </p>
<ul>
<li><a href="https://bitbucket.org/">bitbucket</a></li>
</ul>
<h3>Favicon</h3>
<p>Finally, there's now a favicon of the camera logo of the old soviet era camera that constitutes my personal site logo. There's a simple generator package available on github that can generate a favicon stack to work on just about any browser or platform.</p>
<ul>
<li><a href="https://github.com/RealFaviconGenerator">Version 0.16 of favicongenerator</a></li>
</ul><p>Two years after rebuilding this site, Pelican - the python based static site generator I use - was in need of an update. Here's a brief summary of a few meaningful changes to the build:</p>
<p>(and kudos to my good friend and former Google/FB engineer <a href="https://sahandsaba.com/">Sahand Saba</a> for leading the way on all of these changes)</p>
<h3>Updating Math Rendering</h3>
<p>The original version of this site used Mathjax to render LaTeX within Markdown, which worked well for displaying equations, but there was room for speed and bandwidth improvements. Enter <a href="https://katex.org/">Katex</a>, "the fastest math typesetting library for the web". </p>
<p>It has a simple installation, no dependencies, and it's own CDN. The trick is removing the need to have in-browser javascript doing the rendering. </p>
<p>The github project file is here: <a href="https://github.com/Khan/KaTeX">https://github.com/Khan/KaTeX</a></p>
<p>Installation is as simple as a pelican_katex.py file referenced in the pelicanconf.py file:</p>
<div class="highlight"><pre><span></span>```` PLUGINS = ['pelican_katex'] ````
</pre></div>
<p>And the repo here: <a href="https://github.com/sahands/pelican_katex">Sahand Saba Katex Pelican repo</a></p>
<h3>Configuring HTTPS for Amazon S3</h3>
<p>Looking up at the URL bar, you'll notice everything is now being served in <em>https</em>. This is thanks to a link between Amazon Cloudfront and the S3 bucket that hosts this site. </p>
<p>Quick links to get started are here: </p>
<ul>
<li><a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html">Getting Started with Cloud Front</a></li>
<li><a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-https-cloudfront-to-custom-origin.html">Requiring HTTPS Communication Between Cloudfront and Origin</a></li>
</ul>
<h3>Versioning Using Bitbucket</h3>
<p>Version control is a critical but often overlooked step not only embarrassingly frequently on the enterprise level, but also for many personal projects. Rather than running iterations of the site through copied folders hosted on my personal Dropbox account, I finally got around to version control using bitbucket. Git is of course more standard, but bitbucket provides free private repositories, which is a necessity for the backend python code being used to generate the site. </p>
<ul>
<li><a href="https://bitbucket.org/">bitbucket</a></li>
</ul>
<h3>Favicon</h3>
<p>Finally, there's now a favicon of the camera logo of the old soviet era camera that constitutes my personal site logo. There's a simple generator package available on github that can generate a favicon stack to work on just about any browser or platform.</p>
<ul>
<li><a href="https://github.com/RealFaviconGenerator">Version 0.16 of favicongenerator</a></li>
</ul>Distributional Parameters2017-03-01T12:00:00-08:002017-03-01T12:00:00-08:00Mischa Fishertag:mischafisher.com,2017-03-01:/distributional-parameters.html<p>Part of my thesis involved modeling survival times against parametric distributions, such as the Weibull, log-logistic, and exponential distributions.</p>
<p>One of the fun aspects of distribution theory is seeing how different parameter specifications can make some distributions special forms of other distributions. For today's quick chart, a lead in to the subject by looking at how a couple of commonly used survival analysis parameters resemble one another (with the parameter specifications highlighted in the R code below).</p>
<div class="highlight"><pre><span></span>x_lower <span class="o"><-</span> <span class="m">0</span>
x_upper <span class="o"><-</span> <span class="m">10</span>
max_height2 <span class="o"><-</span> <span class="kp">max</span><span class="p">(</span>dexp<span class="p">(</span>x_lower<span class="o">:</span>x_upper<span class="p">,</span> rate <span class="o">=</span> <span class="m">1</span><span class="p">,</span> log <span class="o">=</span> <span class="kc">FALSE</span><span class="p">),</span>
dweibull<span class="p">(</span>x_lower<span class="o">:</span>x_upper<span class="p">,</span> shape <span class="o">=</span> <span class="m">1</span><span class="p">,</span> log <span class="o">=</span> <span class="kc">FALSE</span><span class="p">),</span>
dlogis<span class="p">(</span>x_lower<span class="o">:</span>x_upper<span class="p">,</span> scale <span class="o">=</span> <span class="m">1</span><span class="p">,</span> log <span class="o">=</span> <span class="kc">FALSE</span><span class="p">))</span>
ggplot<span class="p">(</span><span class="kt">data.frame</span><span class="p">(</span>x <span class="o">=</span> <span class="kt">c</span><span class="p">(</span>x_lower<span class="p">,</span> x_upper<span class="p">)),</span> aes<span class="p">(</span>x <span class="o">=</span> x<span class="p">))</span> <span class="o">+</span> xlim<span class="p">(</span>x_lower<span class="p">,</span> x_upper<span class="p">)</span> <span class="o">+</span>
ylim<span class="p">(</span><span class="m">0</span><span class="p">,</span> max_height2<span class="p">)</span> <span class="o">+</span>
stat_function<span class="p">(</span>fun <span class="o">=</span> dexp<span class="p">,</span> args <span class="o">=</span> <span class="kt">list</span><span class="p">(</span>rate <span class="o">=</span> <span class="m">2</span><span class="p">),</span> aes<span class="p">(</span>colour <span class="o">=</span> <span class="s">"Exponential"</span><span class="p">))</span> <span class="o">+</span>
stat_function<span class="p">(</span>fun <span class="o">=</span> dweibull<span class="p">,</span> args <span class="o">=</span> <span class="kt">list</span><span class="p">(</span>shape <span class="o">=</span> <span class="m">2</span><span class="p">),</span> aes<span class="p">(</span>colour <span class="o">=</span> <span class="s">"Weibull"</span><span class="p">))</span> <span class="o">+</span>
stat_function<span class="p">(</span>fun <span class="o">=</span> dlogis<span class="p">,</span> args <span class="o">=</span> <span class="kt">list</span><span class="p">(</span>scale <span class="o">=</span> <span class="m">2</span><span class="p">),</span> aes<span class="p">(</span>colour <span class="o">=</span> <span class="s">"Logistic"</span><span class="p">))</span> <span class="o">+</span>
scale_color_manual<span class="p">(</span><span class="s">"Distribution"</span><span class="p">,</span> values <span class="o">=</span> <span class="kt">c</span><span class="p">(</span><span class="s">"blue"</span><span class="p">,</span> <span class="s">"green"</span><span class="p">,</span> <span class="s">"red"</span><span class="p">))</span> <span class="o">+</span>
labs<span class="p">(</span>x <span class="o">=</span> <span class="s">"\n x"</span><span class="p">,</span> y <span class="o">=</span> <span class="s">"f(x) \n"</span><span class="p">,</span>
title <span class="o">=</span> <span class="s">"Common Survival Analysis Distribution Density Plots \n"</span><span class="p">)</span> <span class="o">+</span>
theme<span class="p">(</span>plot.title <span class="o">=</span> element_text<span class="p">(</span>hjust <span class="o">=</span> <span class="m">0.5</span><span class="p">),</span>
axis.title.x <span class="o">=</span> element_text<span class="p">(</span>face<span class="o">=</span><span class="s">"bold"</span><span class="p">,</span> colour<span class="o">=</span><span class="s">"blue"</span><span class="p">,</span> size <span class="o">=</span> <span class="m">12</span><span class="p">),</span>
axis.title.y <span class="o">=</span> element_text<span class="p">(</span>face<span class="o">=</span><span class="s">"bold"</span><span class="p">,</span> colour<span class="o">=</span><span class="s">"blue"</span><span class="p">,</span> size <span class="o">=</span> <span class="m">12</span><span class="p">),</span>
legend.title <span class="o">=</span> element_text<span class="p">(</span>face<span class="o">=</span><span class="s">"bold"</span><span class="p">,</span> size <span class="o">=</span> <span class="m">10</span><span class="p">),</span>
legend.position <span class="o">=</span> <span class="s">"top"</span><span class="p">)</span> <span class="o">+</span> theme_economist<span class="p">()</span>
</pre></div>
<p><img class="img-responsive img-center" src="/images/posts/2017-03-01-Distro.png" width="80%"/></p><p>Part of my thesis involved modeling survival times against parametric distributions, such as the Weibull, log-logistic, and exponential distributions.</p>
<p>One of the fun aspects of distribution theory is seeing how different parameter specifications can make some distributions special forms of other distributions. For today's quick chart, a lead in to the subject by looking at how a couple of commonly used survival analysis parameters resemble one another (with the parameter specifications highlighted in the R code below).</p>
<div class="highlight"><pre><span></span>x_lower <span class="o"><-</span> <span class="m">0</span>
x_upper <span class="o"><-</span> <span class="m">10</span>
max_height2 <span class="o"><-</span> <span class="kp">max</span><span class="p">(</span>dexp<span class="p">(</span>x_lower<span class="o">:</span>x_upper<span class="p">,</span> rate <span class="o">=</span> <span class="m">1</span><span class="p">,</span> log <span class="o">=</span> <span class="kc">FALSE</span><span class="p">),</span>
dweibull<span class="p">(</span>x_lower<span class="o">:</span>x_upper<span class="p">,</span> shape <span class="o">=</span> <span class="m">1</span><span class="p">,</span> log <span class="o">=</span> <span class="kc">FALSE</span><span class="p">),</span>
dlogis<span class="p">(</span>x_lower<span class="o">:</span>x_upper<span class="p">,</span> scale <span class="o">=</span> <span class="m">1</span><span class="p">,</span> log <span class="o">=</span> <span class="kc">FALSE</span><span class="p">))</span>
ggplot<span class="p">(</span><span class="kt">data.frame</span><span class="p">(</span>x <span class="o">=</span> <span class="kt">c</span><span class="p">(</span>x_lower<span class="p">,</span> x_upper<span class="p">)),</span> aes<span class="p">(</span>x <span class="o">=</span> x<span class="p">))</span> <span class="o">+</span> xlim<span class="p">(</span>x_lower<span class="p">,</span> x_upper<span class="p">)</span> <span class="o">+</span>
ylim<span class="p">(</span><span class="m">0</span><span class="p">,</span> max_height2<span class="p">)</span> <span class="o">+</span>
stat_function<span class="p">(</span>fun <span class="o">=</span> dexp<span class="p">,</span> args <span class="o">=</span> <span class="kt">list</span><span class="p">(</span>rate <span class="o">=</span> <span class="m">2</span><span class="p">),</span> aes<span class="p">(</span>colour <span class="o">=</span> <span class="s">"Exponential"</span><span class="p">))</span> <span class="o">+</span>
stat_function<span class="p">(</span>fun <span class="o">=</span> dweibull<span class="p">,</span> args <span class="o">=</span> <span class="kt">list</span><span class="p">(</span>shape <span class="o">=</span> <span class="m">2</span><span class="p">),</span> aes<span class="p">(</span>colour <span class="o">=</span> <span class="s">"Weibull"</span><span class="p">))</span> <span class="o">+</span>
stat_function<span class="p">(</span>fun <span class="o">=</span> dlogis<span class="p">,</span> args <span class="o">=</span> <span class="kt">list</span><span class="p">(</span>scale <span class="o">=</span> <span class="m">2</span><span class="p">),</span> aes<span class="p">(</span>colour <span class="o">=</span> <span class="s">"Logistic"</span><span class="p">))</span> <span class="o">+</span>
scale_color_manual<span class="p">(</span><span class="s">"Distribution"</span><span class="p">,</span> values <span class="o">=</span> <span class="kt">c</span><span class="p">(</span><span class="s">"blue"</span><span class="p">,</span> <span class="s">"green"</span><span class="p">,</span> <span class="s">"red"</span><span class="p">))</span> <span class="o">+</span>
labs<span class="p">(</span>x <span class="o">=</span> <span class="s">"\n x"</span><span class="p">,</span> y <span class="o">=</span> <span class="s">"f(x) \n"</span><span class="p">,</span>
title <span class="o">=</span> <span class="s">"Common Survival Analysis Distribution Density Plots \n"</span><span class="p">)</span> <span class="o">+</span>
theme<span class="p">(</span>plot.title <span class="o">=</span> element_text<span class="p">(</span>hjust <span class="o">=</span> <span class="m">0.5</span><span class="p">),</span>
axis.title.x <span class="o">=</span> element_text<span class="p">(</span>face<span class="o">=</span><span class="s">"bold"</span><span class="p">,</span> colour<span class="o">=</span><span class="s">"blue"</span><span class="p">,</span> size <span class="o">=</span> <span class="m">12</span><span class="p">),</span>
axis.title.y <span class="o">=</span> element_text<span class="p">(</span>face<span class="o">=</span><span class="s">"bold"</span><span class="p">,</span> colour<span class="o">=</span><span class="s">"blue"</span><span class="p">,</span> size <span class="o">=</span> <span class="m">12</span><span class="p">),</span>
legend.title <span class="o">=</span> element_text<span class="p">(</span>face<span class="o">=</span><span class="s">"bold"</span><span class="p">,</span> size <span class="o">=</span> <span class="m">10</span><span class="p">),</span>
legend.position <span class="o">=</span> <span class="s">"top"</span><span class="p">)</span> <span class="o">+</span> theme_economist<span class="p">()</span>
</pre></div>
<p><img class="img-responsive img-center" src="/images/posts/2017-03-01-Distro.png" width="80%"/></p>State Budgets and Populations (a.k.a Why Illinois is in the shape it's in)2016-02-03T00:00:00-08:002016-02-03T00:00:00-08:00Mischa Fishertag:mischafisher.com,2016-02-03:/state-budgets-and-populations-aka-why-illinois-is-in-the-shape-its-in.html<p>Given Illinois' current budget impasse, I thought it would be interesting to do a five minute analysis looking at how the size of each state's budget varies in proportion to its population. This is obviously a very shallow examination, and one could spends weeks digging through budget numbers, federal transfers, rural-urban splits, poverty and education levels, industry compositions, unfunded pension liabilities, worker's compensation costs, etc. etc. Still, summary statistics exist for a reason; and 5 minute analyses can be useful exercises.</p>
<p>Pulling data from Wikipedia's maintained list of US State budgets <a href="https://en.wikipedia.org/wiki/List_of_U.S._state_budgets">(here)</a>, and from the U.S. Census' estimate of 2015 state populations <a href="https://www.census.gov/popest/data/datasets.html">(here)</a></p>
<h3>All States</h3>
<p>First, a look at all U.S. States:</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-02-Budgets_01.png" width="80%"/></p>
<p>I knew Illinois' budget has historically been unsustainable, but, I was surprised at just how much of an outlier the state is. Mercatus now has Illinois rated dead last in the country in terms of fiscal solvency <a href="http://mercatus.org/statefiscalrankings">(link here)</a>, and with this chart, one can see why!</p>
<h3>Just the Large States</h3>
<p>Comparing large states to large states, here are all the states with populations above 6 million people.</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-02-Budgets_02.png" width="80%"/></p>
<h3>Just the Small States</h3>
<p>And in the same spirit, the small states:</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-02-Budgets_03.png" width="80%"/></p>
<p>Here it's worth noting that Illinois does not have the largest per capita budget; that honor goes to Alaska. Illinois simply is the largest deviator from the overall trend line in absolute dollar terms. That being said, since State budgets include federal transfer dollars for federal programs (infrastructure, heating assistance, etc. etc.), it's not hard to see why Alaska, which has very few people in it but a lot of federally supported infrastructure, is the highest per capita budget state.</p>
<h3>UPDATE:</h3>
<p>What was intended as a five minute "hey that's interesting!" analysis ended up exploding on the internet. With over 100K page views in 12 hours, the response was certainly unexpected. On that note, a few people on Reddit mentioned they'd be interested in seeing the log of the data. So here it is:</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-02-Budgets_04.png" width="80%"/></p>
<p>More importantly, I think it's important for people to remember that Wikipedia data is not always particularly accurate, nor is it necessarily an apples to apples comparison. Some of the data listed on the page is for single year periods, other bits of data are for multi-year periods. The script I used to plot takes those things into consideration, but, it may have errors given the inconsistency in Wikipedia's data. Time allowing I'll source better data and replot at some point in the future. </p>
<h3>UPDATE 2:</h3>
<p><a href="http://www.nasbo.org/sites/default/files/State%20Expenditure%20Report%20%28Fiscal%202013-2015%29S.pdf">NASBO</a> has a dataset on state spending, I took some time to manually write down the state and spending columns as vectors in R (so there may be transcription errors) and the log of the data produces a result <strong>very</strong> different than Wikipedia's data:</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-02-Budgets_05.png" width="80%"/></p>
<p>So the story of Illinois' terrible fiscal condition could very well be more complicated than can be captured in a single graph. As in most things in life; the issue is complex and nuanced.</p>
<p>I will add though, the NASBO graph doesn't …</p><p>Given Illinois' current budget impasse, I thought it would be interesting to do a five minute analysis looking at how the size of each state's budget varies in proportion to its population. This is obviously a very shallow examination, and one could spends weeks digging through budget numbers, federal transfers, rural-urban splits, poverty and education levels, industry compositions, unfunded pension liabilities, worker's compensation costs, etc. etc. Still, summary statistics exist for a reason; and 5 minute analyses can be useful exercises.</p>
<p>Pulling data from Wikipedia's maintained list of US State budgets <a href="https://en.wikipedia.org/wiki/List_of_U.S._state_budgets">(here)</a>, and from the U.S. Census' estimate of 2015 state populations <a href="https://www.census.gov/popest/data/datasets.html">(here)</a></p>
<h3>All States</h3>
<p>First, a look at all U.S. States:</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-02-Budgets_01.png" width="80%"/></p>
<p>I knew Illinois' budget has historically been unsustainable, but, I was surprised at just how much of an outlier the state is. Mercatus now has Illinois rated dead last in the country in terms of fiscal solvency <a href="http://mercatus.org/statefiscalrankings">(link here)</a>, and with this chart, one can see why!</p>
<h3>Just the Large States</h3>
<p>Comparing large states to large states, here are all the states with populations above 6 million people.</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-02-Budgets_02.png" width="80%"/></p>
<h3>Just the Small States</h3>
<p>And in the same spirit, the small states:</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-02-Budgets_03.png" width="80%"/></p>
<p>Here it's worth noting that Illinois does not have the largest per capita budget; that honor goes to Alaska. Illinois simply is the largest deviator from the overall trend line in absolute dollar terms. That being said, since State budgets include federal transfer dollars for federal programs (infrastructure, heating assistance, etc. etc.), it's not hard to see why Alaska, which has very few people in it but a lot of federally supported infrastructure, is the highest per capita budget state.</p>
<h3>UPDATE:</h3>
<p>What was intended as a five minute "hey that's interesting!" analysis ended up exploding on the internet. With over 100K page views in 12 hours, the response was certainly unexpected. On that note, a few people on Reddit mentioned they'd be interested in seeing the log of the data. So here it is:</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-02-Budgets_04.png" width="80%"/></p>
<p>More importantly, I think it's important for people to remember that Wikipedia data is not always particularly accurate, nor is it necessarily an apples to apples comparison. Some of the data listed on the page is for single year periods, other bits of data are for multi-year periods. The script I used to plot takes those things into consideration, but, it may have errors given the inconsistency in Wikipedia's data. Time allowing I'll source better data and replot at some point in the future. </p>
<h3>UPDATE 2:</h3>
<p><a href="http://www.nasbo.org/sites/default/files/State%20Expenditure%20Report%20%28Fiscal%202013-2015%29S.pdf">NASBO</a> has a dataset on state spending, I took some time to manually write down the state and spending columns as vectors in R (so there may be transcription errors) and the log of the data produces a result <strong>very</strong> different than Wikipedia's data:</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-02-Budgets_05.png" width="80%"/></p>
<p>So the story of Illinois' terrible fiscal condition could very well be more complicated than can be captured in a single graph. As in most things in life; the issue is complex and nuanced.</p>
<p>I will add though, the NASBO graph doesn't capture unfunded pension liabilities; so, it also cannot necessarily be taken at anything more than face value.</p>Renting vs. Owning in Chicago2016-01-23T00:00:00-08:002016-01-23T00:00:00-08:00Mischa Fishertag:mischafisher.com,2016-01-23:/renting-vs-owning-in-chicago.html<p>Moving to Chicago this past weekend prompted the age old question, should one rent or should one buy?</p>
<p>Independent of the qualitative and lifestyle differences between the two choices, I was curious how - strictly speaking - the finances between the two options worked out. Using the Case Schiller Price Index for Condos in the Chicago metro, and the historical return rate on real estate prices, worked into a short R function, produced the below results.</p>
<p>Worth noting, the calculation <strong>included</strong>:</p>
<ul>
<li>The opportunity cost of capital</li>
<li>Historical returns on real estate prices</li>
<li>For Owning: Mortgage, property taxes, closing costs, Home Owners Association fees, cost of ownership</li>
<li>For Renting: Rent, Utilities</li>
</ul>
<p>However, it <strong>did not include</strong>:</p>
<ul>
<li>Mortgage interest Deduction</li>
<li>Non linear mortgage amortization. (This one is linear, rather than being skewed towards the latter years as would actually happen on a traditional amortization schedule)</li>
<li>This assumes a 20% downpayment, rather than some other available options, such as the FHA loans that allow for as low as 3.5%. </li>
</ul>
<p>The Result:</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-01-Housing_01.png" width="80%"/></p>
<p>Given these basic assumptions on relative costs, it seems to confirm the somewhat common folk wisdom that it takes about 4-5 years for the initial hit of the closing costs to be paid off by saved expenses and amortization of the mortgage loan. </p><p>Moving to Chicago this past weekend prompted the age old question, should one rent or should one buy?</p>
<p>Independent of the qualitative and lifestyle differences between the two choices, I was curious how - strictly speaking - the finances between the two options worked out. Using the Case Schiller Price Index for Condos in the Chicago metro, and the historical return rate on real estate prices, worked into a short R function, produced the below results.</p>
<p>Worth noting, the calculation <strong>included</strong>:</p>
<ul>
<li>The opportunity cost of capital</li>
<li>Historical returns on real estate prices</li>
<li>For Owning: Mortgage, property taxes, closing costs, Home Owners Association fees, cost of ownership</li>
<li>For Renting: Rent, Utilities</li>
</ul>
<p>However, it <strong>did not include</strong>:</p>
<ul>
<li>Mortgage interest Deduction</li>
<li>Non linear mortgage amortization. (This one is linear, rather than being skewed towards the latter years as would actually happen on a traditional amortization schedule)</li>
<li>This assumes a 20% downpayment, rather than some other available options, such as the FHA loans that allow for as low as 3.5%. </li>
</ul>
<p>The Result:</p>
<p><img class="img-responsive img-center" src="/images/posts/2016-01-Housing_01.png" width="80%"/></p>
<p>Given these basic assumptions on relative costs, it seems to confirm the somewhat common folk wisdom that it takes about 4-5 years for the initial hit of the closing costs to be paid off by saved expenses and amortization of the mortgage loan. </p>How Much Does Rural Living Predict Broadband Speed?2015-12-08T12:00:00-08:002015-12-08T12:00:00-08:00Mischa Fishertag:mischafisher.com,2015-12-08:/how-much-does-rural-living-predict-broadband-speed.html<p>I was recently brushing up on the current status of the telecommunications industry in the United States, and I became curious about how much a state's rural population predicted its overall average internet connection speed levels.</p>
<p>Pulling data on average speeds from <a href="http://www.broadviewnet.com/blog/2014/08/internet-speeds-by-state-map/">(here)</a>, which sources Akamai's <em>State of the Internet</em> report, and data on the rural/urban population by state from Iowa State University's site <a href="http://www.icip.iastate.edu/tables/population/urban-pct-states">(here)</a>, reveals the below plot:</p>
<p><img class="img-responsive img-center" src="/images/posts/2015-12-Broadband_01.png" width="80%"/></p>
<p>While noticeable, the effect here is pretty minor overall. It would take a shift of about 20% of the state's population into an urban environment to shift average speeds up by a single Mbps.</p>
<h3>How About Globally?</h3>
<p>Curious about how well this predicts things generally, I did the same thing looking at global data. Wikipedia has a concise list of countries by internet connection speeds <a href="https://en.wikipedia.org/wiki/List_of_countries_by_Internet_connection_speeds">(here)</a>, and the World Bank maintains a time series list of urban/rural population <a href="http://data.worldbank.org/indicator/SP.URB.TOTL.IN.ZS">(here)</a>. </p>
<p>Pulling those two datasets together, reveals the following:</p>
<p><img class="img-responsive img-center" src="/images/posts/2015-12-Broadband_02.png" width="80%"/></p>
<p>Visually this looks similar, although the linear regression slope is a little steeper. It would take a shift of about 10% of an average country's population to shift the speed up by a Mbps.</p>
<h3>Conclusion</h3>
<p>At the end of the day, <em>this exercise is likely too simple to shine much light on the phenomenon.</em> Economies of scale in terms of population density would, all else equal, suggest that as density goes up, so would speed. But the 'average' speed by state hides the interesting variation that goes on within the county and city level that I imagine would more apparently show the swings based on not just population density, but, also regulatory factors such as the ease of installation and market entry, access to public conduit and utility maps, etc. etc. Looking at the data more locally would probably be more meaningful than trying to examine aggregate state or country data, because there is likely some regional smoothing that goes on in the 'average speed' metric that underestimates the effect.</p><p>I was recently brushing up on the current status of the telecommunications industry in the United States, and I became curious about how much a state's rural population predicted its overall average internet connection speed levels.</p>
<p>Pulling data on average speeds from <a href="http://www.broadviewnet.com/blog/2014/08/internet-speeds-by-state-map/">(here)</a>, which sources Akamai's <em>State of the Internet</em> report, and data on the rural/urban population by state from Iowa State University's site <a href="http://www.icip.iastate.edu/tables/population/urban-pct-states">(here)</a>, reveals the below plot:</p>
<p><img class="img-responsive img-center" src="/images/posts/2015-12-Broadband_01.png" width="80%"/></p>
<p>While noticeable, the effect here is pretty minor overall. It would take a shift of about 20% of the state's population into an urban environment to shift average speeds up by a single Mbps.</p>
<h3>How About Globally?</h3>
<p>Curious about how well this predicts things generally, I did the same thing looking at global data. Wikipedia has a concise list of countries by internet connection speeds <a href="https://en.wikipedia.org/wiki/List_of_countries_by_Internet_connection_speeds">(here)</a>, and the World Bank maintains a time series list of urban/rural population <a href="http://data.worldbank.org/indicator/SP.URB.TOTL.IN.ZS">(here)</a>. </p>
<p>Pulling those two datasets together, reveals the following:</p>
<p><img class="img-responsive img-center" src="/images/posts/2015-12-Broadband_02.png" width="80%"/></p>
<p>Visually this looks similar, although the linear regression slope is a little steeper. It would take a shift of about 10% of an average country's population to shift the speed up by a Mbps.</p>
<h3>Conclusion</h3>
<p>At the end of the day, <em>this exercise is likely too simple to shine much light on the phenomenon.</em> Economies of scale in terms of population density would, all else equal, suggest that as density goes up, so would speed. But the 'average' speed by state hides the interesting variation that goes on within the county and city level that I imagine would more apparently show the swings based on not just population density, but, also regulatory factors such as the ease of installation and market entry, access to public conduit and utility maps, etc. etc. Looking at the data more locally would probably be more meaningful than trying to examine aggregate state or country data, because there is likely some regional smoothing that goes on in the 'average speed' metric that underestimates the effect.</p>A Brief Exercise Illustrating the Central Limit Theorem2015-11-26T10:20:00-08:002015-11-26T10:20:00-08:00Mischa Fishertag:mischafisher.com,2015-11-26:/a-brief-exercise-illustrating-the-central-limit-theorem.html<p>Succinctly, the Central Limit Theorem can be expressed as:</p>
<blockquote>
<p><em>In probability theory, the central limit theorem (CLT) states that, given certain conditions, the arithmetic mean of a sufficiently large number of iterates of independent random variables, each with a well-defined expected value and well-defined variance, will be approximately normally distributed, regardless of the underlying distribution.</em></p>
</blockquote>
<p>If you're an econometrics student, it's likely the first time you're exposed to the CLT is when discussing the OLS estimator in the context of testing hypothesis about the true parameters in order to form confidence intervals: when relying on asymptotics, the normality of the error distributions is not required because as long as the normal 5 Gauss-Markov assumptions are satisfied, the distribution of the OLS estimator will converge to a normal distribution as <em>n</em> goes to infinity. </p>
<p>The CLT is pretty neat and is given short shrift in the context of econometrics, so, here's a brief experiment one can perform in R to illustrate what happens as the theorem comes into effect.</p>
<p>Starting with the Weibull distribution:</p>
<div class="highlight"><pre><span></span>plot(sort(rweibull(10000, shape=1)), main="The Weibull Distribution")
</pre></div>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-CLT_Weibull.jpg" width="80%"/></p>
<p>We then sample the means of the distribution in increasing replications, then draw histograms from the distributions.</p>
<div class="highlight"><pre><span></span>hist(colMeans(replicate(30,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 30 Replications")
hist(colMeans(replicate(300,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 300 Replications")
hist(colMeans(replicate(3000,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 3,000 Replications")
hist(colMeans(replicate(30000,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 30,000 Replications")
hist(colMeans(replicate(300000,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 300,000 Replications")
hist(colMeans(replicate(3000000,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 3,000,000 Replications")
</pre></div>
<p>Revealing, a very wonderful .gif :</p>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-CLT.gif" width="80%"/></p>
<p>As the replication size increases, the histogram begins to resemble a normal distribution. Neat!</p>
<h3>UPDATE:</h3>
<p>A friend writes:</p>
<blockquote>
<p><em>You should emphasize more the process that you are taking the MEAN of sub samples of your 'population' which is Weibull distributed. Then, by creating a vector of these means, one is able to show that these "means" converge to a normal distribution as N approaches infinity. As it is, to the 'less' experienced reader perhaps will fail to realize that you take the means of sub samples of that pop'n, and then it is the means which become normally distributed.</em></p>
</blockquote>
<p>Good point; thanks Keith!</p><p>Succinctly, the Central Limit Theorem can be expressed as:</p>
<blockquote>
<p><em>In probability theory, the central limit theorem (CLT) states that, given certain conditions, the arithmetic mean of a sufficiently large number of iterates of independent random variables, each with a well-defined expected value and well-defined variance, will be approximately normally distributed, regardless of the underlying distribution.</em></p>
</blockquote>
<p>If you're an econometrics student, it's likely the first time you're exposed to the CLT is when discussing the OLS estimator in the context of testing hypothesis about the true parameters in order to form confidence intervals: when relying on asymptotics, the normality of the error distributions is not required because as long as the normal 5 Gauss-Markov assumptions are satisfied, the distribution of the OLS estimator will converge to a normal distribution as <em>n</em> goes to infinity. </p>
<p>The CLT is pretty neat and is given short shrift in the context of econometrics, so, here's a brief experiment one can perform in R to illustrate what happens as the theorem comes into effect.</p>
<p>Starting with the Weibull distribution:</p>
<div class="highlight"><pre><span></span>plot(sort(rweibull(10000, shape=1)), main="The Weibull Distribution")
</pre></div>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-CLT_Weibull.jpg" width="80%"/></p>
<p>We then sample the means of the distribution in increasing replications, then draw histograms from the distributions.</p>
<div class="highlight"><pre><span></span>hist(colMeans(replicate(30,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 30 Replications")
hist(colMeans(replicate(300,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 300 Replications")
hist(colMeans(replicate(3000,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 3,000 Replications")
hist(colMeans(replicate(30000,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 30,000 Replications")
hist(colMeans(replicate(300000,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 300,000 Replications")
hist(colMeans(replicate(3000000,rweibull(100,shape=1))),breaks="Scott", xlab="Sample Means", main="Histogram for 3,000,000 Replications")
</pre></div>
<p>Revealing, a very wonderful .gif :</p>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-CLT.gif" width="80%"/></p>
<p>As the replication size increases, the histogram begins to resemble a normal distribution. Neat!</p>
<h3>UPDATE:</h3>
<p>A friend writes:</p>
<blockquote>
<p><em>You should emphasize more the process that you are taking the MEAN of sub samples of your 'population' which is Weibull distributed. Then, by creating a vector of these means, one is able to show that these "means" converge to a normal distribution as N approaches infinity. As it is, to the 'less' experienced reader perhaps will fail to realize that you take the means of sub samples of that pop'n, and then it is the means which become normally distributed.</em></p>
</blockquote>
<p>Good point; thanks Keith!</p>Donald Trump's Campaign Contributions2015-11-04T12:00:00-08:002015-11-04T12:00:00-08:00Mischa Fishertag:mischafisher.com,2015-11-04:/donald-trumps-campaign-contributions.html<p>Donald Trump recently passed the 100 day mark as the Republican front runner for the Presidential nomination, which means I was wrong (as many friends have reminded me) about him being a short term fad.</p>
<p>To scratch a curious itch, I decided to pull The Donald's campaign contributions from the <a href="http://www.fec.gov">FEC's personal disclosure database</a>; that is, where has the Donald donated his money over the years. Since I was wrong about how long he'd remain at the front of the Republican primary field, I thought I'd see if I was also wrong in my suspicion that his donations were skewed toward whichever party was more popular at any given moment over the last 20 years. To the data!</p>
<h3>Sum of all Donations, by Party</h3>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-Trump_01.png" width="80%"/></p>
<p>This is the first breakdown of total contributions over the last 20 or so years, by donations to Republicans, Democrats, and himself. Unfortunately, it has no context of time since I suspected his donation volume had increased substantially in the most recent year.</p>
<h3>Sum of All Donations by Party, per year</h3>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-Trump_02.png" width="80%"/></p>
<p>This breaks down the total value of all his donations by party, per year. While it shows the recent uptick in spending, it suffers from a form of political inflation too. Donation values, for normative and legal reasons, have increased substantially across the board in the few recent cycles. So, to correct for that, I also looked at the total number of donations made...</p>
<h3>Total Number of Donations by Party, per year</h3>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-Trump_03.png" width="80%"/></p>
<p>This better captures how far and wide The Donald has spread his political largesse over the past 20 years by looking at the total number of donations made to each party (and to himself). </p>
<h3>Total Number of Donations by Party, per Election Cycle</h3>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-Trump_04.png" width="80%"/></p>
<p>Finally, to remove the cyclical nature of donations, I clumped the years together into election cycles, since, off-year donations tend to be lower than on-year donations. </p>
<p>This to me is the clearest story about his donations. There is clearly a shift in the nature of his partisan giving; and that shift comes pretty close to when President Obama's popularity started to decline. And the spike in donations to Democrats came pretty much at the same time as the big drop in President Bush's popularity. </p>
<p>One could interpret these results in two different ways; first, Donald Trump is an opportunist who simply changed his giving to reflect a change in where he viewed an opportunity to run (specifically, look at the big change between the 2006 election cycle and the current one). Alternatively, one could also think he's a dedicated partisan who stopped his political giving because he grew sick of holding the middle ground. I think that second scenario is unlikely, but, then again I've been wrong before.</p><p>Donald Trump recently passed the 100 day mark as the Republican front runner for the Presidential nomination, which means I was wrong (as many friends have reminded me) about him being a short term fad.</p>
<p>To scratch a curious itch, I decided to pull The Donald's campaign contributions from the <a href="http://www.fec.gov">FEC's personal disclosure database</a>; that is, where has the Donald donated his money over the years. Since I was wrong about how long he'd remain at the front of the Republican primary field, I thought I'd see if I was also wrong in my suspicion that his donations were skewed toward whichever party was more popular at any given moment over the last 20 years. To the data!</p>
<h3>Sum of all Donations, by Party</h3>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-Trump_01.png" width="80%"/></p>
<p>This is the first breakdown of total contributions over the last 20 or so years, by donations to Republicans, Democrats, and himself. Unfortunately, it has no context of time since I suspected his donation volume had increased substantially in the most recent year.</p>
<h3>Sum of All Donations by Party, per year</h3>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-Trump_02.png" width="80%"/></p>
<p>This breaks down the total value of all his donations by party, per year. While it shows the recent uptick in spending, it suffers from a form of political inflation too. Donation values, for normative and legal reasons, have increased substantially across the board in the few recent cycles. So, to correct for that, I also looked at the total number of donations made...</p>
<h3>Total Number of Donations by Party, per year</h3>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-Trump_03.png" width="80%"/></p>
<p>This better captures how far and wide The Donald has spread his political largesse over the past 20 years by looking at the total number of donations made to each party (and to himself). </p>
<h3>Total Number of Donations by Party, per Election Cycle</h3>
<p><img class="img-responsive img-center" src="/images/posts/2015-11-Trump_04.png" width="80%"/></p>
<p>Finally, to remove the cyclical nature of donations, I clumped the years together into election cycles, since, off-year donations tend to be lower than on-year donations. </p>
<p>This to me is the clearest story about his donations. There is clearly a shift in the nature of his partisan giving; and that shift comes pretty close to when President Obama's popularity started to decline. And the spike in donations to Democrats came pretty much at the same time as the big drop in President Bush's popularity. </p>
<p>One could interpret these results in two different ways; first, Donald Trump is an opportunist who simply changed his giving to reflect a change in where he viewed an opportunity to run (specifically, look at the big change between the 2006 election cycle and the current one). Alternatively, one could also think he's a dedicated partisan who stopped his political giving because he grew sick of holding the middle ground. I think that second scenario is unlikely, but, then again I've been wrong before.</p>An Intuitive Explanation of the OLS Estimator for both Traditional and Matrix Algebra2015-10-19T12:00:00-07:002015-10-19T12:00:00-07:00Mischa Fishertag:mischafisher.com,2015-10-19:/an-intuitive-explanation-of-the-ols-estimator-for-both-traditional-and-matrix-algebra.html<p>The Ordinary Least Squares estimator, \( \hat{\beta} \) is the first thing one learns in econometrics. It has two forms, one in standard algebra and one in matrix algebra, but it's important to remember the two are equivalent:</p>
<h3>$$ \hat{\beta} = \frac{\hat{cov}(x,y)}{var(x)} = \mathbf{({X}'X)^{-1}{X}'Y} $$</h3>
<p>I think most students will find it extremely easy to get lost in notation and miss the link to be made with real world data. The following exercise is a helpful way I found to make sure one continues to make the link between traditional 'simple' notation, Matrix Algebra notation, and the underlying data and arithmetic that goes into the ordinary linear regression estimator. </p>
<h3>Deriving the Algebraic Notation for the Simple Bivariate Model</h3>
<p>The familiar simple bivariate model is expressed as an independent observation as a function of an intercept, a regression coefficient, and an error term (respectively):</p>
<p>$$ y_{i} = b_{0} + b_{1}x_{1} + e_{i} $$</p>
<p>Where we wish to minimize the sum of squared errors (SSE):</p>
<p>$$ minimize: SSE = \sum_{i=1}^{N} e_{i}^{2} $$</p>
<p>To do so we isolate the error of the regression to make it a function of the other terms:</p>
<p>$$ e_{i} = y_{i} - b_{0} - b_{1}x_{1} $$</p>
<p>Then substitute: </p>
<p>$$ minimize: \sum_{i=1}^{N} (y_{i} - b_{0} - b_{1}x_{1})^{2} $$</p>
<p>For our purposes, we'll ignore the derivation of the intercept and take it as a given that it is \( \bar{y} - \hat{\beta_{1}}\bar{x} \) and just solve for the \( \hat{\beta} \) slope coefficient. To minimize the errors, we need to take the partial derivative with respect to \( b_{1} \)</p>
<p>$$ \frac{\partial SSE }{\partial b_{1}} = \frac{\partial }{\partial b_{1}} \left [ \sum_{i=1}^{N} (y_{i} - b_{0} - b_{1}x_{1})^{2} \right ] $$</p>
<p>Move the summation operator through since the the derivative of a sum is equal to the sum of the derivatives:</p>
<p>$$ \frac{\partial SSE }{\partial b_{1}} = \sum_{i=1}^{N} \left [ \frac{\partial }{\partial b_{1}} (y_{i} - b_{0} - b_{1}x_{1})^{2} \right ] $$</p>
<p>Take the derivative (using the chain rule), then setting it equal to 0 for the first order condition to find the min/max:</p>
<p>$$ \frac{\partial SSE }{\partial b_{1}} = -2 \sum_{i=1}^{N} x_{i}(y_{i} - b_{0} - b_{1}x_{1}) = 0 $$</p>
<p>Then multiply by \( - \frac{1}{2} \) to simplify:</p>
<p>$$ 0 = \sum_{i=1}^{N} x_{i}(y_{i} - b_{0} - b_{1}x_{1}) $$</p>
<p>Substitute the solution for the intercept, \( b_{0} \), that we took as a given above:</p>
<p>$$ 0 = \sum_{i=1}^{N} x_{i}(y_{i} - (\bar{y} - \hat{\beta_{1}}\bar{x} ) - b_{1}x_{1}) $$</p>
<p>Then rearrange and distribute the summation operator to solve for \( \hat{\beta_{1}} \) :</p>
<p>$$ \hat{\beta_{1}} = \frac{\sum_{i=1}^{N} (y_{i} - \bar{y} )x_{i}}{ \sum_{i=1}^{N} (x_{i} - \bar{x})x_{i} } $$</p>
<p>Which is algebraically equivalent to:</p>
<p>$$ \frac{\hat{cov}(x,y)}{var(x …</p><p>The Ordinary Least Squares estimator, \( \hat{\beta} \) is the first thing one learns in econometrics. It has two forms, one in standard algebra and one in matrix algebra, but it's important to remember the two are equivalent:</p>
<h3>$$ \hat{\beta} = \frac{\hat{cov}(x,y)}{var(x)} = \mathbf{({X}'X)^{-1}{X}'Y} $$</h3>
<p>I think most students will find it extremely easy to get lost in notation and miss the link to be made with real world data. The following exercise is a helpful way I found to make sure one continues to make the link between traditional 'simple' notation, Matrix Algebra notation, and the underlying data and arithmetic that goes into the ordinary linear regression estimator. </p>
<h3>Deriving the Algebraic Notation for the Simple Bivariate Model</h3>
<p>The familiar simple bivariate model is expressed as an independent observation as a function of an intercept, a regression coefficient, and an error term (respectively):</p>
<p>$$ y_{i} = b_{0} + b_{1}x_{1} + e_{i} $$</p>
<p>Where we wish to minimize the sum of squared errors (SSE):</p>
<p>$$ minimize: SSE = \sum_{i=1}^{N} e_{i}^{2} $$</p>
<p>To do so we isolate the error of the regression to make it a function of the other terms:</p>
<p>$$ e_{i} = y_{i} - b_{0} - b_{1}x_{1} $$</p>
<p>Then substitute: </p>
<p>$$ minimize: \sum_{i=1}^{N} (y_{i} - b_{0} - b_{1}x_{1})^{2} $$</p>
<p>For our purposes, we'll ignore the derivation of the intercept and take it as a given that it is \( \bar{y} - \hat{\beta_{1}}\bar{x} \) and just solve for the \( \hat{\beta} \) slope coefficient. To minimize the errors, we need to take the partial derivative with respect to \( b_{1} \)</p>
<p>$$ \frac{\partial SSE }{\partial b_{1}} = \frac{\partial }{\partial b_{1}} \left [ \sum_{i=1}^{N} (y_{i} - b_{0} - b_{1}x_{1})^{2} \right ] $$</p>
<p>Move the summation operator through since the the derivative of a sum is equal to the sum of the derivatives:</p>
<p>$$ \frac{\partial SSE }{\partial b_{1}} = \sum_{i=1}^{N} \left [ \frac{\partial }{\partial b_{1}} (y_{i} - b_{0} - b_{1}x_{1})^{2} \right ] $$</p>
<p>Take the derivative (using the chain rule), then setting it equal to 0 for the first order condition to find the min/max:</p>
<p>$$ \frac{\partial SSE }{\partial b_{1}} = -2 \sum_{i=1}^{N} x_{i}(y_{i} - b_{0} - b_{1}x_{1}) = 0 $$</p>
<p>Then multiply by \( - \frac{1}{2} \) to simplify:</p>
<p>$$ 0 = \sum_{i=1}^{N} x_{i}(y_{i} - b_{0} - b_{1}x_{1}) $$</p>
<p>Substitute the solution for the intercept, \( b_{0} \), that we took as a given above:</p>
<p>$$ 0 = \sum_{i=1}^{N} x_{i}(y_{i} - (\bar{y} - \hat{\beta_{1}}\bar{x} ) - b_{1}x_{1}) $$</p>
<p>Then rearrange and distribute the summation operator to solve for \( \hat{\beta_{1}} \) :</p>
<p>$$ \hat{\beta_{1}} = \frac{\sum_{i=1}^{N} (y_{i} - \bar{y} )x_{i}}{ \sum_{i=1}^{N} (x_{i} - \bar{x})x_{i} } $$</p>
<p>Which is algebraically equivalent to:</p>
<p>$$ \frac{\hat{cov}(x,y)}{var(x)} = \frac{ \frac{1}{n} \sum_{i=1}^{n} (x_{i} - \bar{x})(y_{i} - \bar{y} )}{ \frac{1}{n} \sum_{i=1}^{n} (x_{i} - \bar{x})^{2} } = \hat{\beta_{1}}$$</p>
<h3>Deriving the Matrix Algebra Notation</h3>
<p>Despite typically not being taught until the senior undergraduate, or graduate, level: the derivation for the matrix notation is actually a little more straightforward (as long as one remembers the rules for matrix algebra (which I typically do not)). </p>
<p>First, visualize the linear model again, but this time in matrix notation where <strong>Y</strong> and <strong>e</strong> are vectors of the observations and <strong>X</strong> is the matrix of the independent variables and their observations:</p>
<p>$$ \mathbf{Y = XB + e} $$</p>
<p>Just as before, we want to minimize the sum of squared errors</p>
<p>$$ minimize: SSE = \mathbf{e'e} $$</p>
<p>Rearranging and substituting yields: </p>
<p>$$ SSE = \mathbf{(Y - XB)'(Y - XB)} $$</p>
<p>Push the transpose operator through:</p>
<p>$$ SSE = \mathbf{(Y' - B'X')(Y - XB)} $$</p>
<p>Multiply the whole equation out:</p>
<p>$$ SSE = \mathbf{Y'Y - Y'XB - B'X'Y + B'X'XB} $$</p>
<p>Simplify the two equivalent terms in the middle:</p>
<p>$$ SSE = \mathbf{Y'Y - 2Y'XB + B'X'XB} $$</p>
<p>Then again as before we'll take the partial derivative for the first order condition:</p>
<p>$$ \frac{\partial SSE }{\mathbf{\partial B}} = \frac{\partial }{\mathbf{\partial B}} (\mathbf{Y'Y - 2Y'XB + B'X'XB)}$$</p>
<p>And set to 0 to find the minimum:</p>
<p>$$ \frac{\partial SSE }{\mathbf{\partial B}} = \mathbf{-2X'Y + 2X'XB} = 0 $$</p>
<p>Then isolate \( \mathbf{B} \) and simplify:</p>
<p>$$ \mathbf{B} = \mathbf{(X'X)^{-1}X'Y} $$</p>
<h3>Getting Real World Data</h3>
<p>Pulling data from <a href="http://www.autotrader.com">autotrader.com</a> for Honda CR-V's, I came up with 9 observations across three simple variables: Price, Mileage, and Year. </p>
<table>
<thead>
<tr>
<th><div class="text-center"> PRICE </div></th>
<th><div class="text-center">YEAR</div></th>
<th><div class="text-center">MILEAGE</div></th>
</tr>
</thead>
<tbody>
<tr>
<td>$19998</td>
<td>2012</td>
<td>16568</td>
</tr>
<tr>
<td>$16995</td>
<td>2011</td>
<td>68399</td>
</tr>
<tr>
<td>$22491</td>
<td>2013</td>
<td>23813</td>
</tr>
<tr>
<td>$27571</td>
<td>2014</td>
<td>15156</td>
</tr>
<tr>
<td>$25998</td>
<td>2014</td>
<td>17201</td>
</tr>
<tr>
<td>$24000</td>
<td>2012</td>
<td>28946</td>
</tr>
<tr>
<td>$15495</td>
<td>2010</td>
<td>87440</td>
</tr>
<tr>
<td>$13290</td>
<td>2007</td>
<td>83060</td>
</tr>
<tr>
<td>$8449</td>
<td>2006</td>
<td>153549</td>
</tr>
</tbody>
</table>
<p>Using car price as a function of car year in a simple bivariate model, one can find the OLS slope coefficient in R using the simple call:</p>
<div class="highlight"><pre><span></span>options("scipen"=100, "digits"=4)
price &lt;- c(19998,16995,22491,27571,25998,24000,15495,13290,8444)
mileage &lt;- c(16568,68399,23813,15156,17201,28946,87440,83060,153549)
year &lt;- c(2012,2011,2013,2014,2014,2012,2010,2007,2006)
crv &lt;- data.frame(mileage,price,year)
lm(crv$price ~ crv$year)
</pre></div>
<p>Which yields the basic result that the asking price of the car goes down by $2103 for each year it gets older. </p>
<h3>The Bivariate Example Using Simple Algebra and Arithmetic</h3>
<p>This is the important part. It's tedious, but straightforward, and writing it out all by hand will really remind you that computers are remarkable tools.</p>
<p>The derivation results from Part I tell us:</p>
<p>$$ \frac{\hat{cov}(x,y)}{var(x)} = \frac{ \frac{1}{n} \sum_{i=1}^{n} (x_{i} - \bar{x})(y_{i} - \bar{y} )}{ \frac{1}{n} \sum_{i=1}^{n} (x_{i} - \bar{x})^{2} } = \hat{\beta_{1}}$$</p>
<p>In this case \( \bar{x} \) is the sample mean for years, which is 2011, and \( \bar{y} \) is the sample mean for price, which is $19365.</p>
<p>Expanding the estimator out to its completely tangible form yields the ridiculously cumbersome equation:</p>
<h6>$$ \frac{((2012-2011)(19998-19365))+ ((2011-2011)(16995-19365))+ ((2013-2011)(22491-19365))+ ((2014-2011)(27571-19365))+ ((2014-2011)(25998-19365))+ ((2012-2011)(24000-19365))+ ((2010-2011)(15495-19365))+ ((2007-2011)(13290-19365))+ ((2006-2011)(8449-19365))}{((2012-2011)^{2})+((2011-2011)^{2})+((2013-2011)^{2})+((2014-2011)^{2})+((2014-2011)^{2})+((2012-2011)^{2})+((2010-2011)^{2})+((2007-2011)^{2})+((2006-2011)^{2})}$$</h6>
<p>Which simplifies to </p>
<p>$$ \frac{138787}{66} = \$2102.83 $$ </p>
<p>Almost exactly what R told us the coefficient would be!</p>
<h3>The Bivariate Example Using Matrix Algebra</h3>
<p>Things get a little trickier here, but the process is the same, the bivariate estimator is:</p>
<p>$$ \mathbf{\hat{B}} = \mathbf{(X'X)^{-1}X'Y} $$</p>
<p>Substituting our real world data for the general matrices leaves the form:</p>
<p>$$ \left({{\begin{bmatrix}
1 & 2012 \\
1 & 2011 \\
1 & 2013 \\
1 & 2014 \\
1 & 2014 \\
1 & 2012 \\
1 & 2010 \\
1 & 2007 \\
1 & 2006
\end{bmatrix}}}'{\begin{bmatrix}
1 & 2012 \\
1 & 2011 \\
1 & 2013 \\
1 & 2014 \\
1 & 2014 \\
1 & 2012 \\
1 & 2010 \\
1 & 2007 \\
1 & 2006
\end{bmatrix}}\right)^{-1} {{\begin{bmatrix}
1 & 2012 \\
1 & 2011 \\
1 & 2013 \\
1 & 2014 \\
1 & 2014 \\
1 & 2012 \\
1 & 2010 \\
1 & 2007 \\
1 & 2006
\end{bmatrix}}}' \begin{bmatrix}
19998\\
16995\\
22491\\
27571\\
25998\\
24000\\
15495\\
13290\\
8449
\end{bmatrix} = \mathbf{\hat{B}}$$</p>
<p>Which simplifies from 4 sets of matrices down to two as:</p>
<p>$$ \begin{bmatrix}
61274.67171717 & -30.4696969606 \\
-30.469696969696 & 0.015151515151515
\end{bmatrix}
\begin{bmatrix}
174 287 \\
350 629 944
\end{bmatrix} $$</p>
<p>Which multiplies through to:</p>
<p>$$ \begin{bmatrix}
-420 9380\\
2102.83
\end{bmatrix} $$</p>
<p>The first row being the intercept in our regression model; the second being our estimated slope coefficient calculated in both R and using the expanded summation operator by hand!</p>
<h3>The Takeaway</h3>
<p>Derivations, proofs, and advanced topics in econometrics can be really tricky to wrap your ahead around at first. By going through the motions of making real world data stand in for the notation, it can a) help you really understand the proofs and derivations, and b) remind you just how many trillions of human hours computers save in answering complicated technical questions!</p>Customizing Pelican for Static and Dynamic Content2015-09-13T13:55:00-07:002015-09-13T13:55:00-07:00Mischa Fishertag:mischafisher.com,2015-09-13:/customizing-pelican-for-static-and-dynamic-content.html<p>This is a a brief summary of a few changes I think were extremely helpful in tweaking the stock <em>Pelican</em> build for a customized, statically generated website:</p>
<h3>Rendering Math Markup</h3>
<p>Mathjax seemed to be the simplest solution. Embedding the information via a short snippet of code linking to their CDN, and then using simple LaTeX notation within Markdown takes only a minute or two:</p>
<div class="highlight"><pre><span></span><span class="o">&</span><span class="nt">lt</span><span class="o">;</span><span class="nt">script</span> <span class="nt">src</span><span class="o">=</span><span class="s1">'https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'</span><span class="o">&</span><span class="nt">gt</span><span class="o">;&</span><span class="nt">lt</span><span class="o">;/</span><span class="nt">script</span><span class="o">&</span><span class="nt">gt</span><span class="o">;</span>
</pre></div>
<h3>Using Bootstrap for Responsive CSS</h3>
<p>Erik Flowers Bootstrap grid introduction is a very nice resource <a href="http://www.helloerik.com/bootstrap-3-grid-introduction">(Click Here)</a></p>
<h3>Customizing Your Own Theme</h3>
<p>Web developer Robert Iwancz made a great bare bones framework on which one can build out almost anything. <a href="http://www.voidynullness.net/blog/2014/03/30/introducing-voidy-bootstrap-pelican-theme/">(Click Here for his VoidyNullness Theme)</a></p>
<h3>Using Pelican for a Static Landing Page</h3>
<p>Find your .md markdown file that you want to use as your home page content and add the following metadata to the top of the file: (the second change is to prevent your home page from appearing in the menu twice, depending on your theme). </p>
<div class="highlight"><pre><span></span><span class="n">save_as</span><span class="o">:</span> <span class="n">index</span><span class="o">.</span><span class="na">html</span>
<span class="n">status</span><span class="o">:</span> <span class="n">hidden</span>
</pre></div>
<p>Then in a separate .md file that will become your blog content, add the metadata identifier at the top to assign the appropriate HTML template from your templates folder:</p>
<div class="highlight"><pre><span></span>Title: (Your Blog Title)
Date: (The Date)
Category: Page
Template: (the title of your blog template, without the .html extension
</pre></div>
<p>Finally, copy the content generating loops from the default page templates into your new blog html template:</p>
<div class="highlight"><pre><span></span><span class="cp">{%</span> <span class="k">block</span> <span class="nv">content_body</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">block</span> <span class="nv">article</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">if</span> <span class="nv">articles</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">for</span> <span class="nv">article</span> <span class="k">in</span> <span class="o">(</span><span class="nv">articles_page.object_list</span> <span class="k">if</span> <span class="nv">articles_page</span> <span class="k">else</span> <span class="nv">articles</span><span class="o">)</span> <span class="cp">%}</span><span class="x"></span>
<span class="x">&lt;article&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">for</span> <span class="nv">file</span> <span class="k">in</span> <span class="nv">CUSTOM_INDEX_ARTICLE_HEADERS</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">include</span> <span class="s2">"includes/"</span> <span class="o">+</span> <span class="nv">file</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">else</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">include</span> <span class="s2">"includes/article_header.html"</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endfor</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">if</span> <span class="nv">ARTICLE_FULL_FIRST</span> <span class="k">is</span> <span class="nf">defined</span> <span class="k">and</span> <span class="nb">loop</span><span class="nv">.first</span> <span class="k">and</span> <span class="k">not</span> <span class="nv">articles_page.has_previous</span><span class="o">()</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;div class="content-body"&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">if</span> <span class="nv">article.standfirst</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;p class="standfirst"&gt;</span><span class="cp">{{</span> <span class="nv">article.standfirst</span><span class="o">|</span><span class="nf">e</span> <span class="cp">}}</span><span class="x">&lt;/p&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{{</span> <span class="nv">article.content</span> <span class="cp">}}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">include</span> <span class="s2">"includes/comments.html"</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;/div&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">else</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">include</span> <span class="s2">"includes/index_summary.html"</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="x">&lt;/article&gt;</span>
<span class="x">&lt;hr /&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endfor</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">endblock</span> <span class="nv">article</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">block</span> <span class="nv">pagination</span> <span class="cp">%}</span><span class="x"></span>
<span class="x">&lt;nav class="index-pager"&gt;</span>
<span class="cp">{%</span> <span class="k">if</span> <span class="nv">articles_page</span> <span class="k">and</span> <span class="nv">articles_paginator.num_pages</span> <span class="p">&</span><span class="nv">gt</span><span class="p">;</span> <span class="m">1</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;ul class="pagination"&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">if</span> <span class="nv">articles_page.has_previous</span><span class="o">()</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li class="prev"&gt;</span>
<span class="x"> &lt;a href="</span><span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span><span class="x">/</span><span class="cp">{{</span> <span class="nv">articles_previous_page.url</span> <span class="cp">}}</span><span class="x">"&gt;</span>
<span class="x"> &lt;i class="fa fa-chevron-circle-left fa-fw fa-lg"&gt;&lt;/i&gt; Previous</span>
<span class="x"> &lt;/a&gt;</span>
<span class="x"> &lt;/li&gt;</span>
<span class="cp">{%</span> <span class="k">else</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li class="prev disabled"&gt;&lt;span&gt;</span>
<span class="x"> &lt;i class="fa fa-chevron-circle-left fa-fw fa-lg"&gt;&lt;/i&gt; </span>
<span class="x"> Previous&lt;/span&gt;</span>
<span class="x"> &lt;/li&gt;</span>
<span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">for</span> <span class="nv">num</span> <span class="k">in</span> <span class="nv">articles_paginator.page_range</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">if</span> <span class="nv">num</span> <span class="o">==</span> <span class="nv">articles_page.number</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li class="active"&gt; &lt;span&gt;</span><span class="cp">{{</span> <span class="nv">num</span> <span class="cp">}}</span><span class="x">&lt;/span&gt; &lt;/li&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">else</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li&gt;</span>
<span class="x"> &lt;a href="</span><span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span><span class="x">/</span><span class="cp">{{</span> <span class="nv">articles_paginator.page</span><span class="o">(</span><span class="nv">num</span><span class="o">)</span><span class="nv">.url</span> <span class="cp">}}</span><span class="x">"&gt;</span><span class="cp">{{</span> <span class="nv">num</span> <span class="cp">}}</span><span class="x">&lt;/a&gt;</span>
<span class="x"> &lt;/li&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">endfor</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">if</span> <span class="nv">articles_page.has_next</span><span class="o">()</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li class="next"&gt;</span>
<span class="x"> &lt;a href="</span><span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span><span class="x">/</span><span class="cp">{{</span> <span class="nv">articles_next_page.url</span> <span class="cp">}}</span><span class="x">"&gt;</span>
<span class="x"> Next &lt;i class="fa fa-chevron-circle-right fa-fw fa-lg"&gt;&lt;/i&gt;</span>
<span class="x"> &lt;/a&gt;</span>
<span class="x"> &lt;/li&gt;</span>
<span class="cp">{%</span> <span class="k">else</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li class …</span></pre></div><p>This is a a brief summary of a few changes I think were extremely helpful in tweaking the stock <em>Pelican</em> build for a customized, statically generated website:</p>
<h3>Rendering Math Markup</h3>
<p>Mathjax seemed to be the simplest solution. Embedding the information via a short snippet of code linking to their CDN, and then using simple LaTeX notation within Markdown takes only a minute or two:</p>
<div class="highlight"><pre><span></span><span class="o">&</span><span class="nt">lt</span><span class="o">;</span><span class="nt">script</span> <span class="nt">src</span><span class="o">=</span><span class="s1">'https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'</span><span class="o">&</span><span class="nt">gt</span><span class="o">;&</span><span class="nt">lt</span><span class="o">;/</span><span class="nt">script</span><span class="o">&</span><span class="nt">gt</span><span class="o">;</span>
</pre></div>
<h3>Using Bootstrap for Responsive CSS</h3>
<p>Erik Flowers Bootstrap grid introduction is a very nice resource <a href="http://www.helloerik.com/bootstrap-3-grid-introduction">(Click Here)</a></p>
<h3>Customizing Your Own Theme</h3>
<p>Web developer Robert Iwancz made a great bare bones framework on which one can build out almost anything. <a href="http://www.voidynullness.net/blog/2014/03/30/introducing-voidy-bootstrap-pelican-theme/">(Click Here for his VoidyNullness Theme)</a></p>
<h3>Using Pelican for a Static Landing Page</h3>
<p>Find your .md markdown file that you want to use as your home page content and add the following metadata to the top of the file: (the second change is to prevent your home page from appearing in the menu twice, depending on your theme). </p>
<div class="highlight"><pre><span></span><span class="n">save_as</span><span class="o">:</span> <span class="n">index</span><span class="o">.</span><span class="na">html</span>
<span class="n">status</span><span class="o">:</span> <span class="n">hidden</span>
</pre></div>
<p>Then in a separate .md file that will become your blog content, add the metadata identifier at the top to assign the appropriate HTML template from your templates folder:</p>
<div class="highlight"><pre><span></span>Title: (Your Blog Title)
Date: (The Date)
Category: Page
Template: (the title of your blog template, without the .html extension
</pre></div>
<p>Finally, copy the content generating loops from the default page templates into your new blog html template:</p>
<div class="highlight"><pre><span></span><span class="cp">{%</span> <span class="k">block</span> <span class="nv">content_body</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">block</span> <span class="nv">article</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">if</span> <span class="nv">articles</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">for</span> <span class="nv">article</span> <span class="k">in</span> <span class="o">(</span><span class="nv">articles_page.object_list</span> <span class="k">if</span> <span class="nv">articles_page</span> <span class="k">else</span> <span class="nv">articles</span><span class="o">)</span> <span class="cp">%}</span><span class="x"></span>
<span class="x">&lt;article&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">for</span> <span class="nv">file</span> <span class="k">in</span> <span class="nv">CUSTOM_INDEX_ARTICLE_HEADERS</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">include</span> <span class="s2">"includes/"</span> <span class="o">+</span> <span class="nv">file</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">else</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">include</span> <span class="s2">"includes/article_header.html"</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endfor</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">if</span> <span class="nv">ARTICLE_FULL_FIRST</span> <span class="k">is</span> <span class="nf">defined</span> <span class="k">and</span> <span class="nb">loop</span><span class="nv">.first</span> <span class="k">and</span> <span class="k">not</span> <span class="nv">articles_page.has_previous</span><span class="o">()</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;div class="content-body"&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">if</span> <span class="nv">article.standfirst</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;p class="standfirst"&gt;</span><span class="cp">{{</span> <span class="nv">article.standfirst</span><span class="o">|</span><span class="nf">e</span> <span class="cp">}}</span><span class="x">&lt;/p&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{{</span> <span class="nv">article.content</span> <span class="cp">}}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">include</span> <span class="s2">"includes/comments.html"</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;/div&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">else</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">include</span> <span class="s2">"includes/index_summary.html"</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="x">&lt;/article&gt;</span>
<span class="x">&lt;hr /&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endfor</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">endblock</span> <span class="nv">article</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">block</span> <span class="nv">pagination</span> <span class="cp">%}</span><span class="x"></span>
<span class="x">&lt;nav class="index-pager"&gt;</span>
<span class="cp">{%</span> <span class="k">if</span> <span class="nv">articles_page</span> <span class="k">and</span> <span class="nv">articles_paginator.num_pages</span> <span class="p">&</span><span class="nv">gt</span><span class="p">;</span> <span class="m">1</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;ul class="pagination"&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">if</span> <span class="nv">articles_page.has_previous</span><span class="o">()</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li class="prev"&gt;</span>
<span class="x"> &lt;a href="</span><span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span><span class="x">/</span><span class="cp">{{</span> <span class="nv">articles_previous_page.url</span> <span class="cp">}}</span><span class="x">"&gt;</span>
<span class="x"> &lt;i class="fa fa-chevron-circle-left fa-fw fa-lg"&gt;&lt;/i&gt; Previous</span>
<span class="x"> &lt;/a&gt;</span>
<span class="x"> &lt;/li&gt;</span>
<span class="cp">{%</span> <span class="k">else</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li class="prev disabled"&gt;&lt;span&gt;</span>
<span class="x"> &lt;i class="fa fa-chevron-circle-left fa-fw fa-lg"&gt;&lt;/i&gt; </span>
<span class="x"> Previous&lt;/span&gt;</span>
<span class="x"> &lt;/li&gt;</span>
<span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">for</span> <span class="nv">num</span> <span class="k">in</span> <span class="nv">articles_paginator.page_range</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">if</span> <span class="nv">num</span> <span class="o">==</span> <span class="nv">articles_page.number</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li class="active"&gt; &lt;span&gt;</span><span class="cp">{{</span> <span class="nv">num</span> <span class="cp">}}</span><span class="x">&lt;/span&gt; &lt;/li&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">else</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li&gt;</span>
<span class="x"> &lt;a href="</span><span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span><span class="x">/</span><span class="cp">{{</span> <span class="nv">articles_paginator.page</span><span class="o">(</span><span class="nv">num</span><span class="o">)</span><span class="nv">.url</span> <span class="cp">}}</span><span class="x">"&gt;</span><span class="cp">{{</span> <span class="nv">num</span> <span class="cp">}}</span><span class="x">&lt;/a&gt;</span>
<span class="x"> &lt;/li&gt;</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">endfor</span> <span class="cp">%}</span><span class="x"></span>
<span class="cp">{%</span> <span class="k">if</span> <span class="nv">articles_page.has_next</span><span class="o">()</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li class="next"&gt;</span>
<span class="x"> &lt;a href="</span><span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span><span class="x">/</span><span class="cp">{{</span> <span class="nv">articles_next_page.url</span> <span class="cp">}}</span><span class="x">"&gt;</span>
<span class="x"> Next &lt;i class="fa fa-chevron-circle-right fa-fw fa-lg"&gt;&lt;/i&gt;</span>
<span class="x"> &lt;/a&gt;</span>
<span class="x"> &lt;/li&gt;</span>
<span class="cp">{%</span> <span class="k">else</span> <span class="cp">%}</span><span class="x"></span>
<span class="x"> &lt;li class="next disabled"&gt;</span>
<span class="x"> &lt;span&gt;&lt;i class="fa fa-chevron-circle-right fa-fw fa-lg"&gt;&lt;/i&gt; Next&lt;/span&gt;</span>
<span class="x"> &lt;/li&gt;</span>
<span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
<span class="x">&lt;/ul&gt;</span>
<span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span><span class="x"></span>
</pre></div>
<h3>Hiding Pages from the Menu</h3>
<p>Quite simple, just change the metadata on any given page's .md file to:</p>
<div class="highlight"><pre><span></span><span class="n">Status</span><span class="o">:</span> <span class="n">Hidden</span>
</pre></div>
<h3>Embedding Data from a CSV</h3>
<p>To generate my reading list, I use a piece of javascript that references the base .csv containing all the books. The script is fantastic because it allows all the flexibility of using a .csv to store the data, without needing a server side scripting language (which a static site on Amazon S3 doesn't have the luxury of relying on).</p>
<div class="highlight"><pre><span></span><span class="o">&</span><span class="nt">lt</span><span class="o">;</span><span class="nt">div</span> <span class="nt">class</span><span class="o">=</span><span class="s2">"d3-chart"</span><span class="o">&</span><span class="nt">gt</span><span class="o">;&</span><span class="nt">lt</span><span class="o">;/</span><span class="nt">div</span><span class="o">&</span><span class="nt">gt</span><span class="o">;</span>
<span class="o">&</span><span class="nt">lt</span><span class="o">;</span><span class="nt">script</span> <span class="nt">src</span><span class="o">=</span><span class="s2">"https://d3js.org/d3.v3.min.js"</span><span class="o">&</span><span class="nt">gt</span><span class="o">;&</span><span class="nt">lt</span><span class="o">;/</span><span class="nt">script</span><span class="o">&</span><span class="nt">gt</span><span class="o">;</span>
<span class="o">&</span><span class="nt">lt</span><span class="o">;</span><span class="nt">script</span> <span class="nt">src</span><span class="o">=</span><span class="s2">"d3.min.js?v=3.2.8"</span><span class="o">&</span><span class="nt">gt</span><span class="o">;&</span><span class="nt">lt</span><span class="o">;/</span><span class="nt">script</span><span class="o">&</span><span class="nt">gt</span><span class="o">;</span>
<span class="o">&</span><span class="nt">lt</span><span class="o">;</span><span class="nt">script</span> <span class="nt">type</span><span class="o">=</span><span class="s2">"text/javascript"</span><span class="nt">charset</span><span class="o">=</span><span class="s2">"utf-8"</span><span class="o">&</span><span class="nt">gt</span><span class="o">;</span>
<span class="nt">d3</span><span class="p">.</span><span class="nc">text</span><span class="o">(</span><span class="s2">"/images/data.csv"</span><span class="o">,</span> <span class="nt">function</span><span class="o">(</span><span class="nt">data</span><span class="o">)</span> <span class="p">{</span>
<span class="err">var</span> <span class="err">parsedCSV</span> <span class="err">=</span> <span class="err">d3.csv.parseRows(data)</span><span class="p">;</span>
<span class="err">var</span> <span class="err">container</span> <span class="err">=</span> <span class="err">d3.select(".d3-chart")</span>
<span class="err">.append("table")</span>
<span class="err">.selectAll("tr")</span>
<span class="err">.data(parsedCSV).enter()</span>
<span class="err">.append("tr")</span>
<span class="err">.selectAll("td")</span>
<span class="err">.data(function(d)</span> <span class="err">{</span> <span class="err">return</span> <span class="err">d</span><span class="p">;</span> <span class="p">}</span><span class="o">)</span><span class="p">.</span><span class="nc">enter</span><span class="o">()</span>
<span class="p">.</span><span class="nc">append</span><span class="o">(</span><span class="s2">"td"</span><span class="o">)</span>
<span class="p">.</span><span class="nc">text</span><span class="o">(</span><span class="nt">function</span><span class="o">(</span><span class="nt">d</span><span class="o">)</span> <span class="p">{</span> <span class="err">return</span> <span class="err">d</span><span class="p">;</span> <span class="p">}</span><span class="o">);</span>
<span class="err">}</span><span class="o">);</span>
<span class="o">&</span><span class="nt">lt</span><span class="o">;/</span><span class="nt">script</span><span class="o">&</span><span class="nt">gt</span><span class="o">;</span>
</pre></div>
<h3>Server and PHP free Contact Forms</h3>
<p>FormSpree is ridiculously headache free and works very well <a href="http://www.formspree.io">(Click Here)</a>. </p>
<h3>Adding a CSS Parallax Effect</h3>
<p>Using Javascript is an option since the browser does the work, rather than a server; however, I found a pure CSS approach to be cleaner and easier. <a href="http://keithclark.co.uk/articles/pure-css-parallax-websites/">Keith Clark's demo HERE</a> is superb. </p>The Pros and Cons of Using a Static Site Generator2015-09-12T16:55:00-07:002015-09-12T16:55:00-07:00Mischa Fishertag:mischafisher.com,2015-09-12:/the-pros-and-cons-of-using-a-static-site-generator.html<p>In a recent effort to re-gear this site toward the quantitative and away from the strictly artistic, I rebuilt the site from scratch with one singular aim: make posting effortless. In that spirit, I was directed by a <a href="http://sahandsaba.com/" title="Sahand Saba">good friend</a> toward static site generators; a new development in the 8 or so years since I had last looked at any of the technologies surrounding web development. </p>
<p>With the new site up and running, here is a brief list of the pros and cons, as I see them, of using a static site generator:</p>
<h3>Pros:</h3>
<ol>
<li>Effortless posting:</li>
</ol>
<p>Write posts in Markdown, then upload with a few keystrokes straight from the terminal. </p>
<ol>
<li>Cheap and scalable hosting: </li>
</ol>
<p>I'm using Amazon's S3 for hosting, and Route 53 for DNS services. They're almost free in low traffic, and infinitely scalable in high traffic. </p>
<ol>
<li>No backend to maintain: </li>
</ol>
<p>PHP, SQL, and the slow load times and unresponsiveness of shared servers on most hosting plans are a thing of the past. (I'm looking at you GoDaddy.com)</p>
<ol>
<li>Easy to backup or migrate: </li>
</ol>
<p>I have all the website files in the location of my laptop being backedup to Dropbox in realtime. In addition, version control through something like Git is also handy, particularly when messing around with the underlying Python scripts that generate the site.</p>
<h3>Cons:</h3>
<ol>
<li>Steep learning curve: </li>
</ol>
<p>The list of technologies one has to look at include: <em>HTML, CSS, Python, JavaScript, Markdown, the unix terminal, Pelican, FontAwesome, Jinja, Bootstrap, s3cmd, pip, brew,</em> and virtual environments. </p>
<ol>
<li>Longer to get setup: </li>
</ol>
<p>With a squarespace account you can be up and running in minutes. And it will look a lot prettier by default.</p>
<p>For me those were the biggest pros and cons I weighed in setting this site up. (Your mileage may vary.)</p><p>In a recent effort to re-gear this site toward the quantitative and away from the strictly artistic, I rebuilt the site from scratch with one singular aim: make posting effortless. In that spirit, I was directed by a <a href="http://sahandsaba.com/" title="Sahand Saba">good friend</a> toward static site generators; a new development in the 8 or so years since I had last looked at any of the technologies surrounding web development. </p>
<p>With the new site up and running, here is a brief list of the pros and cons, as I see them, of using a static site generator:</p>
<h3>Pros:</h3>
<ol>
<li>Effortless posting:</li>
</ol>
<p>Write posts in Markdown, then upload with a few keystrokes straight from the terminal. </p>
<ol>
<li>Cheap and scalable hosting: </li>
</ol>
<p>I'm using Amazon's S3 for hosting, and Route 53 for DNS services. They're almost free in low traffic, and infinitely scalable in high traffic. </p>
<ol>
<li>No backend to maintain: </li>
</ol>
<p>PHP, SQL, and the slow load times and unresponsiveness of shared servers on most hosting plans are a thing of the past. (I'm looking at you GoDaddy.com)</p>
<ol>
<li>Easy to backup or migrate: </li>
</ol>
<p>I have all the website files in the location of my laptop being backedup to Dropbox in realtime. In addition, version control through something like Git is also handy, particularly when messing around with the underlying Python scripts that generate the site.</p>
<h3>Cons:</h3>
<ol>
<li>Steep learning curve: </li>
</ol>
<p>The list of technologies one has to look at include: <em>HTML, CSS, Python, JavaScript, Markdown, the unix terminal, Pelican, FontAwesome, Jinja, Bootstrap, s3cmd, pip, brew,</em> and virtual environments. </p>
<ol>
<li>Longer to get setup: </li>
</ol>
<p>With a squarespace account you can be up and running in minutes. And it will look a lot prettier by default.</p>
<p>For me those were the biggest pros and cons I weighed in setting this site up. (Your mileage may vary.)</p>