Markdown & Notebooks

Markdown & Notebooks provide interactive documents for more effective data communication

Impetus

In current data analytics and communication, there are a wide variety of platforms on which we can provide summaries and insights regarding our work. Each of these end points requires a non-insignificant amount of effort to learn these systems. Moreover, they all are cul de sacs in that all the effort you exert to learn one will not allow you to get the benefits of any other platform than the one you just learned.

This is stupid. - R. Dyer

Enter Pandoc, the universal document converter. Some really smart programmers have put together a set of software that allows you to convert from or two (and hence between) different document types given that most documents are regularly structured. With Pandoc, it does not matter if you do or do not have Word or Powerpoint or EPub or LaTeX or whatever, as long as you can create one of the supported types, you can convert that input into a huge variety of output types.

Pandoc Supported Conversions

Conversion Formats

This is critical for us because Code is just text. Once it is evaluated, it can replaced with:

• Numerical values from one or more calculations,
• Textual content from analyses or manipulation, or
• Graphical content from plots.

As such, we can embed R code within raw text to create our analyses and documents.

Markdown

For maximum usability, the document that we embed our code into should be as widely available as possible—unhindered by the necessity of having a particular program just to view the content. For this, R uses Markdown, created by John Gruber & Aaron Swartz in 2004. Markdown was created so that people are enabled “…to write using an easy-to-read and easy-to-write plain text format…”

Because everything is text, it is easy share and collaborate using Markdown, and for R, it is how we can make a wide array of output document types including (but not limited to):

• Conventional documents (PDF, Word, RTF, etc.)
• HTML pages with interactive elements (this document here is an interactive html document).
• Presentations (LaTeX, Powerpoint, JavaScript, etc.)
• Dashboards with interactive content.

Text Markup

When we make a document, presentation, or any other output, there are only a finite set of different text components we can put into the document. The document itself does not need to be heavy or bloated, it is just text (though surprisingly, a blank Word document on my laptop with nothing in the document itself is still 12KB in size!). Common elements include:

• Headers & Titles
• Typography such as italic, bold, underline, strike through
• Lists (numbered or as bullets)
• Pictures and links
• Page numbers, tables of contents, etc.

What Markdown does is allows you to type these components and use ‘marking’ around the elements to make them different from regular text. It is really, amazingly simple.

Title and headers are created by prepending a hashtag

# Header 1
## Header 2
### Header 3
#### Header 4

are knit into the following headers styles.

Header 1

Header 2

Header 3

Header 4

The actual appearance of the headers are determined by where it is being presented (e.g., in Word it will take the default typography and font attributes, etc.).

In text markdown examples are shown below and are contained within paragraphs of text. Individual paragraphs are delimited by either a blank line between them or two spaces at the end of the sentence.

Markdown	Rendered As
plain text	plain text
*italic*	italic
**bold**	bold
~~strike through~~	strike through

You can also embed links and images. Both of these are configured in two parts. For links, you need to specify the text that can be clicked upon and it must be surrounded in square brackets. The link to the web or file or image is right next to the square brackets and is contained within parentheses. The difference between link and image is that images have alternative text (or at times captions) and the whole thing has an exclamation mark in front of it. Here are some examples.

Markdown	Rendered As
[CES](https://ces.vcu.edu)	CES


Lists (both numbered and unordered) are created using dashes or asterisks.

- Bullet 1
- Bullet 2
- Bullet 3

Will be turned into an unordered list as:

• Bullet 1
  
• Bullet 2
  
• Bullet 3

Whereas the following raw text.¹

0. First
1. Second
2. Third

Will be rendered in list format as:

1. First
   
2. Second
   
3. Third

Actually, you can just use 1. in front of every line if you like, it will auto-number them for you when it makes a list. Helpful if you are making a numbered list that you are moving things around in.

Code & Text

On of the strengths of RMarkdown is the ability to mix code and text together in one place. This allows us to bring all of our analyses and data as close to one another as possible, helping with reproducibility and error reduction.

Inline Code

You can easily integrate code, into the text, either to be displayed OR to be evaluated. For example, in R you get the value of \(\pi\) by the constant pi. Type that into the console and it will return 3.1415927.

If you look at the RMarkdown for that paragraph above, it looks like the following before knitting:

You can easily integrate code, into the text, either to be displayed *OR* to be evaluated.  For example, in `R` you get the value of $\pi$ by the constant `pi`. Type that into the console and it will return 3.1415927.

Notice the following parts:

• Symbols: The \(\pi\) symbol is created by the name of the symbol surrounded by dollar signs. $\pi$. There are a ton of symbols and equations you can use, all borrowed from LaTeX, so if you need complicated equations or symbols, this is not a problem.
• Text rendered as code (in typography) but not evaluated: Both the `R` and the `pi` are examples here. Nothing is evaluated, but it looks like code.
• Evaluated R Code: Any code between `r and ` will be evaluated as R code within the text. When you knit the document, it will be run and the contents between the `r and ` are replaced by the output of the R code. The example here was `pi` at the end of the last sentence.

Code Chunks

In addition to code within the text, RMarkdown supports code chunks, which can be one or many lines of raw R code. This code is executed and the results are merged into the markdown in the document (text, graphical, interactive widgets, whatever) before knitting.

Each chunk is enclosed within boundary rows, the top row must contain three acute accents (back ticks - `) followed by the letter r in curly brackets ```{r}. The end of the chunk is indicated by three back ticks on their own line such as ```. Everything between these two enclosing lines is treated as R code and is subject to evaluation when you re-knit the document.

Here is what a chunk looks like in markdown that prints out a simple message "This is text from a chunk.

```{r}
print(“This is text from a chunk”)
```

When it is evaluated, the R interpreter removes the first and last rows, and executes the code within them. By default, the code is presented as a box in the output as well as any output that is produced from the code.

print("This is text from a chunk")

[1] "This is text from a chunk"

The first line in the chunk can also be used to modify the behavior of the code. There are several options that you can place within the curly brackets, including:

• {r eval=FALSE}: Will not evaluate (e.g., run) the code. The default value is TRUE.
  
• {r echo=FALSE}: Will not show the code in the document. This is great for our final version of our analyses, we want the output but not the code chunks showing. The default value is TRUE.
  
• {r message=FALSE, warning=FALSE, error=FALSE}: These suppress the messages that R prints out on occasion.

See the reference guide for several more options you can put into the header of each chunk.

Code Chunks in Document

There are some very fundamental issues regarding chunks, the R environment, and documents that should be pointed out here.

1. The R environment (see tab labeled Environment in the RStudio interface) has all the variables and new functions that you have created listed and available for use.
2. An R Markdown document is not a ‘living’ environment. If you make a change in a chunk, you must rerun that chunk to have the output available and inserted into the Environment. It does not do it automagically.
3. When you knit a document, the only data it has is what is actually in the document itself. It does not look to the general Environment for variables and functions. This means that if you create a variable or load data using the Console and then reference it in the Document, it will fail when you try to knit the document.
4. All the code and variables in a document (if they are not within a chunk with eval=FALSE) is visible to everything in the document below where it was defined.
   
5. Chunks are evaluated from the top of the document to the bottom of the document.
   
6. The options for each chunk are available from the setup menu on the top right of the chunk itself (the gear icon). Additional options include a button to run all the chunks prior to this one as well as running this particular chunk (see image).

Option buttons for each chunk include a quick menu for optoins (gear), the ability to run all the chunks above this one (triangle and line button in the middle), and run this particular chunk (play button).

Document Types

RMarkdown can be converted easily into a variety of document types. Let’s take a look at how the base Markdown document can be converted into a wide variety of output types.

RMarkdown Document

In RStudio, select File \(\to\) New File \(\to\) R Markdown… and you should see the following dialog.

New R Markdown Dialog from RStudio

For most of this class, we will be using HTML output. This creates a stand-alone document, with all the content embedded within it as a self-contained HTML web page. This is preferable at this point because it we will be working with interactive graphics, included maps, overlays, graphics, dynamic data tables, etc. None of these components can be embedded within PDF or Word documents, though you could take a picture of them and use that… But that would defeat the purpose and utility of interactive graphics and output now wouldn’t it?

The raw document in RStudio looks like this:

There are a couple different parts to this document itself that need to be highlighted.

1. The meta data: The part at the top of the document, lines 1-6, are meta data that summarize the document and specify how it would be converted into a final document. The process of converting the markdown to a final document type is called knitting (and there is a knit button above the document). For manuscripts and web pages and presentations, the values of title author and date are used for the top of the document.

2. Code Chunks: Raw code (R or other languages) that is to be evaluated when we knit the document is enclosed within chunks. These chunks are delimited by three back ticks (the tick is found on the key just to the left of the ‘1’ key on English (USA) keyboards). The first and last line must have these three ticks and everything contained within them will be evaluated as R code when you knit the document.

3. Chunk Options: On the first line of each code chunk, there is a curly bracket with r and potentially other items within it. The r specifies that the code will be R code and the other components are either an alias to for the chunk (e.g., the setup, cars and pressure on lines 8, 18 & 26) or options that apply to the overall chunk (the rest of the stuff between the curly brackets on the same lines). The names allow you to refer to the chunk from the text (such as referencing a Figure whose number will be automatically determined when knit) and the options allow you to determine the behavior of the chunk (figure size, orientation, showing, evaluating, suppressing warnings, etc.).

RStudio Interface Components

Surrounding the markdown document are several components of the RStudio IDE that help you be more productive. Here is a quick tour.

The Knit button at the top left of the window allows you to knit the docuemnt into different output types. This will default to the kind of document you selected in the original document dialog. If you change it, the value in the markdown meta data after output will change between html_document, pdf_document, and word_document.

In addition to R code, the insert menu on the top right of the interface lets you insert a chunk for R and some other programming languages. Mostly in this class, we will focus on R code and there is a keyboard shortcut to insert a chunk.

Please go back and read about code chunks and restrictions within documents (above).

The document structure menu at the bottom left of the window shows you the structure of the document you are making based upon headers and sections within it.

R Notebooks

An R Notebook is a special kind of Markdown document, which we will be using in this class. It adds a bit of interactivity to the normal Markdown documents. The differences are:

1. Instead of a Knit button at the top of the page, there is a Preview button.
   
2. Every time you save the document, an HTML version of the document will be rendered in the Viewer tab of the RStudio interface.
   
3. The output of code chunks within the Viewer for the document will show exactly the same output as is rendered in the raw markdown document. This means if you have code and the output has not been run, it will not show that in the Viewer.
4. You can show and hide all the code in a notebook using the button at the top of the page or on a per-chunk basis.
5. The HTML output of the notebook also contains a copy of the rmd file within it. This means that we can post a notebook online and a viewer can download the raw markdown for that document to their own computer and run it (at the top right of this very document is a button that has an option to Download Rmd).

For these reasons, we will be using notebooks for this class.

---

1. This is a footnote and is defined by enclosing square brackets and a carat symbol (^) where you want to put the footnote in the text (e.g., [^1]) and then at the bottom of the document add the text (this part) prepended by [^1]:. The linking to the footnote and back to the place you put it will be automagically inserted.↩︎