3  Finding your way in R and Python

3.1 Programming

Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning. - Rick Cook

Programming is the art of solving a problem by developing a sequence of steps that make up a solution, and then very carefully communicating those steps to the computer. To program, you need to know how to

• break a problem down into smaller, easily solvable problems
• solve small problems
• communicate the solution to a computer using a programming language

In this class, we’ll be using both R and Python, and we’ll be using these languages to solve problems that are related to working with data. At first, we’ll start with smaller, simpler problems that don’t involve data, but by the end of the semester, you will hopefully be able to solve some statistical problems using one or both languages.

It will be hard at first - you have to learn the vocabulary in both languages in order to be able to put commands into logical “sentences”. The problem solving skills are the same for all programming languages, though, and while those are harder to learn, they’ll last you a lifetime.

3.2 Hello world

I particularly like the way that Python for Everybody (Severance 2016) explains vocabulary:

Unlike human languages, the Python vocabulary is actually pretty small. We call this “vocabulary” the “reserved words”. These are words that have very special meaning to Python. When Python sees these words in a Python program, they have one and only one meaning to Python. Later as you write programs you will make up your own words that have meaning to you called variables. You will have great latitude in choosing your names for your variables, but you cannot use any of Python’s reserved words as a name for a variable.

When we train a dog, we use special words like “sit”, “stay”, and “fetch”. When you talk to a dog and don’t use any of the reserved words, they just look at you with a quizzical look on their face until you say a reserved word. For example, if you say, “I wish more people would walk to improve their overall health”, what most dogs likely hear is, “blah blah blah walk blah blah blah blah.” That is because “walk” is a reserved word in dog language. Many might suggest that the language between humans and cats has no reserved words.

The reserved words in the language where humans talk to Python include the following:

and       del       global      not       with
as        elif      if          or        yield
assert    else      import      pass
break     except    in          raise
class     finally   is          return
continue  for       lambda      try
def       from      nonlocal    while    

That is it, and unlike a dog, Python is already completely trained. When you say ‘try’, Python will try every time you say it without fail.

We will learn these reserved words and how they are used in good time, but for now we will focus on the Python equivalent of “speak” (in human-to-dog language). The nice thing about telling Python to speak is that we can even tell it what to say by giving it a message in quotes:

print('Hello world!')

Hello world!

And we have even written our first syntactically correct Python sentence. Our sentence starts with the function print followed by a string of text of our choosing enclosed in single quotes. The strings in the print statements are enclosed in quotes. Single quotes and double quotes do the same thing; most people use single quotes except in cases like this where a single quote (which is also an apostrophe) appears in the string.

R has a slightly smaller set of reserved words:

if          else     repeat      while        
for         in       next        break  
TRUE        FALSE    NULL        Inf         
NA_integer_ NA_real_ NA_complex_ NA_character_  
NaN         NA       function    ...

In R, the “Hello World” program looks exactly the same as it does in python.

print('Hello world!')

[1] "Hello world!"

In many situations, R and python will be similar because both languages are based on C. R has a more complicated history, because it is also similar to Lisp, but both languages are still very similar to C and run C or C++ code in the background.

3.3 Talking to Python and R - Interactive mode

R and python both have an “interactive mode” that you will use most often. In the previous chapter, we talked about scripts and markdown documents, both of which are non-interactive methods for writing R and python code. But for the moment, let’s work with the interactive console in both languages in order to get familiar with how we talk to R and python.

Let’s start by creating a Qmd file (File -> New File -> Quarto Document) - this will let us work with R and python at the same time.

Add an R chunk to your file by typing ```{r} into the first line of the file, and then hit return. RStudio should add a blank line followed by ```.

Add a python chunk to your file by typing ```{python} on a blank line below the R chunk you just created, and then hit return. RStudio should add a blank line followed by ```.

Your file should look like this:

If instead your file looks like this:

you have visual markdown mode on. To turn it off, click on the A icon at the top right of your editor window:

If we are working in interactive mode, why did I have you start out by creating a markdown document? Good Question! RStudio allows you to switch back and forth between R and python seamlessly, which is good and bad - it’s hard to get a python terminal without telling R which language you’re working in! You can create a python script if you’d prefer to work in a script instead of a markdown document, but that would involve working in 2 separate files, which I personally find rather tedious.

• The R Console
• The Python Console

In your R chunk or script, type in 2+2 and hit Ctrl+Enter (or Cmd+Enter on a mac). Look down to the Console (which is usually below the editor window) and see if 4 appears. If you’re like me, output shows up in two places at once:

Location	Picture
Chunk
Script
Console

R will indicate that it is waiting for your command with a > character in the console. If you don’t see that > character, chances are you’ve forgotten to finish a statement - check for parentheses and brackets.

When you are working in an R script, any output is shown only in the console. When you are working in an R code chunk, output is shown both below the chunk and in the console.

If you want, you can also just work within the R console. This can be useful for quick, interactive work, or if, like me, you’re too lazy to pull up a calculator on your machine and you just want to use R to calculate something quickly. You just type your R command into the console:

The first two statements in the above example work - “Hello” is a string, and is thus a valid statement equivalent to typing “2” into the console and getting “2” back out. The second command, print("Hello"), does the same thing - “Hello” is returned as the result. The third command, I love R, however, results in an error - there is an unexpected symbol (the space) in the statement. R thinks we are telling it to do something with variables I and love (which are not defined), and it doesn’t know what we want it to do to the two objects.

Suppose we define I and love as variables by putting a value into each object using <-, which is the assignment operator. Then, typing “I love” into the console generates the same error, and R tells us “hey, there’s an unexpected symbol here” - in this case, maybe we meant to add the two variables together.

In your python chunk or script, type in 2+2 and hit Ctrl+Enter (or Cmd+Enter on a mac). Look down to the Console (which is usually below the editor window) and see if 4 appears. If you’re like me, output shows up in two places at once:

Location	Picture
Chunk
Script
Console

Notice that in the console, you get a bit of insight into how RStudio is running the python code: we see reticulate::repl_python(), which is R code telling R to run the line in Python. The python console has >>> instead of > to indicate that python is waiting for instructions.

Notice also that the only difference between the R and python script file screenshots is that there is a different logo on the documents: . Personally, I think it’s easier to work in a markdown document and keep my notes with specific chunks labeled by language when I’m learning the two languages together, but when you are writing code for a specific project in a single language, it is probably better to use a script file specific to that language.

If you want to start the python console by itself (without a script or working in a markdown document), you can simply type reticulate::repl_python() into the R console.

R is nice enough to remind you that to end the conversation with python, you just need to type “exit” or “quit”.

If you want to start a python console outside of RStudio, bring up your command prompt (Darwin on mac, Konsole on Linux, CMD on Windows) and type python3 into that window and you should see the familiar >>> waiting for a command.

3.4 Talking to Python and R - Script Mode

In the last chapter, we played around with scripts and markdown documents for python and R. In the last section, we played with interactive mode by typing R and python commands into a console or running code chunks interactively using the Run button or Ctrl/Cmd + Enter (which is the keyboard shortcut).

You may be learning to program in R and python because it’s a required part of the curriculum, but hopefully, you also have some broader ideas of what you might do with either language - process data, make pretty pictures, write a program to trigger the computer uprising…

Scripts are best used when you have a thing you want to do, and you will need to do that thing many times, perhaps with different input data. Suppose that I have a text file and I want to pull out the most common word in that file. In the next few examples, I will show you how to do this in R and python, and at the same time, demonstrate the difference between interactive mode and script mode in both languages. In each example, try to compare to the previous example to identify whether something is running as a full script or in interactive mode, and how it is launched (in R? at the command line?).

Severance (2016) provides a handy python program to count words. This program is meant to be run on the command line, and it will run for any specified text file.

::: ex

name = input('Enter file:')
handle = open(name, 'r')
counts = dict()

for line in handle:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0) + 1

bigcount = None
bigword = None
for word, count in list(counts.items()):
    if bigcount is None or count > bigcount:
        bigword = word
        bigcount = count

print(bigword, bigcount)

# Code: http://www.py4e.com/code3/words.py

the 8854

name = "data/dickens-oliver-627.txt"
handle = open(name, 'r')
counts = dict()

for line in handle:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0) + 1

bigcount = None
bigword = None
for word, count in list(counts.items()):
    if bigcount is None or count > bigcount:
        bigword = word
        bigcount = count

print(bigword, bigcount)

# Code: http://www.py4e.com/code3/words.py

the 8854

# Read in the file
text <- readLines("dickens-oliver-627.txt")

# Split the lines of text into separate words
text <- strsplit(text, " ")

# Simplify the list
text <- unlist(text)

# Count up the number of occurrences of each word
word_freq <- table(text)

# Sort the table by decreasing frequency
word_freq <- sort(word_freq, decreasing = T)

# Show the counts for the most common 10 words
word_freq[1:10]

# Read in the file
text <- readLines("dickens-oliver-627.txt")

# Split the lines of text into separate words
text <- strsplit(text, " ")

text[[1]]

 [1] "The"       "Project"   "Gutenberg" "Etext"     "of"        "Oliver"   
 [7] "Twist"     "by"        "Charles"   "Dickens"  

# Simplify the list
text <- unlist(text)
text[[1]]

[1] "The"

text[1:20]

 [1] "The"       "Project"   "Gutenberg" "Etext"     "of"        "Oliver"   
 [7] "Twist"     "by"        "Charles"   "Dickens"   "#13"       "in"       
[13] "our"       "series"    "by"        "Charles"   "Dickens"   "Copyright"
[19] "laws"      "are"      

# Count up the number of occurrences of each word
word_freq <- table(text)
word_freq[1:5]

text
            _I_     --'    --by --kneel 
   4558       4       1       1       1 

# Sort the table by decreasing frequency
word_freq <- sort(word_freq, decreasing = T)
word_freq[1:5]

text
 the  and        to   of 
8854 4902 4558 3767 3763 

# Take arguments from the command line
args <- commandArgs(TRUE)

# Read in the file
text <- readLines(args[1])

# Split the lines of text into separate words
text <- strsplit(text, " ")

# Simplify the list
text <- unlist(text)

# Count up the number of occurrences of each word
word_freq <- table(text)

# Sort the table by decreasing frequency
word_freq <- sort(word_freq, decreasing = T)

# Show the counts for the most common 10 words
word_freq[1:10]

3.5 Getting help

In both R and python, you can access help with a ? - the order is just slightly different.

Suppose we want to get help on a for loop in either language.

In R, we can run this line of code to get help on for loops.

?`for`

Because for is a reserved word in R, we have to use backticks (the key above the TAB key) to surround the word for so that R knows we’re talking about the function itself. Most other function help can be accessed using ?function_name.

In python, we use for? to access the same information.

for?

(You will have to run this in interactive mode for it to work in either language)

w3schools has an excellent python help page that may be useful as well - usually, these pages will have examples. A similar set of pages exists for R help on basic functions