Installing an older version of R in a conda environment

This will walk you through installed an old version (3.4.1) of R inside a conda environment. We assume you have anaconda installed and are somewhat familiar with it’s commands.

Installing R

First we create a new environment:

> conda create -n myRenv3_4 anaconda

and then switch to the new environment with:

> source activate myRenv3_4

To confirm we have this running properly we can ask where the pip command will run from (hopefully it is tied to a location inside our environment):

> which pip
/anaconda/envs/myRenv3_4/bin/pip

Note that R is not installed by default inside the environment:

> which R
/usr/bin/R

Next we search for the R version we want to install. Because conda’s search will match any packages that contain our search term as a substring (in our case the letter “r”) we search instead for “r-base”:

> conda search r-base
Fetching package metadata .........
r-base                       3.1.2                         0  defaults        
                             ...     
                             3.4.1                         0  defaults        
                             3.4.1                         1  defaults        
                             3.4.2                haf99962_0  defaults        
                             3.4.3                h290ecf8_0  defaults        
                             3.4.3                h290ecf8_1  defaults        
                             3.4.3                h1e0a451_2  defaults        
                             3.5.0                h1c2f66e_1  defaults        
                             3.5.0                h1e0a451_1  defaults        
                             3.5.1                h1e0a451_2  defaults        

Now that we’ve confirmed the package/version we want is available we R v3.4.1 with the following:

> conda install -c conda-forge r=3.4.1

One can also try installed with -c r instead of conda-forge but I’ve found that not all versions are available via r.

To confirm that this installed properly (you might have to source deactivate and source activate to get :

> which R
/anaconda/envs/myRenv3_4/bin/R

When we open R we should see the following:

> R
R version 3.4.1 (2017-06-30) -- "Single Candle"

but on some installs I’ve seen the following error:

> R
/anaconda/envs/myRenv3_4/lib/R/bin/exec/R: symbol lookup error: /anaconda/envs/myRenv3_4/lib/R/bin/exec/../../lib/../../libreadline.so.6: undefined symbol: PC

This can be fixed with an update to the readline package (see this ticket for more info):

> conda install -c conda-forge readline=6.2

Next, the Native R kernel for Jupyter is installed with:

> conda install -c r r-irkernel
Incompatible packages with our current R version

Some packages won’t be available for v3.4.1, for example the effects package:

> install.packages("effects")
Warning message:
package ‘effects’ is not available (for R version 3.4.1)

To get around this we will install oldr for installing packages compatible with R v3.4.

First we install the R package devtools:

> conda install -c r r-devtools

and then inside R we install oldr:

> require(devtools)
> devtools::install_github("duckmayr/oldr")

This often results in the following error:

Downloading GitHub repo duckmayr/oldr@master
from URL https://api.github.com/repos/duckmayr/oldr/zipball/master
Installation failed: error in running command

which can be fixed by (see this ticket):

> options(unzip = "internal")
> devtools::install_github("duckmayr/oldr")
> oldr::install.compatible.packages("effects", "3.4.1", "/anaconda/envs/myRenv3_4/lib/R/library")

Here we also have to specify the R version we are running and the location of the install.

Running a Remote Jupyter Notebook

I realize this is just a lesson in ssh port forwarding but I get asked this question very frequently so I’m making this my first post.

How to connect to a remote Jupyter Notebook

Step 1: Connect to the remote server

Log into your remote server and start a screen or tmux session:

> screen -R myNotebook

Step 2: Start the Jupyter server

Before starting the notebook we note that notebook files will be written to the directory from which you are running the notebook. So we will first create a directory called my_notebooks, move into that directory and then start the server:

> mkdir my_notebooks && cd "$_"
> jupyter notebook --no-browser --port=myportnumber

Here myportnumber is a free port on the remote machine (for example 8887). Note if port is already in use on the remote server you will get the follow message (in this example myportnumber=8887):

> jupyter notebook --no-browser --port=8887
....
[I 09:03:23.972 NotebookApp] The port 8887 is already in use, trying another random port.
[I 09:03:23.975 NotebookApp] Serving notebooks from local directory: /home/sgiorgi
[I 09:03:23.975 NotebookApp] 0 active kernels 
[I 09:03:23.975 NotebookApp] The IPython Notebook is running at: http://localhost:8888/

To fix this use 8888 instead of myportnumber in the next command. In the last line above the http://localhost:8888/ is referring to your remote machine. In the next step we will map the remote host / port to your local host /port.

Step 3: Map the Jupyter server on the remote machine to your local machine

The general command for this mapping is

> ssh -i /path/to/privatekey/directory -NL port:localhost:myportnumber username@remote-host-name

So on your local machine run (preferably also inside a screen session):

> ssh -i /path/to/public/key -NL 8886:localhost:8888 username@remote-host-name

Here myportnumber refers to the port on your remote machine (8888 in this example) and port refers to an open port on your local machine (8886 in this example). There is no reason the two numbers need to be different. You can easily connect port 8888 on your remote machine to port 8888 on your local machine with ... 8888:localhost:8888 ...

From the ssh manual:

-i identity_file
             Selects a file from which the identity (private key) for public key authentication is read.  The default is ~/.ssh/identity for protocol version 1, and
             ~/.ssh/id_dsa, ~/.ssh/id_ecdsa, ~/.ssh/id_ed25519 and ~/.ssh/id_rsa for protocol version 2.  Identity files may also be specified on a per-host basis in the con‐
             figuration file.  It is possible to have multiple -i options (and multiple identities specified in configuration files).  ssh will also try to load certificate
             information from the filename obtained by appending -cert.pub to identity filenames.

-N      Do not execute a remote command.  This is useful for just forwarding ports (protocol version 2 only).
-L [bind_address:]port:host:hostport
             Specifies that the given port on the local (client) host is to be forwarded to the given host and port on the remote side.  This works by allocating a socket to
             listen to port on the local side, optionally bound to the specified bind_address.  Whenever a connection is made to this port, the connection is forwarded over the
             secure channel, and a connection is made to host port hostport from the remote machine.  Port forwardings can also be specified in the configuration file.  IPv6
             addresses can be specified by enclosing the address in square brackets.  Only the superuser can forward privileged ports.  By default, the local port is bound in
             accordance with the GatewayPorts setting.  However, an explicit bind_address may be used to bind the connection to a specific address.  The bind_address of
             “localhost” indicates that the listening port be bound for local use only, while an empty address or ‘*’ indicates that the port should be available from all
             interfaces.

Step 4: Open the notebook in a web browser

Open a web browser and go to localhost:port (in this example http://localhost:8886).

Tunnel through multiple servers

Here we setup a connection between your local machine (with port localport) and a remote server middlehost (with port middleport). From middlehost we establish a connection to the port finalport on the finalhost machine.

ssh -i /path/to/privatekey/directory -L localport:localhost:middleport username@middlehost -L middleport:localhost:finalport -N username@finalhost