using R + ess-remote with screen in emacs

Dear list,

I brought up this issue before but a good solution never arised: being able to use screen on a remote server (so if something goes wrong on my side I can always resume that R session) inside of emacs in order to utilize ESS. The closest thing I found to a good work flow was to use ansi-term or multi-term and copying and pasting code in Emacs (ESS only worked nicely with shell-mode; reason at the end). Some would advise to use screen and open emacs inside of that screen, and voila, you have the luxuries of screen (attaching the session anywhere) and emacs+ESS (keybindings, etc). However, I prefer to use one main emacs session on my laptop/netbook (all my configurations are there), where I can have multiple projects and multiple R jobs opened at once.

I would like to share what I have working (for the time being), with the help of Michael Zeller in case others are interested.

  1. Place the following in .emacs:
  <pre class="src src-sh">;; used to send screen keybindings to shell<span style="color: #00ffff;"> in</span> emacs

(define-key shell-mode-map (kbd “C-l”) (lambda (seq) (interactive “k”) (process-send-string nil seq))) (define-key inferior-ess-mode-map (kbd “C-l”) (lambda (seq) (interactive “k”) (process-send-string nil seq)))

  1. In xterm (or the likes), ssh to the remote server and start screen. Detach it. (Need to do this first as starting the initial screen in emacs shell-mode becomes very ugly with the printing; resuming the same screen session also becomes messy in xterm)
  2. In emacs, M-x shell. Set:
  <pre class="src src-sh">$ <span style="color: #eedd82;">TERM</span>=xterm <span style="color: #ff4500;">## </span><span style="color: #ff4500;">give clearing capabilities to shell-mode</span>

  1. ssh to remote server. screen -r to resume the screen session. start R. M-x ess-remote.
  2. Send R code from R source files like before!
  3. To detach or do anything screen related, precede EACH keybinding with C-l. For example, C-a C-d to detach will now be C-l C-a C-l C-d. Yes this is cumbersome, but I don’t imagine screen keybinding to be used much at this stage since we are developing and debugging R code for say a simulation study =].

I would also like to note (for archival reasons) that ess-remote does not work with ansi-term and multi-term because of the inferior-ess-mode command, which stems from comint-mode and inferior-ess-send-input. If you remove this command in ess-remote, you don’t get an error but u can only send one line of code at a time from the R file.

Hope this helps someone out there.

Run screen in emacs with ansi-term (combine this with emacs + ess + remote R)

This is actually an update to this post, but since I discovered a few more things, I’ll write a new post. To run screen within a shell buffer in emacs, I tried M-x shell and fired up screen (ditto with M-x term). It gave me this error: Clear screen capability required. I found the solution to this here. To fix this issue, do M-x ansi-term (use /bin/bash when asked of course). screen now works inside emacs. Combine this with running a remote R session in emacs, and there you have it, the perfect work flow for developing and running computationally intensive R code! I can utilize screen to not have my R sessions interrupted, and I can utilize ESS to send code to an R session/buffer. I have to say, this WILL be the way I use R for any computationally-intensive project!

UPDATE

So screen doesn’t work in emacs after I ssh to a remote server inside ansi-term. I get the error: Cannot find terminfo entry for 'eterm-color'. To fix this, I put the following in my remote server’s .bashrc file (info from here.):

if [ "$TERM" = "eterm-color" ] ; then
 TERM="vt100"
fi

UPDATE AGAIN (better solution)

This page (Remote Term Type section) shows how to fix the e-term color issue. Do make sure you create the .terminfo folder if its not there:

$ scp /usr/share/emacs/22.1/etc/e/eterm-color username@remoteserver:~/.terminfo/e/eterm-color
$ scp /usr/share/emacs/22.1/etc/e/eterm-color.ti username@remoteserver:~/.terminfo/e/eterm-color.ti

UPDATE AGAIN 2

So copying files into .terminfo didn’t fix everything. I guess SunOS servers don’t look in my home directory for those files. I guess we can copy things into /usr/share/lib/terminfo/?/* (more information at man terminfo), but I don’t have access to this location in some of my servers. I will have to resort back to the old trick (changing TERM). This time, change it to xterm (this doesn’t give me funny characters in emacs ansi-term); found this at the bottom of this page. Put the following in the remote .bashrc file:

TEMP=`uname`
if [ $TEMP = "SunOS" ]; then
 if [ "$TERM" = "eterm-color" ] ; then
 TERM="xterm"
 fi
fi

I hope there aren’t any more issues. What the previous trick does is check if system is SunOS, and if so, use xterm. I got the unix command information from here. I got the uname command info from here.

FINAL UPDATE

To get eterm-color to work in SunOS, put the following in my .bashrc file:

##following to get eterm-color working in SunOS
TERMINFO=${HOME}/.terminfo
export TERMINFO

I guess I’ve been doing this, but I never exported TERMINFO. Didn’t know this was needed. Make sure the eterm-files are copied over (see top of post). Now everything should work, hopefully flawlessly. To summarize, copy eterm files into ~/.terminfo, and put the TERMINFO stuff in ~/.bashrc.

Now screen works in emacs. An issue that arised from this method is that when screen is run inside emacs, I can’t execute ess-remote anymore because I can’t press M-x. In ansi-term, C-x C-j : behave like emacs, cursor can go anywhere C-x C-k : behave like a terminal (default) This is documented here and here. Press C-x C-j and I can press M-x again. However, ess-remote still doesn’t work.

I guess when I use screen, I am forced to copy and paste code. If I really must use screen with ESS, do the regular M-x shell. After logging into the remote server, execute “TERM\=’vt100′” in the shell. Now, run screen -> R -> ess-remote. I can send code with keypresses now, but screen steals some of my emacs key bindings. To fix this, put the following,

escape \^Oo

in my remote ~/.screenrc file to switch the default command key from C-a to C-o so it doesn’t conflict with my emacs key bindings (documented here).

More information on ansi term (like remaping C-x to C-c) can be found here.

This was a long post. Summary:

  1. ansi-term in emacs behaves VERY much like a terminal. I can run vi, emacs, etc, inside of it. Emacs behavior is ‘term’ and ‘shell’.
  2. I can change things by editing the env variable, TERM.
  3. Change keybinding in the remote .screenrc file.

NEED TO DO: get ess-remote to work with ansi-term and screen in emacs!

UPDATE2: It seems the best way to do things so far is to use ansi-term -> ssh to remote server -> screen -> R, then go to line run (C-c C-j) and copy and paste code from there. To get screen commands to work (like detach, etc), need to go back to char run (C-c C-k). Remember, I now use C-o instead of C-a (defined in .screenrc); this only works on a regular terminal or in emacs with ansi-term, not using ‘shell’ in emacs with the hack I mentioned up there.

Run a remote R session in emacs: emacs + ESS + R + ssh

I don’t know how, but somehow, I stumbled on how to run a remote R session in emacs.

Since Spring 2006 I’ve always used emacs with ESS to run R (did it on windows, switched to linux for years, and most recently, on my macbook). I liked this workflow because I get the same usual interface across multiple platforms. Plus, I use emacs for everything computery or scientific, like using coding Python or C.

Regarding the same interface across multiple platforms, I use, for example, emacs + ESS + R whenever I remotely log into the remote servers dedicated to computing. I just need to ssh into the server, fire up emacs and fire up R. However, I almost always write all of my code on the local computer, and when I’m ready to run the final code, I either run it as a batch script (utlizing nohup and &), through screen (to keep the session runnning after I log out, see my post on R with unix tools), or through emacs. These days, I’ve been doing it with screen mainly so disconnects to the server won’t interrupt my script.

I just found yet another way to do this: write code on my local computer and then send code to a remote R session in my local emacs. I ran into this by googling ‘emacs ess multiple R session’. Instructions are described here (section 3.3: ESS on remote computers). We need this file for things to work. This site clarified how to get graphics to work.

Instructions as follow:

  1. Download the ssh.el file.
  2. Install it the usual way or place it in ~/elisp or ~/.emacs.d. In your emacs init file, add:
 <pre class="src src-sh">;; add path to emacs

;; http://edward.oconnor.cx/2005/09/installing-elisp-files (add-to-list ‘load-path “~/elisp”)

;; load ssh.el file from elisp ;; this is to run ESS remotely on another computer in my own emacs, or just plai n old reading remote files ;; (require ‘ssh)

  1. Fire up emacs. Type ‘M-x ssh’. For the host settings, do something like ‘-X -C username@server.com’ (X is for X windows forwarding, C is for compression of graphics, so plots can be displayed faster). Type in password.
  2. You wil be logged into a shell session on your server. Fire up R by typing R then enter. R is now running in an emacs buffer. Type M-x ess-remote. For dialect, select r.
  3. Open up any .R file on your computer, and use the usual keyboard shortcuts to send code to the remote R session.

W can also achieve the same results without the ssh.el file. In emacs, type M-x shell. In the shell buffer, ssh into the server and then run R. Type M-x ess-remote and everything should still work.

Next thing to get working is to open remote files in my local emacs.

Emacs: AucTeX + Rubber + Sweave

I got rubber to work with AucTeX and Sweave (Rnw) files with the help of this.

Basically, combined with my other stuff, I tweaked my .emacs file to look like:

<pre class="src src-sh">;;following is AucTeX with Sweave -- works

;;http://andreas.kiermeier.googlepages.com/essmaterials (setq TeX-file-extensions ‘(“Snw” “Rnw” “nw” “tex” “sty” “cls” “ltx” “texi” “texinfo”)) (add-to-list ‘auto-mode-alist ‘(“\.Rnw\'” . Rnw-mode)) (add-to-list ‘auto-mode-alist ‘(“\.Snw\'” . Snw-mode)) (add-hook ‘Rnw-mode-hook (lambda () (add-to-list ‘TeX-command-list (“Sweave” “R CMD Sweave %s” TeX-run-command nil (latex-mode) :help “Run Sweave”) t) (add-to-list ‘TeX-command-list (“LatexSweave” “%l %(mode) %s” TeX-run-TeX nil (latex-mode) :help “Run Latex after Sweave”) t) ;; following 3 lines for rubber, taken from same site as next paragraph, http://www.nabble.com/sweave-and-auctex-td23492805.html, xpdf to open (add-to-list ‘TeX-command-list (“RubberSweave” “rubber -d %s && open ‘%s.pdf'” TeX-run-command nil t) t) (setq TeX-command-default “Sweave”)))

;; AucTeX with rubber ;;http://www.nabble.com/sweave-and-auctex-td23492805.html (eval-after-load “tex” ‘(add-to-list ‘TeX-command-list ;;‘(“Rubber” “rubber -d %t && xpdf ‘%s.pdf‘” TeX-run-command nil t) t)) ;; change by vinh (“Rubber” “rubber -d %t && open ‘%s.pdf'” TeX-run-command nil t) t))

Now, when an Rnw file is open, I can press C-c C-c, select Sweave. Then repeat, select RubberSweave (or LatexSweave).

Editing/adding on to Sweave features in ESS

I really like David Whiting’s Sweave methods in ESS: M-n s M-n P and I get a compiled pdf file. However, i wanted to change it so Sweave uses the cacheSweaveDriver() when sweaving. I knew the the functions were defined in the ess-swv.el file (in my carbon emacs in Mac OS X, it is at /Applications/Emacs.app/Contents/Resources/site-lisp/ess/ess-swv.el). Now, I edited the file to create a new way to Sweave:

<pre class="src src-emacs-lisp"><span style="color: #ff4500;">;;; </span><span style="color: #ff4500;">changed by vinh</span>

(defun ess-swv-run-in-R2 (cmd &optional choose-process) “Run \[cmd] on the current .Rnw file. Utility function not called by user.” (let* ((rnw-buf (current-buffer))) (if choose-process ;; previous behavior (ess-force-buffer-current “R process to load into: “) ;; else (update-ess-process-name-list) (cond ((= 0 (length ess-process-name-list)) (message “no ESS processes running; starting R”) (sit-for 1); so the user notices before the next msgs/prompt (R) (set-buffer rnw-buf) ) ((not (string= “R” (ess-make-buffer-current))); e.g. Splus, need R (ess-force-buffer-current “R process to load into: “)) ))

(save-excursion (ess-execute (format “require(tools)”)) ;; Make sure tools is loaded. (basic-save-buffer); do not Sweave/Stangle old version of file ! (let* ((sprocess (get-ess-process ess-current-process-name)) (sbuffer (process-buffer sprocess)) (rnw-file (buffer-file-name)) (Rnw-dir (file-name-directory rnw-file)) (Sw-cmd (format “local({..od <- getwd(); setwd(%S); %s(%S, cacheSweaveDriver()); setwd(..od) })” Rnw-dir cmd rnw-file)) ) (message “%s()ing %S” cmd rnw-file) (ess-execute Sw-cmd ‘buffer nil nil) (switch-to-buffer rnw-buf) (ess-show-buffer (buffer-name sbuffer) nil)))))

(defun ess-swv-weave2 () “Run Sweave on the current .Rnw file.” (interactive) (ess-swv-run-in-R2 “Sweave”))

(define-key noweb-minor-mode-map “\M-nw” ‘ess-swv-weave2)

Now, I can do M-n w to Sweave with cacheSweaveDriver(). Tried it, didn’t work. After many trials, i decided to put this code in my .emacs file. Works. Hmm, wierd. Is ESS using a different ess-swv.el file? Tried deleting it from where i think it is. M-n s still works. Weird!!!

After some more searching, found out that the stuff is going on in the ess-swv.elc file. Deleted both files, M-n s doesn’t work anymore. Googled what *.elc is. It is a byte-compiled version of .el (to make things run faster in emacs, sometimes). So I did it to mine.

Byte compile for emacs: http://www.cs.utah.edu/dept/old/texinfo/emacs18/emacs\_26.html

So I open my ess-swv.el file in emacs, typed M-x byte-compile-file

Voila! .elc file created. Close emacs and run it again!