Introduction

• Waiting for your R code to run is a pain
• We’re going to talk about ways to make it faster
  • Without editing your code
• Two approaches:
  • Faster R
  • HPC at UoM

Optimising code

• Problem specific
• Profiling is a good starting point
  • Built into R studio

Optimising how your code runs

• Most PCs have 2, 4 or 8 cores.
  • R will only use one of these by default
• Two approaches to using more of them:
  • Explicitly parallise code
    • Not covering this in this talk
  • Use a version of R with libraries that use > 1 core

Microsoft R Open

• MS R Open uses optimised versions of BLAS and LAPACK.
  • Typically quicker even on a single thread
  • (Usually) even quicker if we use multiple threads
• Speed up will depend on what you’re doing
  • Linear algebra

Benchmarking R

Used the Revobenchmark in the version.compare package:

• Matrix multiplication
• Cholesky factorisation
• QR Decomposition
• Singular value decomposition
• PCA
• Linear Discriminant Analysis

(See MS R Blog for more details)

Compatibility between standard R and Microsoft R open

• Version lags (slightly) behind standard R
• “100% compatible with” standard R
• Uses its own version of CRAN; MRAN
  • Frozen snapshot of CRAN for each release of MS R Open
  • Can install CRAN packages with repos="URL" option
  • checkpoint library lets you use packages at arbitrary snapshot dates

What if this isn’t enough?

The CSF

• George Leaver talked about the iCSF in March
  • Interactive facility
• We’ll focus on the CSF
  • “Traditional” batch system
  • Write a job script, submit it, wait for it to run
• Aimed at research groups who have contributed to it
  • Limited free at the point of use availability
• Has several versions of standard R and MS R Open installed.

Getting started on the CSF

• Applying for an account

• Training course

• Demonstrations:
  1. Running a job on multiple cores
  2. Running the same job many times (job arrays)

Setting the number of threads

• Unclear why performance suffers if all cores used
  • Little benefit in > 8 cores
  • Less queuing time; easier to schedule jobs

• Set cores required with

  #$ -pe smp.pe 8 

  in your job script

• You must include the line:

  export OMP_NUM_THREADS=$NSLOTS

  in your job submission script

The parallel library and MS R Open

• Take care not to “double parallise”
  • Each processes spawned by parallel will spawn its own threads

  • can limit threads used by each process with

    setMKLthreads(<n>)

  • Experimentation will be needed to find the best balance between the two approaches

Getting help/training

• Research IT training courses
  • Next CSF course 20 November.
  • Next R courses 28 November and 5 December
  • Courses on Unix shell and Makefiles may also be of interest
• Feel free to get in touch directly, or via the Research IT request form
  • david.mawdsley@manchester.ac.uk
  • pen.richardson@manchester.ac.uk
• These slides are availabile at https://mawds.github.io/RUGCSF/RUGCSF.html

Conclusions

• Switching over to MS R Open can offer big speedups
  • No modification of code needed
  • Though some care needed with package versions
• The CSF is ideal for
  • bigger jobs
  • repeated jobs (using job arrays)
• Important to set the number of threads correctly