README.Rmd

---
title: "PARALLEL: Stata module for parallel computing"
author: ""
date: ""
output: 
  md_document:
    variant: "markdown_github"
---

<!-- This file was originally used to create README.md, but is currently stale (README.md has been updated directlyed). -->

# PARALLEL: Stata module for parallel computing

Parallel lets you **run Stata faster**, sometimes faster than MP itself. By organizing your job in several Stata instances, parallel allows you to work with out-of-the-box parallel computing. Using the the `parallel` prefix, you can get **faster simulations, bootstrapping, reshaping big data**, etc. without having to know a thing about parallel computing. With **no need of having Stata/MP** installed on your computer, parallel has showed to dramatically speedup computations up to two, four, or more times depending on how many processors your computer has.

See also the HTML version of the program [help file](https://rawgit.com/gvegayon/parallel/master/ado/parallel.html).

Stata 2017 conference presentation: <https://github.com/gvegayon/parallel/blob/master/talks/20170727_stata_conference/20170727_stata_conference_handout.pdf>

SSC at Boston College: <http://ideas.repec.org/c/boc/bocode/s457527.html> (though the SSC version is a bit out-of-date, see below)

1. [Installation](#installation)
2. [Minimal examples](#minimal-examples)
2. [Authors](#authors)

Citation {#citation}
=======

When using `parallel`, please include the following:

Vega Yon GG, Quistorff B. parallel: A command for parallel computing. The Stata Journal. 2019;19(3):667-684. doi:10.1177/1536867X19874242

Or use the following bibtex entry:

```bib
@article{
  VegaYon2019,
  author = {George G. {Vega Yon} and Brian Quistorff},
  title ={parallel: A command for parallel computing},
  journal = {The Stata Journal},
  volume = {19},
  number = {3},
  pages = {667-684},
  year = {2019},
  doi = {10.1177/1536867X19874242},
  URL = {https://doi.org/10.1177/1536867X19874242},
  eprint = {https://doi.org/10.1177/1536867X19874242}
}
```

Installation {#installation}
=======

If you have a previous installation of `parallel` installed from a different source (SSC, specific folder, specific URL) you should uninstall that first. Once installed it is suggested to restart Stata. 

SSC
---

For accessing SSC version of parallel

``` stata
ssc install parallel, replace
mata mata mlib index
```

Development Version (Latest/Master)
--------------------------

For accessing the latest development version of parallel (from here) using Stata version \>=13

``` stata
net install parallel, from(https://raw.github.com/gvegayon/parallel/master/) replace
mata mata mlib index
```

For Stata version \<13, download as zip, unzip, and then replace the above `net install` with

``` stata
net install parallel, from(full_local_path_to_files) replace
```

Development Version (Other Releases)
------------------------------------

Access other development releases via the [Releases Page](https://github.com/gvegayon/parallel/releases). You can use the release tag to install over the internet. For example,

``` stata
net install parallel, from(https://raw.github.com/gvegayon/parallel/v1.15.8.19/) replace
mata mata mlib index
```

Or you can download the release and install locally (for Stata \<13).

Minimal examples {#minimal-examples}
===============

The following minimal examples have been written to introduce how to use the module. Please notice that the only examples actually designed to show potential speed gains are [parfor](#parfor) and [bootstrap](#bootstraping).

The examples have been executed on a Dell Vostro 3300 notebook running Ubuntu 14.04 with an Intel Core i5 CPU M 560 (2 physical cores) with 8Gb of RAM, using Stata/IC 12.1 for Unix (Linux 64-bit x86-64).

For more examples and details please refer to the module's help file or the wiki [Gallery page](https://github.com/gvegayon/parallel/wiki/Gallery).

```{r setup, echo=FALSE}
knitr::opts_chunk$set(autodep = TRUE, echo=FALSE, comment='')
```

## Simple parallelization of egen

When conducted over groups, parallelizing `egen` can be useful. In the following example we show how to use `parallel` with `by: egen`.

```{stata}
parallel setclusters 2, f
sysuse auto
parallel, by(foreign): egen maxp = max(price)
tab maxp
```

Which is the ``parallel'' way to do:

```{stata}
sysuse auto
bysort foreign: egen maxp = max(price)
tab maxp
```


## Bootstrapping {#examples-bootstrap}

In this example we'll evaluate a regression model using bootstrapping which, together with simulations, is one of the best ways to use parallel

```{stata}
sysuse auto, clear
parallel setclusters 4, f
timer on 1
parallel bs, reps(5000): reg price c.weig##c.weigh foreign rep
timer off 1
timer list
```

Which is the ``parallel way'' to do:

```{stata}
sysuse auto, clear
timer on 2
bs, reps(5000) nodots: reg price c.weig##c.weigh foreign rep
timer off 2
timer list
```


## Simulation

From the `simulate` stata command:

```{stata}
parallel setclusters 2, f
program define lnsim, rclass
  version 12.1
  syntax [, obs(integer 1) mu(real 0) sigma(real 1) ]
  drop _all
  set obs `obs'
  tempvar z
  gen `z' = exp(rnormal(`mu',`sigma'))
  summarize `z'
  return scalar mean = r(mean)
  return scalar Var  = r(Var)
end
parallel sim, expr(mean=r(mean) var=r(Var)) reps(10000): lnsim, obs(100)

summ
```

which is the parallel way to do

```{stata}
program define lnsim, rclass
  version 12.1
  syntax [, obs(integer 1) mu(real 0) sigma(real 1) ]
  drop _all
  set obs `obs'
  tempvar z
  gen `z' = exp(rnormal(`mu',`sigma'))
  summarize `z'
  return scalar mean = r(mean)
  return scalar Var  = r(Var)
end
simulate mean=r(mean) var=r(Var), reps(10000) nodots: lnsim, obs(100)

summ
```

## parfor {#examples-parfor}

In this example we create a short program (`parfor`) which is intended to work as a `parfor` program, this is, looping through 1/N in a parallel fashion

```{stata}
// Cleaning working space
clear all
timer clear

// Set up
set seed 123
local n = 5e6
set obs `n'
gen x = runiform()
gen y_pll = .
clonevar y_ser = y_pll

// Loop replacement function
prog def parfor
	args var
	forval i=1/`=_N' {
		qui replace `var' = sqrt(x) in `i'
	}
end

// Running the algorithm in parallel fashion
timer on 1
parallel setclusters 4, f
parallel, prog(parfor): parfor y_pll
timer off 1

// Running the algorithm in a serial way
timer on 2
parfor y_ser
timer off 2

// Is there any difference?
list in 1/10
gen diff = y_pll != y_ser
tab diff

// Comparing time
timer list
di "Parallel is `=round(r(t2)/r(t1),.1)' times faster"

```

Building {#building}
================
If you need to use `parallel` on an older version of Stata than what we build here, you can build and install the package locally.

You will need to install [Stata devtools](https://github.com/gvegayon/devtools) to build the package and `log2html` to build the html version of the help.

Then you can go to `ado/` and either `do compile.do` or `do compile_and_install.do` depending on whether you want to just build the package (`.mlib`) or also install. There are also several build build checks in the `makefile` that can easily be run from Linux.

Authors {#authors}
======
George G. Vega [aut,cre]
g.vegayon %at% gmail

Brian Quistorff [aut]
brian-work %at% quistorff . com