# \[ANN\] Parallel and distributed execution of command lines, pardi!

**URL:** <https://discuss.ocaml.org/t/ann-parallel-and-distributed-execution-of-command-lines-pardi/4015>\
**Category:** Community\
**Tags:** cli, parallel\
**Created:** [July 2, 2019, 2:09am UTC](https://discuss.ocaml.org/t/ann-parallel-and-distributed-execution-of-command-lines-pardi/4015 "2019-07-02T02:09:38Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![UnixJunkie](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/unixjunkie/32/638_2.png) [@UnixJunkie](https://discuss.ocaml.org/u/UnixJunkie)\
**Post date:** [July 2, 2019, 2:09am UTC](https://discuss.ocaml.org/t/ann-parallel-and-distributed-execution-of-command-lines-pardi/4015/1 "2019-07-02T02:09:38Z")

</div>

Dear OCaml community,

I am pleased to announce the first release of pardi (which the community recently helped to debug):

> **[UnixJunkie/pardi](https://github.com/UnixJunkie/pardi)**
>
> Parallel and Distributed execution of command lines, pardi ! - UnixJunkie/pardi

Pardi is a command line tool to parallelize programs which are not parallel;  
provided that you can cut an input file into independent chunks.

For example, to compress a file in parallel using 1MB chunks:

```auto
pardi -d b:1048576 -m s -i <YOUR_BIG_FILE> -o <YOUR_BIG_FILE>.gz \
        -w 'xz -c -9 %IN > %OUT'

```

Using the right option, you can cut an input file by lines (e.g. SMI files),  
by number of bytes (for binary files),  
by separating lines verifying a regexp (quite generic)  
or by a block separating line (e.g. MOL2/SDF/PDB file formats).

If processing a single record of your input file is too fine grained,  
you can play with the -c option to reach better parallelization  
(try 10,20,50,100,200,500,etc).

```auto
usage:
pardi ...
  {-i|--input} <file>: where to read from (default=stdin)
  {-o|--output} <file>: where to write to (default=stdout)
  {-n|--nprocs} <int>: max jobs in parallel (default=all cores)
  {-c|--chunks} <int>: how many chunks per job (default=1)
  {-d|--demux} {l|b:<int>|r:<regexp>|s:<string>}: how to cut input 
  file into chunks (line/bytes/regexp/sep_line; default=line)
  {-w|--work} <string>: command to execute on each chunk
  {-m|--mux} {c|s|n}: how to mux job results in output file
(cat/sorted_cat/null; default=cat)

```

Pardi should be available soon in the opam repository.