demo-parallel-tasks

Example used to demonstrate running tasks in parallel.

Image is in public domain and taken from https://en.wikipedia.org/wiki/Automation.

Preparation

$ mkdir -p result
$ export MAGICK_THREAD_LIMIT=1
$ convert 01.jpg -paint 5 result/01.jpg

The -paint 5 can be increased for longer runs and more "rough painting". The export MAGICK_THREAD_LIMIT=1 is important to avoid that Imagemagick itself runs on several threads.

Links created with:

for number in $(seq -f %02g 2 10); do
    ln -s 01.jpg ${number}.jpg
done

Using Makefile

See the Makefile.

Using a shell loop

Running one after another:

for file in *.jpg; do
    convert ${file} -paint 5 result/${file}
done

Run in batches of four:

counter=0
for file in *.jpg; do
    echo starting converting ${file}
    convert ${file} -paint 5 result/${file} &
    counter=$(( counter + 1 ))
    [ $(( ${counter} % 4 )) -eq 0 ] && wait
done
wait

xargs can run in parallel

$ find * -maxdepth 0 -name "*.jpg" | xargs      -I _ convert _ -paint 5 result/_
$ find * -maxdepth 0 -name "*.jpg" | xargs -P 4 -I _ convert _ -paint 5 result/_

Using GNU parallel

$ find * -maxdepth 0 -name "*.jpg" | parallel 'convert {} -paint 5 result/{}'

• Thanks to Ashwin Vishnu for showing me this!
• It also has an interesting / peculiar way of enforcing citations (but this did not work for me).