koboloadeR: Survey Data Crunching

koboloadeR is a metapackage, that brings together a series of specialised packages in an organised data analysis workflow, to conduct data discovery and analysis for data collected through KoboToolbox, ODK, ONA or any xlsform compliant data collection platform.

This package first builds on the capacity of UNHCR Kobo server but it can also be used from any structured dataset. It comes as a companion tool to the Integrated Framework for Household Survey.

koboloadeR aims at helping humanitarian data analysts to focus in data interpretation by saving the time needed to quickly generate the graphs and charts required to discover insights from a dataset. It also ensure analysis reproducibility through a separation of the analysis configuration and the analysis process. The package allows to account for sample weights and hierachical dataset structure (both capacities that are not available through the default reporting engine or the excel-analyzer).

Approach

The main concept behind the package is to implement a survey data analysis plan and configuration directly within the same xlsform excel file that has been used to develop the questionnaire. A few additional column are created in this excel document, the package read those column to generate a series of predefined report.

You can have a look at some examples of output reports here.

The approach offered through the package has the following advantages:

• End users do not need to code in R and to master the language in order to use the package;

• The data analysis plan is de facto fully documented and described;

• The resulting data crunching reports are fully reproducible;

• Analysis iterations are facilitated;

• Good practices are enforced through the package.

A more detailed introduction to the concepts used in the package is presented in the Data Crunching article.

Environment setup

Software installation

1. Install either Java JRE or Java JDK: JAVA is required to manipulate excel files.

Under Windows, You will need to have JRE or JDK register under the Path system variable (Right click on “This PC/Computer”, Go to ->Properties->Advanced system settings->Advanced->Environment Variables, then under System variables select Path => Click Edit.

If you install, the JDK (Java Development kit), Please make sure that JAVA_HOME is actually recorded as an Environment Variable.

If you install, the JRE (Java Runtime Environment), Please make sure that JRE_HOME is actually recorded as an Environment Variable.

Once in R, you may double-check that Environement variable are correctly, i.e. JAVA_HOME or JRE_HOME, set by

Sys.getenv()

If JAVA is not correctly set, you will see an installatin error at a latter stage when loading the package RJava.

Note in some case, you may need to reboot your computer to ensure that this environement variable is properly accounted for.

2. Install R: follow instruction from the installer.

3. Only for windows user Install RTools: This executable is needed to install the package from github. Follow instruction from the installer.

4. Install R Studio : follow instruction from the installer

You can now Launch R Studio . You may check the list of common issues in the Common Troubleshooting article.

Package installation from Github

Note that the package is still in beta-version. We hope to have soon a release available on CRAN.

• Open R studio interface and within the R console, install devtools package:

install.packages("devtools")

• Install koboloadeR:

library(devtools)
install_github("unhcr/koboloadeR", ref = "gh-pages") 

• You are all set! You can know use koboloadeR. If you have a problem consult the common troubleshooting part at the end of this page.

Project Walk Through

• In R Studio, select File, click New project. A box opens

• Choose New Directory

• Choose Empty project

• Type the name of the folder where you want to put your data

• Select where you want to put this folder

• Click Create project

Then setup a few things: run those two lines:

library (koboloadeR) # This loads koboloadeR package

kobo_projectinit() # Creates folders necessary and transfer files needed

It might take a while as a few other packages have to be installed or loaded. Once the see the " >" again at the beginning of the line, you are ready to start

Graphical user interface (GUI)

All instructions and options for the project configuration and analysis plan settings can be done through a dedicated GUI.

kobo_shiny("app_main_koboloadeR.R")

For better performances, select "Open in Browser" on the top of the window.

Console mode

Alternatively, you can use the console mode by running the file run-analysis.R that is automatically copied in the code folder after you run the kobo_projectinit() function .

This will be likely the quickest options, once your are used to the package.

Note however that this implies that you configure correctly on your own the full configuration within the xlform file.

Presentation of features

• Getting data from server - How to retrieve data from the server within R?

• Data Analysis Plan within your xlsfrom - How to extend the xlsform to include your analysis plan?

• Using console script - How to use the package without using the GUI in Shiny?

• Sampling: how to generate a sample from a registry or to post-stratify data when having a low response rate?

• Data Anonymisation and disclosure risk measurement: how to create anonymised data?

• Data Cleaning: how to use the package for reproducible and documented data cleaning?

• Predicting and scoring: how to use survey in conjunction wiht registration data to build risk prediction and vulnerability scoring?

• Dissiminating: how to dissiminate both survey microdata using DDI and variable crosstabulation on CKAN?

Contributing

Contributions to the packages are welcome. Please read first the contribution guidelines, follow the code of conduct and use the issue template.

To go in more details, the suggested workflow is presented below (note that all of it is not yet fully implented - see issue tracking for more details). You can read the function documentations directly.

CRAN Notes

Test results