Overview¶
decu
stands for “Decu is a Experimental Computation
Utility”. decu
is a suite of command line tools to automate the
menial tasks involved in the development of experimental computation
projects.
We define a “experimental computation” script as a script that reads some data, performs an experiment (run an algorithm, train a model, plot a figure, etc), saves some results to disk, and then quits. Some tasks that are usually involved in this are:
- file I/O: choosing file names for results and plots
- multiprocessing: running many experiments/scripts in parallel
- timekeeping: how long algorithms take to run
- cloud service integration
- logging
decu
was born from the realization that none of these tasks have
anything to do with (and in fact get in the way of) the actual
experimentation being done. Furthermore, experimental computation scripts
tend to mix together code dedicated to these tasks with code intended to
run the actual algorithms of interest, thus making said algorithms harder
to maintain and debug.
The main goal of decu
is to provide standardization and automation
of these tasks, with end code that clearly separates experimental
computation from bookkeeping and other auxiliary code.
decu
is built with Best Practices for Data Science, by Greg Wilson
et al, in mind. See 1
and 2.