Can I trust FMUT results?

FMUT is still in beta release and should be used with some caution. The functions that perform the core statistical calculations have been tested quite a bit using several different approaches (e.g., comparison to results from other software and simulation studies). I feel confident using FMUT in my own work.

HOWEVER: Because the software has not been used widely and tested by large numbers of users with different assumptions and different needs, it is possible there are still bugs.

For example, a while back a colleague of mine got some weird results using FMUT. It turns out she had loaded the wrong data—she was trying to do analyses on an extended epoch, but had loaded data with shorter epochs. The time window she was specifying in the FMUT functions therefore extended far beyond the actual length of the epoch. The code handling time windows was set up to find the time point closest to the one specified. (This is because you may ask for a 300 to 400 ms time window, but if your data is sampled at, say, 512 Hz, the closest thing you can get is 300.8 to 400.4.) So when my colleague specified a time window far beyond the epoch length, FMUT simply performed analyses on a time window that went to the end of the existing epoch and did not produce any error message. I had never encountered this issue because in my testing of the software, I had never thought to test an impossible time window!

This problem was corrected in the 0.3.2 release. But there are probably more errors like this that still exist, and they may produce incorrect results without it being obvious why.

So while I have tested the software quite a bit and feel confident it produces correct results in the large majority of use cases, it is still labelled as being in beta for a reason. Proceed at your own risk, and always look at your results with a critical eye (probably a good idea with any software!). If they don't make sense, or something looks wrong, please email me (fieldsec@westminster.edu) or the Mass Univariate Toolbox list and we'll try to figure it out.

If you are a MATLAB programmer and would like to see some of the testing FMUT has been through, see: https://github.com/ericcfields/FMUT/tree/master/testing