====== Habit Task Data Parsing ======

[[:tools:habittask]] EEG, MR, and Behave data is coalesced by scripts in the task directory ''/Volumes/Hera/Projects/Habit/task/results'' but relies on task parsing scripts in ''/Volumes/Hera/Projects/Habit/mr/habit'' (MR directory, but parsing behave and eeg too). 
Age (when available) is pulled from yet another location, ''../../redcap/data/ages.tsv'' (''make -C /Volumes/Hera/Projects/Habit/redcap data/ages.tsv'')


Task files of interest are ''lab.data.tsv'' and ''lab.summary.csv''

<code>
trial_file=/Volumes/Hera/Projects/Habit/task/results/lab.data.tsv # also see lab.summary.csv
wc -l < $trial_file
# 80279

head -n 3 $trial_file
id      initials        age     sex     task    vdate   timepoint       run     ver     avatar  trial   block   left_pos        left_prob      up_pos   up_prob right_pos       right_prob      score   rt      n_keys  first_key_rt    first_key       picked  picked_prob     picked_step    avoided  avoided_prob    avoided_step    iti_onset       chose_onset     waiting_onset   feedback_onset  timeout_onset   trial_choices   survey_age      pick_dist       optimal_choice  blocknum        blocktype       picked_unified  avoid_unified
11885   NA      24      NA      habit_eeg       NA      20221012        1       eeg_1   shark   1       Lc20Uc50Rf100   c       20      c      50       f       100     true    557     1       510     39      right   100     2       up      50      1       211415.96       212424.14      212980.92        213243.78       NA      up-right        24      far     TRUE    1       init    first100        first50
11885   NA      24      NA      habit_eeg       NA      20221012        1       eeg_1   shark   2       Lc20Uc50Rf100   c       20      c      50       f       100     false   450     1       430     37      left    20      1       up      50      1       213536.44       214554.2       215004.04        215236.92       NA      left-up 24      close   FALSE   1       init    first20 first50

</code>


===== Data =====
JSON files are exported from the browser at the end of the task. They're on bea_res like:
<code>
/Volumes/L/bea_res/Data/Tasks/Habit/MR/11734_20221111/sub-11734_task-mr_habit_ses-20221005_run-1_1668193517800.json
</code>

===== Scripts =====

==== Session information ====
Age is pulled from redcap and qualtrics. Fallback db query fills for ages know via other studies. 
''/Volumes/Hera/Projects/Habit/behave/redcap/02_visit_ages.R'' does all of this. Despite living in the [[tools:redcap]] scripts dir, it pulls to [[tools:qualtrics]] and DB).

==== Task data ====
Extracting and coalescing data happens with MR scripts. See ''make json2tsv'' (tsv per task file) and ''txt/all_habit.json'' (single large file).
<code>
cd /Volumes/Hera/Projects/Habit/mr/habit
make  json2tsv
# runs eg.
#  ./read_idv_task.bash "/Volumes/L/bea_res/Data/Tasks/Habit/MR/12010_20251124/12010_mr_habit_20251001_1_nover.json" "/Volumes/L/bea_res/Data/Tasks/Habit/MR/12010_20251124/12010_mr_habit_20251001_1_nover.tsv"
</code>

==== Unifying task conditions ====

''read.R'' adds  ''picked_unified''  ''avoid_unified''. All trials are put into one long file.

These transformation and scoring scripts are 
  * on rhea as ''/Volumes/Hera/Projects/Habit/task/results''
  * symlinked as ''/Volumes/Hera/Projects/Habit/mr/habit/task-results''
  * tracked with task presentation code: https://github.com/LabNeuroCogDevel/choice-landscape/tree/master/results

===== Automated =====

Scripts from age extraction to final ''lab.data.tsv'' should run unattended nightly using [[:tools:cron]] and [[:tools:make]].

<code>
crontab -l|grep habit -i # foranw
30 2 * * * make -C /Volumes/Hera/Projects/Habit/behave/redcap
00 4 * * * make -C /Volumes/Hera/Projects/Habit/mr
00 5 * * * make -C /Volumes/Hera/Projects/Habit/task results/lab.summary.csv

</code>