Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
tools:habittask:parse [2025/12/05 14:19] – removed - external edit (Unknown date) 127.0.0.1tools:habittask:parse [2025/12/05 14:22] (current) – [Data] will
Line 1: Line 1:
 +====== Habit Task Data Parsing ======
  
 +[[:tools:habittask]] EEG, MR, and Behave data is coalesced by scripts in the task directory ''/Volumes/Hera/Projects/Habit/task/results'' but relies on task parsing scripts in ''/Volumes/Hera/Projects/Habit/mr/habit'' (MR directory, but parsing behave and eeg too). 
 +Age (when available) is pulled from yet another location, ''../../redcap/data/ages.tsv'' (''make -C /Volumes/Hera/Projects/Habit/redcap data/ages.tsv'')
 +
 +
 +Task files of interest are ''lab.data.tsv'' and ''lab.summary.csv''
 +
 +<code>
 +trial_file=/Volumes/Hera/Projects/Habit/task/results/lab.data.tsv # also see lab.summary.csv
 +wc -l < $trial_file
 +# 80279
 +
 +head -n 3 $trial_file
 +id      initials        age     sex     task    vdate   timepoint       run     ver     avatar  trial   block   left_pos        left_prob      up_pos   up_prob right_pos       right_prob      score   rt      n_keys  first_key_rt    first_key       picked  picked_prob     picked_step    avoided  avoided_prob    avoided_step    iti_onset       chose_onset     waiting_onset   feedback_onset  timeout_onset   trial_choices   survey_age      pick_dist       optimal_choice  blocknum        blocktype       picked_unified  avoid_unified
 +11885   NA      24      NA      habit_eeg       NA      20221012        1       eeg_1   shark   1       Lc20Uc50Rf100   c       20      c      50       f       100     true    557     1       510     39      right   100     2       up      50      1       211415.96       212424.14      212980.92        213243.78       NA      up-right        24      far     TRUE    1       init    first100        first50
 +11885   NA      24      NA      habit_eeg       NA      20221012        1       eeg_1   shark   2       Lc20Uc50Rf100   c       20      c      50       f       100     false   450     1       430     37      left    20      1       up      50      1       213536.44       214554.2       215004.04        215236.92       NA      left-up 24      close   FALSE   1       init    first20 first50
 +
 +</code>
 +
 +
 +===== Data =====
 +JSON files are exported from the browser at the end of the task. They're on bea_res like:
 +<code>
 +/Volumes/L/bea_res/Data/Tasks/Habit/MR/11734_20221111/sub-11734_task-mr_habit_ses-20221005_run-1_1668193517800.json
 +</code>
 +
 +===== Scripts =====
 +
 +==== Session information ====
 +Age is pulled from redcap and qualtrics. Fallback db query fills for ages know via other studies. 
 +''/Volumes/Hera/Projects/Habit/behave/redcap/02_visit_ages.R'' does all of this. Despite living in the [[tools:redcap]] scripts dir, it pulls to [[tools:qualtrics]] and DB).
 +
 +==== Task data ====
 +Extracting and coalescing data happens with MR scripts. See ''make json2tsv'' (tsv per task file) and ''txt/all_habit.json'' (single large file).
 +<code>
 +cd /Volumes/Hera/Projects/Habit/mr/habit
 +make  json2tsv
 +# runs eg.
 +#  ./read_idv_task.bash "/Volumes/L/bea_res/Data/Tasks/Habit/MR/12010_20251124/12010_mr_habit_20251001_1_nover.json" "/Volumes/L/bea_res/Data/Tasks/Habit/MR/12010_20251124/12010_mr_habit_20251001_1_nover.tsv"
 +</code>
 +
 +==== Unifying task conditions ====
 +
 +''read.R'' adds  ''picked_unified''  ''avoid_unified''. All trials are put into one long file.
 +
 +These transformation and scoring scripts are 
 +  * on rhea as ''/Volumes/Hera/Projects/Habit/task/results''
 +  * symlinked as ''/Volumes/Hera/Projects/Habit/mr/habit/task-results''
 +  * tracked with task presentation code: https://github.com/LabNeuroCogDevel/choice-landscape/tree/master/results
 +
 +===== Automated =====
 +
 +Scripts from age extraction to final ''lab.data.tsv'' should run unattended nightly using [[:tools:cron]] and [[:tools:make]].
 +
 +<code>
 +crontab -l|grep habit -i # foranw
 +30 2 * * * make -C /Volumes/Hera/Projects/Habit/behave/redcap
 +00 4 * * * make -C /Volumes/Hera/Projects/Habit/mr
 +00 5 * * * make -C /Volumes/Hera/Projects/Habit/task results/lab.summary.csv
 +
 +</code>