Create an experiment with exp

Each experiment is a module (i.e., a ‘.py’ script). Usually, it does at least the following:

  • imports computer settings (import computer), see Computer setup
  • sets up paths (PATH variable)
  • defines a stimuli presentation class (inherited from Experiment)
  • defines an analysis class (inherited from object or, for fMRI analyses, inherited from Analysis).

Let’s see how it works out in our demo file (demos/scripts/main.py).

Importing computer settings

As discussed in Computer setup, psychopy_ext encourages users to keep computer settings in a plain text file (computer.py) in the same folder as your scripts (so typically in the scripts/ folder). Import this file at the top of your experiment so that all classes could use it:

import computer

If you need to modify certain settings (e.g., valid_responses that defines keyboard keys that are allowed for responding), do so during the initialization of your class (see Initialization: __init__())

Setting up paths: set_paths()

Paths specify where various bits of project information (data, logs etc) should be stored. The default is the following (found in set_paths()):

def set_paths(exp_root='', computer=default_computer, fmri_rel=''):
    fmri_root = os.path.join(computer.root, fmri_rel)
    if exp_root != '':
        exp_root += '/'
    paths = {
        'root': computer.root,
        'exp_root': exp_root,
        'fmri_root': fmri_root,
        'analysis': os.path.join(exp_root, 'analysis/'),  # where analysis files are stored
        'logs': os.path.join(exp_root, 'logs/'),
        'data': os.path.join(exp_root, 'data/'),
        'report': 'report/',
        'data_behav': os.path.join(fmri_root, 'data_behav/'),  # for fMRI behav data
        'data_fmri': os.path.join(fmri_root,'data_fmri/'),
        'data_struct': os.path.join(fmri_root,'data_struct/'),  # anatomical data
        'spm_analysis': os.path.join(fmri_root, 'analysis/'),
        'rec': os.path.join(fmri_root,'reconstruction/'), # CARET reconstructions
        'rois': os.path.join(fmri_root,'rois/'),  # ROIs (no data, just masks)
        'data_rois': os.path.join(fmri_root,'data_rois/'), # preprocessed and masked data
        'sim': exp_root,  # path for storing simulations of models
        }

PATHS variable is defined at the top of your script after all imports are done:

import computer  # computer params defined in the previous section
PATHS = exp.set_paths(exp_root='', computer=computer)

exp_root='' is the default setting recommended for a single experiment. It means that data is stored where the ‘run.py’ file is, in the ‘data’ folder. But if you have more than one experiment, exp_root='minimal' would be better (for the “minimal” experiment; for the “main”, you would call it exp_root='main'). That means that for this experiment will be in the ‘data’ folder inside the ‘minimal’ folder.

The experiment: Experiment

An experiment is conceptualized as a wrapper of one or more tasks that a participant is supposed to perform. For example, first you may want to train them on a certain task, and next you want to measure their performance. Each task is defined as a class (using Task, see Defining tasks: Task), and the two tasks can neatly be combined into a single continuous experiment with the Experiment class.

Note that if you only have a single task, you do not need to work with tasks at all. For your convenience, defining stimuli and trials directly in the Experiment class is possible (and encouraged).

So the experiment is defined in a class. The convenience of having classes is that we can now inherit multiple handy routines from exp.Experiment. Some of these routines are discussed below; otherwise, check the API reference.

So let’s define the Confsup class:

class Confsup(exp.Experiment):
    """
    The configural superiority effect experiment

    Explanation of the task here.
    """

Your experiments should always inherit from exp.Experiment. Also, to encourage the good practice of docstrings, task description is supposed to be provided in this docstring. exp.Experiment will take it and use it by default.

Initialization: __init__()

Here is a quick example:

def __init__(self, name='exp', info=('subjid', 'confsup_'), actions='run', **kwargs):
    super(Confsup, self).__init__(name=name, info=info, actions=actions,
                                  computer=computer,
                                  paths=PATHS, **kwargs)

    self.computer.valid_responses = {'num_4': 0, 'num_5': 1, 'num_1': 2, 'num_2': 3}
    self.stim_size = 3.  # in deg
    ...
    self.tasks = [_Train, _Test]

First, look at the parameters passed to the __init__() method:

  • name

    This is the name of this experiment which is used to call this particular experiment from CLI and which is seen as a tab label in the GUI.

  • info

    A dictionary or a tuple of information that you want to record in the output file. Usually, you want to record participant id, but sometimes also session number etc. Upon calling exp.Experiment.__init__(), info is updated to be OrderedDict (we use OrderedDict instead of a regular dictionary so that the order of entired is retained in the output file.) info will be updated to contain at least the following:

    OrderedDict([('subjid', 'subj')])
    

    So the default value for subjid is always subj unless, of course, you provided thsi value yourself, as in our example, in which case it becomes info['subjid'] = 'confsup'.

  • rp

    We omitted rp in this example because the default rp was sufficient. It stores other run parameters that you may want to manipulate via the CLI/GUI. By default:

    self.rp = OrderedDict([  # these control how the experiment is run
        ('no_output', False),  # do you want output? or just playing around?
        ('debug', False),  # not fullscreen presentation etc
        ('autorun', 0),  # if >0, will autorun at the specified speed
        ('unittest', False),  # like autorun but no breaks at show_instructions
        ('register', False),  # add and commit changes, like new data files?
        ('push', False),  # add, commit and push to a hg repo?
        ])
    
  • actions

    A list of function names in this class that are “callable” from CLI and are seen as buttons in the GUI.

  • computer

    The computer module (see Computer setup) that holds information of your computer parameters.

  • paths

    A dictionary of various paths for storing data (see Setting up paths: set_paths() above).

  • data_fname

    File name where data will be stored. (Similar to PsychoPy’s dataFilename.) This produces a Datafile instance (stored in self.datafile) which we use later to write data.

  • **kwargs

    This argument allows for other keywords arguments to be passed.

    ..note:: When you inherit this class, you must have at least info and rp (or simply **kwargs) keywords because Control expects them. In fact, all “callable” classes (the ones that can be accessed via GUI or CLI) must accept at least info and rp. Read more about this in Initialization procedure.

Next, the parent class (i.e., Experiment) is initialized. Observe the parameters that it accepts.

You should specify all “global” parameters such as stimulus size during the initialization of the Experiment. This is encouraged so that all parameters are in one place and defined before any other functions are called. Also notice how we use this opportunity to redefine valid_responses.

Finally, if you have more than one task, you have to provide it here by defining a self.tasks variable as list of tasks (classes) that you want to run in the oreder you want to run them. If you only have a single task, don’t bother with creating a Task and simply define your stimuli and trials in the same Experiment class.

Setup: setup()

Typical setup procedure consists of:

  • collecting run time information
  • setting up logging and seed (set_logging())
  • creating a window for stimuli presentation (create_win())

All these steps are conveniently wrapped into a setup() and you should try to use it unless you need something special. This means that usually the setup function is not redefined.

Run: run()

When you call run() (or run_exp()), the following steps are performed:

  • Setting up of the experiment (Setup: setup())
  • Instructions are shown (before_exp()). By default, the docstring of the experiment class is used thus encouraging you to keep up a good practice of commenting your code. Moreover, if you have docutils installed, reST syntax will be parsed automatically! And if you don’t have it, you can still format text with HTML tags (see more on pyglet’s website). You can also redefine what’s shown by default simply by redefining before_exp().
  • Each task is called in a row
  • Final message (“Thank you”) is show (after_exp())

This routine is usually suffient and is not redefined.

Defining tasks: Task

Stimuli and trials are defined using a Task class.

Arguments that each Task requires:

  • parent (Experiment)

    The Experiment to which this Tast belongs.

Other parameters:

  • name (str, default: ‘’)

    Name of the experiment. Currently not used anywhere.

  • version (str, default: ‘0.1’)

    Version of your experiment. Also not used.

  • method ({‘sequential’, ‘random’}, default: ‘random’)

    Order of trials:

    • sequential: trials and blocks presented sequentially
    • random: trials presented randomly, blocks sequentially
    • fullRandom: converted to ‘random’

    Note that there is no explicit possibility to randomize the order of blocks. This is intentional because you in fact define block order in the blockcol.

  • data_fname (str, default=None)

    The name of the main data file for storing output. If None, reuses Datafile instance from its parent; otherwise, a new one is created (stored in self.datafile).

  • blockcol (str, default: None)

    Column name in self.exp_plan that defines which trial should be presented during which block.

Create stimuli: create_stimuli()

Here you should define all your stimuli and put them in a self.s dictionary. This need not be a complete specification of all stimuli properties as usually some properties are defined during the runtime. However, at the very least you should create the “placeholder” objects.

Example:

self.create_fixation()
sh = self.stim_size/2
diag45 = exp.ThickShapeStim(
    self.win,
    lineColor = self.stim_color,
    lineWidth = self.stim_width,
    fillColor = self.stim_color,
    closeShape = False,
    vertices = [[-sh, -sh], [sh, sh]]
    )
diag135 = exp.ThickShapeStim(
    self.win,
    lineColor = self.stim_color,
    lineWidth = self.stim_width,
    fillColor = self.stim_color,
    closeShape = False,
    vertices = [[-sh, sh], [sh, -sh]]
    )
corner = exp.ThickShapeStim(
    self.win,
    lineColor = self.stim_color,
    lineWidth = self.stim_width,
    fillColor = None,
    closeShape = False,
    vertices = [[-sh, sh], [-sh, -sh], [sh, -sh]]
    )

self.s = {
    'fix': self.fixation,
    'parts': exp.GroupStim(stimuli=diag45, name='parts'),
    'parts_odd': exp.GroupStim(stimuli=diag135, name='parts_odd'),
    'whole': exp.GroupStim(stimuli=[corner, diag45], name='whole'),  # arrow
    'whole_odd': exp.GroupStim(stimuli=[corner, diag135], name='whole_odd')  # triangle
    }

Create trial structure: create_trial()

Trials are composed of events (Event) that can be of are defined using the following parameters:

  • dur – duration of an event in seconds; could be a number or a list of numbers for each trial separately
  • display – which stimuli are shown during an event. This might not always be possible so you can also pass None and access stimuli later from self.s
  • func – function controling what to do with those stimuli. There are several predefined for you:
    • exp.Experiment.wait_until_response() which waits until response is produced and exits then
    • exp.Experiment.idle_event() which simply sits and waits until its time is up while catching key presses
    • exp.Experiment.feedback() for providing feedback after the trial with a fixation color change (correct response – green, incorrect - red)

Example:

self.trial = [exp.Event(self,  # parent of this event
                        dur=0.300,  # in seconds
                        display=self.s['fix'],
                        func=self.idle_event),
              exp.Event(self,
                        dur=float('inf'),  # this means present until response
                        display=None,  # we'll select which condition to
                                       # present during the runtime with
                                       # :func:`set_stimuli`
                        func=self.during_trial),
              exp.Event(self,
                        dur=.300,
                        display=self.s['fix'],
                        func=self.feedback)
             ]

Create an experiment plan: create_exp_plan()

This is the place to define properties of each trial. It is advisable to start by defining the total trial duration in the self.trial_dur variable (though if you don’t, set_TrialHandler() will do that for you later).

Each dictionary entry will be recorded in a separate column in the output file so think about good data sharing practices (White et al. (2013)):

  • One column - one value (i.e., a number, a string, or a boolean). No lists, dictionaries etc. You don’t really need them for stimulus construction during the runtime – instead, implement stimulus construction in the trial’s func function. Trust me, you can do it!
  • Dates are formatted as YYYY-MM-DD per ISO 8601 and XKCD’s Public service announcement
  • Avoid special characters and commas (as the output is a plain comma-separated file).
  • Blank values are supposed to be ‘’. No None, no NA or numpy.nan.

What should go into a data file? Your data file is supposed to describe completely what you did so that it was perfectly reproducible by somebody else. That should be your guiding principle.

Example:

exp_plan = []
for rep in range(self.nreps):  # repeat the defined number of times
    for cond, (context, posname) in self.paratable.items():
        pos = (cond - 1) % 4
        exp_plan.append(OrderedDict([
            ('rep', rep),
            ('cond', cond),
            ('context', context),
            ('posname', posname),
            ('pos', pos),
            ('onset', ''),
            ('dur', ''),
            ('corr_resp', pos),
            ('subj_resp', ''),
            ('accuracy', ''),
            ('rt', ''),
            ]))
self.exp_plan = exp_plan

Notice how we already define onset, dur, subj_resp, accuracy, and rt, even though currently they are empty. These values will be modified during the run time. You don’t have to list them here but it’s nicer to see all values that will be recorded.

Note

Unlike in PsychoPy, here we record data (responses, reaction times etc.) in self.exp_plan. Nothing is kept in self.data and it is not saved in the resulting data file. This is done in order to facilitate an easy import of data to pandas.DataFrame. You will appreciate the power of pandas in Statistics with stats.

Note

If you need to break your experiment into blocks, include a column that indicates a sequence number of that block. For example, the ‘rep’ column could be used to break the experiment into five intervals (of eight trials each). You need to define which column holds this information during the initialization by passing blockcol parameter to Task.

Setup: setup_task()

Setup can only bi initiated after the setup of the parent Experiment has been completed. Typical task setup procedure consists of:

  • creating stimuli (create_stimuli())
  • creating a trial structure (create_trial())
  • create a list of trials with each trial’s properties defined (create_exp_plan())
  • defining self.trial_dur if not defined yet
  • adjusting the list of trials for auto run
  • blocks during the experiment are inferred (get_blocks()) using the self.blockcol parameter (read more in Create an experiment plan: create_exp_plan())

All these steps are conveniently wrapped into a setup() and you should try to use it unless you need something special. This means that usually the setup function is not redefined.

Running task: run_task()

The experiment starts by setting up everything (setup()). Then, instructions are shown just like explained before in Run: run() using before_task(). A data output file is opened for writing (it’s a cvs format with a header).

Once this is ready, the experiment starts with the presentation of task instructions (self.instructions). Then the experimental loop is controlled by the run_blocks function which goes through each block in self.blocks (from get_blocks()). Each block runs through its own TrialHandler, and each trial goes through events. Key presses are recorded throughout using the post_trial() function (where the mapping between key input and self.computer.valid_responses is done, and accuracy and response time are computed). Data is then recorded immediatelly at the end of each trial in the datafile..

Committing and pushing data to a repository

Motivation

In the ideal world, how should data be treated? It should be registered at the point of its acquiry meaning that one should be able to go back and see it as it came out. Data files are not immutable. Sometimes there is a spelling mistake that you want to fix, or an extra column that you realized only two participants later you should have included. So you want to overwrite your data which poses a problem that it may go wrong but you notice it too late. This is where the confidence of having the original version help.

Of course, you could keep all versions of your data files but this is both inefficient (a year later, will you remember whether ‘data_corr’ or ‘data_final’ was the correct one?) and unncessary. It is much easier to track (and record) changes. Welcome revision control systems (learn more).

This approach provides a more stringent data and source code handling. Bonus: simple plot sharing as can be seen in the utl repo.

What’s available

register when you want to put a tag at some important point of your experiment development, for example, when you’re about to test the first participant or when you do a pilot, so that you can always go back to that point in time and see how your code looked exactly. ‘Registration’ is inspired by the Open Science Framework commit after data collection so that data files are added to the revision control system right away push to put your data on the remote repository immediately. This is recommended over commit unless you run experiments without the internet connection.

You can either add these flags when you run the experiment (except register), for example, so that data is pushed right away, or, if you forgot to do so intially, just run python run.py --push (or another flag) and the operation will be completed. (Note: for register and commit, a tag or a message has to be included.)

Other features

Autorun

You can simulate experiments by automatically running them. Just provide a value for autorun. I usually run them at 100x speed (thus, I enter 100). The experiment will run and record simulated responses by itself, stopping at each instructions screen and waiting for your response. If you rather have the experiment go no stop (good for making sure evertyhing works fine), select the unittest flag instead (it runs at 100x speed).

No output

Use no_output flag to run without creating or changing any output files. This is useful when debugging. For example, if the experiment fails before anything is shown, no (empty) data file is generated.

Debugging

debug flag is used to open the windon not in a fullscreen mode that comes by default. This is useful when you’re doing debugging since you want to be able to access the terminal.

Data analysis class

There is no generalized way to run an analysis, so psychopy_ext does not provide a class for that. However, it should be constructed in a similar manner to Experiment, for example:

class Analysis(object):
    def __init__(self, name='analysis', info={'subjid': 'confsup_'}):
        self.name = name
        self.info = info
        self.exp = exp.Experiment(info=self.info)

You can then define various analysis routines, for example:

def run(self):
    pattern = PATHS['data'] + '%s.csv'
    df = self.exp.get_behav_df(pattern=pattern)
    agg_acc = stats.accuracy(df, cols='context', values='accuracy', yerr='subjid')
    agg_rt = stats.aggregate(df[df.accuracy=='correct'], cols='context',
                             values='rt', yerr='subjid')

    plt = plot.Plot(ncols=2)
    plt.plot(agg_acc, kind='bar')
    plt.plot(agg_rt, kind='bar')
    plt.show()

You’ll notice that now we use stats and plot. Learn how to work with them: Statistics with stats.