Designing Experiments Using a Scientific Method

A scientific method is a plan that is followed in performing a scientific experiment and writing up the results. It is not a set of instructions for just one experiment, nor was it designed by just one person. The scientific method has evolved over time after many scientists performed experiments and wanted to communicate their results to other scientists. The scientific method allows experiments to be duplicated and results to be communicated uniformly.

Methods & techniques

Research process

Before you start to define your experiment

We would like you to try to answer a research question on the usability, applicability, or effectiveness of VR/AR in a built VR or AR environment. You will make the environment with your team.

1: Explore, think about your personal motivations and your team's ambitions.

• Think about movies: do you remember any kind of magic device that appealed to you?

• Think about personal experiences: travels or people that felt like magic.

• What kind of human interaction would you like to explore: hands, feet, head, fingers, body?

• What kind of devices do you have and how could you explore those interactions? See Gabbard's taxonomy of usability characteristics below:

Taxonomy of usability characteristics, Joseph Gabbard, 1997

Defining your experiment

2: Form a hypothesis

When preparing to do research, a scientist must form a hypothesis, which is an educated guess about a particular problem or idea, and then work to support it and prove that it is correct, or refute it and prove that it is wrong.

3: Define your research question

Main objective for research: what do I want to find out?
Main research question
Subquestions

4: Think through your research method

- What method will I use for testing usability, applicability 
  and effectiveness? 
- What is the goal of this method?
- What are benefits of using this method? Any possible counter effects?

5: Have a real good look at your evaluation metrics

• Independent and Dependent Variables A well-designed experiment needs to have an independent variable and a dependent variable. The independent variable is what the scientist manipulates in the experiment. The dependent variable changes based on how the independent variable is manipulated. Therefore, the dependent variable provides the data for the experiment.

• Validity and Reliability

Validity

Validity implies the extent to which the research instrument measures, what it is intended to measure. What is the accuracy of the measurement. A valid instrument is always reliable.

Reliability

Reliability refers to the degree to which scale produces consistent results, when repeated measurements are made. Is the experiment executed in a precise manner and is it consistent with former experiments. A reliable instrument need not be a valid instrument.

Additional info about Validity and Reliability