Ryan Lab Group Meetings

Fall 2011 (2118)

Prior to start of semester

Wed, 8/24/11

Met with Levi Roan. Gave him the script and stimulus materials for Problem Format - Fall 2011. Gave him a tour of the subject pool website and the Ryan Lab group folder. He is able to access the folder. He is keeping a log. He did a great job boxing up the Interleaving studies to make room for new data in the lab.

He will set up the study on the subject pool once it is cleaned out from last semester. He will work with Jean to find class rooms to run large groups of subjects.

I will make a first set of copies of the materials for him.

Fall 2011 (2118)

First Week of Fifteen, Mon. 08-29-11

Now that the subject pool website is ready to go for Fall 2011, Levi Roan will post the experiment on Problem Formats and begin recruiting and running subjects. He could use another member to run subjects with him. Perhaps that will be Greg.

| back to top | next |

Fall 2011 (2118)

Second Week of Fifteen, Mon. 09-05-11

Thurs. 09-08-11

Met with Ted Brown. Got him started on "Things a New Volunteer Needs to Do". He will start literature searching on Attitudes towards math and abstract learning.

Fri. 09-09-11

Met with Sean Snoddy. He will check with Steve Craig and Kaitlyn Rhyner to see what their availability is to continue the work on CP4. He will also see when we can all meet and get back to me. I need to see the data they collected so far so we can decide how to revise the instructions. Then they can run more subjects. I have a paper (a recent book chapter by Rittle-Johnson and Star on problem comparison) requested from inter-library loan. I'll distribute it to all of them when I get it.

| previous | back to top | next |

Fall 2011 (2118)

Third Week of Fifteen, Mon. 09-12-11

| previous | back to top | next |

Fall 2011 (2118)

Fourth Week of Fifteen, Mon. 09-19-11

Mon 09/19/11

Met with Greg Dreibelbis. Because he is a junior, I'll have him work on the CP 4 project. I gave him a paper copy of Chi et al. 1994 (the circulatory system study), and emailed him Ryan 2005 (Problem comparison via similarity judgment) and the manuscript submitted to the British Journal of Ed. Psych (problem comparison via mechanical feature matching).

Tues. 09/20/11

Met with Ted Brown. He brought in some examples of studies he has found on attitudes towards statistics. We looked as two of them. They were about validating attitude measures. We were not able to pull up the others because the info he brought in was article titles, but no authors, or the specific search terms he used to pull up the articles. He will go back, find the articles again, and get the full reference information and .pdf's if he can. I showed him around Zotero and told him about LibreOffice.

We also emailed Rick Miller to add Ted to the Ryan Lab group folder.

Wed. 09/21/11

Met with Sean Snoddy, Greg Dreibelbis, and Katlin Rhyner regarding CP4. Here is the description of CP4:
===============================================================
Conditions: MF - no SE vs MF with SE

Procedure:

Pretest (4 problems)
Training (Worked examples, unguided practice)
Posttest (4 problems)

Pre-test and Posttest problems:
(Learning) Sales-Final
(Learning) Mixture-Final
(Transfer just across surface features) Distance - Final
(Transfer across both surface features and procedure) Distance - Initial

Worked example problems:
For MF - no SE - two pairs
Each pair: Sales-Final and Mixture-Final
For MF with SE - just one pair (but with SE)
Sales-Final and Mixture-Final

They will be trained to SE in the Worked Examples
They will do their own spontaneously generated SE's in the Unguided Practice
===============================================================

The main goal is to get a stronger training and transfer effect from the MF with SE than the MF - no SE

But, we also want to use their self-explanations to see what they are getting from the Matching Features.

A problem has been that they produce uninformative SEs. So in this meeting we worked on how to re-write the SE prompts and training to get better SEs. It may be the case that even if we get better SEs, we might get SEs that are very similar to what they heard in training, which would call into question the extent to which they were truly self-generated. However, if we can get reasonably good SEs during the unguided practice, at least they will be referring to new problems, and so they will have to generate references to those problems.

The general template for the SEs will be something like this:
================================================================
Prompt: Why does the $300 in the first problem go with the 2 shots in the second problem?
Example of SE: Even thought the $300 is an amount of money and the 2 shots is an amount of a substance, they are both the first initial amounts in the problems.
==================================================================

| previous | back to top | next |

Fall 2011 (2118)

Fifth Week of Fifteen, Mon. 09-26-11

Mon., 09/26/11

Met with Ted Brown. We tried to create a shared folder in Zotero. So far we got both of us to be able to access the folder. But all of the articles are in his root account and we have yet to figure out how to move them to the shared folder. Ted will work on getting the papers to me somehow.

I put Ted on sona - systems as a researcher for Levi's study. I put the two of them in touch.

Fri., 09/30/11

Met with the CP4 crew - Sean Snoddy, Greg Dreibelbis, and Katlin Rhyner. They had re-written the SE prompts. We worked on them a little more and made hand-written notes on our copies of the materials. They will finish re-writing them, and finish re-writing a complete script. Then they will begin running subjects.

| previous | back to top | next |

Fall 2011 (2118)

Sixth Week of Fifteen, Mon. 10-03-11

| previous | back to top | next |

Fall 2011 (2118)

Seventh Week of Fifteen, Mon. 10-10-11

| previous | back to top | next |

Fall 2011 (2118)

Eighth Week of Fifteen, Mon. 10-17-11

Thurs. 10/20/11

Met with Ted Brown.

Deci, Nezdek, and Sheiman (1981) found a relationship between teacher's being more supportive of autonomy and their students' intrinsic motivation. The study was correlational. Ted will rewrite to reflect that the article was on a correlational study.

Bandalos, Finney, and Geske (2003) found relationships between learning goals and achievment (through study strategies). There was a strange finding that deep processing strategies were negatively related to achievement. However, the achievement was onlymeasured in the short term. So they hypothesized that the deep processing strategies may have been inneffective for students because of their beginning state of knowledge, and the deep processing may have actually led to greater long term retention. Ted will summarize the discussion of that article.

I'll see if we could meet sometime with Dr. Baranczyk to see how we're coming along on a potential study.

| previous | back to top | next |

Fall 2011 (2118)

Ninth Week of Fifteen, Mon. 10-24-11

Fri. 10/28/11

Met with Ted Brown. We have solved the problem of getting all of the articles into the shared folder.

We determined that Hilton, Schau, and Olsen (2004) found that the SATS can be used to assess attitudes towards stats in a pre-post design. We also found the website for the SATS, and found that we can request it for our research. Apparently there is no cost, we just have to get permission and share our findings.

The Bandalos study emphasized that in teaching stats, it may be necessary to let students spend time working procedures with little understanding at first, because expecting them to do "deep processing" seems to be harmful when they are still very unfamiliar with these very foreign concepts. We also did a forward search on the Bandalos article and found 26 citations. Ted will look through those to see if there are any that are relevant to our ideas of doing a motivational manipulation in stats class.

I haven't yet contacted Dr. Baranczyk. I'll get that done.

| previous | back to top | next |

Fall 2011 (2118)

Tenth Week of Fifteen, Mon. 10-31-11

Fri., 11/4/11

Met with Michele and Ted. Discussed narrowing down to a study on applying Value/Expectancy theory to Statistics. Ted has not found Goal Setting theory applied to Statistics in the lit. So maybe that could be next.

The study would be a quasi design - using 4 of our 8 sections of stats as one group, 4 as the other. The two conditions would be Raising Value and Expectancy vs. Not Raising Value and Expectancy. The design would be a pretest of attitudes SATS and performance (average of first two tests), followed by the manipulation, followed by post tests of attitudes and performance. Maybe we should throw in a measure of value as a manipulation check.

The manipulation would consist of once a week taking 3 to 5 minutes of class time to show a .ppt presentation. For the experimental group, these would be examples of how statistics is valuable in everday life and in psychology related careers. For the contol group, the presentations would be of some kind of historical/enrichment material. For example, you could tell the story of how William Gossett developed the t test for the Guiness brewery, or how a normal curve can be approximated with a Pascal's triangle.

We would predict that there would be greater positive changes in both attitudes and performance among the experimental groups than among the control groups.

Concerns:

Should we manipulate goal setting insttead of value. Michele already has some goal setting results to show from her General psych classes.
Shouldn't we include an expectancy raising manipulation with the value raising manipulation. For example, for the Experimental group, should the presentations all be value raising, or should half of them be little practice exercises at just the right level so that students would likely do well, but surprise themselves with their good performance. What would the equivalent control activity be?
Do we risk inducing demand characteristics by giving them the same SATS before and after with presentations that are obviously intended to persuade them of the value of statistics in between?

Our goal would be to present this at APA in August 2 - 5 2012 in Orlando.

Deadline to submit is Dec 1.
300 word summary for each presenter
Michele would present her goal setting stuff from her General classes
I would present my value and expectancy raising stuff from my Stats classes
Michele would get Christina? (a colleague she is working with) to be the discussant
I might have to join APA - Prof. Dev. money and Dept. money could be used for travel and maybe for the APA dues

Michele will try to get Christina and do the submission to APA
Ted will look for examples of how statistics is valuable - APA and publisher's websits on teaching resources, Amazon books (does Malcolm Gladwell have a book on spurious correlations and their real causes?)
I'll work on the IRB application and getting the SATS
I'll see if Taylor Tolton or some other new research assistant can join Ted and help him

We'll meet again next week. Friday at 10am.

| previous | back to top | next |

Fall 2011 (2118)

Eleventh Week of Fifteen, Mon. 11-07-11

Fri.. 11/11/11

Met just with Michele regarding our motivation study. Ted is sick. He'll email what he has found so far.

Michele reports Christina Wilson (fellow grad student of Michele's - ABD at Colorado State) is willing to participate in the symposium by being the discussant.

Regarding how to avoid demand characteristics: We will ask the stat profs what other things they might like to measure. We have that whole list of measures for the Exp classes to pull from. We will embed the SATS in among those other measures. We could use the longer version (about 35 items) split the SATS in half, and embed each half in among some other measures. One half could be among one set, the other half among another set. That way, the subjects would see a completely different questionnaire before and after the manipulation. We would counterbalance which half was used for pre vs. post across subjects.

Regarding including a way to raise expectancy as well as value: I'll contact Greg because he seems to already try to do that. I'll ask him for suggestions as to some procedures to strengthen our manipulation. Also, as explained below, I'll have to ask him if, and how, he would be willing to be involved, since we would be asking him to help us manipulate something that he already does.

I'll contact the stat profs and ask for their participation in the study. I'll see what way they would be willing to do it. Would they want us to randomly assign one of their two sections to each condition (block on prof), or would they prefer to run both of their sections as the same condition. If we include the expectancy manipulation, and if it is like what Greg already does, and if we decide to randomly assign one of each two sections to each condition, then I'll have to ask Greg if he would be willing to NOT do it for one of his classes.

We may not be able to submit this for a symposium this year. We might just get it started this Spring and submit it as a symposium next year. We might just get a poster this year.

I'll contact Taylor Tolton to see if she is coming on board.

Had lunch with Greg. He would be willing to be involved, but would not want to actually avoid doing his usual confidence building with one of his classes. However, we discussed taking whatever he does to build confidence, and using those ideas to construct some short activities that could be used in the experimental condition so that he would be doing them in addition to what he usually does. That way, he could have one of his classes in the control condition and one in the experimental condition, and the other profs could do the same. That would unconfound prof with condition, but would be something Greg would be willing to do.

Greg also said that he has a measure of statistical self-efficacy that he can look up and give us.

| previous | back to top | next |

Fall 2011 (2118)

Twelfth Week of Fifteen, Mon. 11-14-11

| previous | back to top | next |

Fall 2011 (2118)

Thirteenth Week of Fifteen, Mon. 11-21-11

| previous | back to top | next |

Fall 2011 (2118)

Fourteenth Week of Fifteen, Mon. 11-28-11

| previous | back to top |

Fall 2011 (2118)

Fifteenth Week of Fifteen, Mon. 12-05-11

Fri. 12/09/11

Ted was not able to make the meeting. Taylor summarized Marshall, M. D. (2004). Expectations and Realizations: The Role of Expectancies in Achievement Settings. Motivation & Emotion, 28(4), 347-361.

Experiment 1 - Correlational in the sense that the expectancy factor was only observed, not manipulated

Dependent - Performance on the target test: 10 Remote Associates Test (RAT) items
Independents:
     Expectancy: Their self-report rating of how well they thought they would do on the target test after taking a 3 RAT item pre-test. The ratings were used to categorize subjects as either low, med, or high expectancy.
     Difficulty: Easy vs Difficult target test

Results: For Easy target test - Relatively high performance (~7), no difference in performance as a function of Expectancy
              For Diff target test - Relatively lower performance ( ~3 - 4), low expectancy lowest performance compared to med and high. No diff between med and high

Experiment 2 - Attempted to manipulate expectancy

Dependent - Performance on the target test: 10 Remote Associates Test (RAT) items
Independents:
     Expectancy: Their self-report rating of how well they thought they would do on the target test, but this time, after taking a 10 item RAT pretest that was either hard (low expectancy), moderately difficult (med expectancy), or easy (high expectancy)
     Difficulty: Easy vs Difficult target test

Manipulation check: Number of items correct on the pretest was as expected. The hardest pretest produced the lowest scores, the moderately difficult pretest produced medium scores, and the easiest pretest produced the highest scores. However, self-reported ratings of expectancy did not follow that pattern. The expectancies followed the scores for the easiest and moderately difficult pretests, that is, highest expectancy and moderate expectancy respectively. But for the hardest pretest, although the scores were the lowest, the expectancy was about equal to that for the moderately difficult test, that is, it was also moderate.

Results: Same pattern as in Experiment 1.

The authors also did an analysis of partial data to examine what would have happened if the manipulation check had come out as expected. They selected those participants whose expectancy scores were comensurate with their performance on the pretest (they confirmed that the expectancy scores were significantly different ). Then they examined performance on the target test for just those subjects. They found that the same pattern of results with larger effect sizes, both in the data and as Cohen's d's.