Ryan Lab Group Meetings

Spring 2013 (2132)

Prior to start of semester

Fri., 12/28/12

The independent study (CP 4.5) several possible conditions to use SE on MF, each to be compared to a control condition that is basically similar to CP4. One of these will be used in CP5

All of the conditions involve counterbalancing not only the Pre/Post, but also the Test/Training as shown below:

Counterbalance 1
Pre and Post test problems are: Discount-Final          Mixture-Final             Travel-Final            Travel-Initial
        The training pairs are:                                                 Discount-Final/Mixture-Final

Counterbalance 2
Pre and Post test problems are: Discount-Final          Travel-Final             Mixture-Final            Mixture-Initial
        The training pairs are:                                                 Discount-Final/Travel-Final

The first condition (the control condition) is adapted from CP4 and is the most similar to it. Call it the MFplusSE condition

There are 6 pairs of training problems.

In all of the pairs, the first problem is presented as steps one through five of arithmetic on one page, and the second is presented as steps one through five on the second page. The steps are presented flush left. On the right side of the page there are boxes with instructions for performing the steps. (All of this will be changed in the second condition).
In the first pair (A), the training problems are presented as worked examples with written instructions for the steps. Next is written problem comparison instructions. Next is a set of SE prompts WITH model responses to the SE prompts (the model responses are very similar to the instructions given with the worked examples).

In the next two pairs (B and C) everything is the same as in A except that there are no problem comparison instructions, and there are SE prompts, but there are no model responses. Instead the subject is asked to generate their own responses.

In the last three pairs (D, E, and F) everything is the same as in B and C, except that now the working of the examples is dropped out. So, the subject is required to work the examples and to provide their own model responses.

The second condition would be the MFplusSEplusBACKWARDS FADING

There are more changes between the first and second conditions than just the adding in of the backwards fading. The pairs are presented in a different manner. The first step of arithmetic for both problems is presented first. Then the next steps are presented. But for each step that is presented, those steps are presented for both problems. The steps are centered in the page, with the arithmetic for problem one above that for problem two. There are no boxes with instructions for performing the steps. Instead, there is one line of instructions right before the centered steps of arithmetic. Nowhere are there any problem comparison instructions. Nowhere are there any prompts for SE's either with or without model responses.

The first pair of problems has all of the steps presented. There is nothing faded out. In the second pair, the last step is faded out. In each of the following pairs one additional step is faded out, working backwards through the steps, until, on the last pair, all of the steps have been faded out.

The fading out is done by presenting blank lines where the one line of instructions before the steps of arithmetic had been presented. Then below that, there are just labels to indicate that the steps of arithmetic are to be performed there.

Thus, the presentation of the blank lines serves as a prompt for the subject to fill in either just what they remember as the instructions for that step, or something else. The something else could be a self-explanation of the step. One of the dependent variables could be what kind of response the subjects gives. Also, the other dependent variables, such as the major one, the pre/post gain, could be conditioned on what kind of response the subject gave.

We also talked about the possibility of using the pretest scores as a filter for a separate analysis of subjects who do poorly versus those that do well on the pretest. Sean will try taking a look at that kind of conditional analysis of the CP 4 data.

This all means that there is some question as to whether the name of this condition, as it stands, should include the MF or the SE, although it definitely includes the backwards fading.

Sean's rationale: CP4 showed no difference in pre/post gains between the MFonly condition and the MFplusSE condition. Instead, higher posttest scores were related to higher pretest scores and to higher math aptitude scores. Therefore, Sean is not concerned at this point that the second condition differs from the first in more than one way. At this point, the idea is just to try a "everything but the kitchen sink" manipulation in order to try to increase the pre/post gains. Then, if those gains can be increased, in subsequent experiments more targeted manipulations can be made in order to tease out what particular change resulted in the improved gains.

Sean has done a tremendous amount of work on this and has obviously put a lot of thought into it. I want to encourage him so that he maintains his enthusiasm and does not burn out.

The following projects will have to wait for a subsequent meeting.

CP5

DRM

my Feature Matching paper for Cog Tech

We will meet again on Monday, 01/21/13, at 9am in my office.

Mon., 01/21/13

Met with Sean. Regarding CP4.5. I checked his correction on the materials for CP4.5.
So at the beginning of the Spring- For CP4.5 Sean needs to finish some corrections to the materials, get the IRB approved, get it up on sona. Then Sean will run the subjects. He will be running multiple subjects at a time.

CP 5 - Still working on the literature search. It could be a study designed to see why CP4.5 worked, if it did. If it does not, then CP 5 could be a multiple condition study based on the lit. search.

DRM - Need photocopies of materials. Put time slots on sona and start to run. He will try Amy first

Rewrite of Cog Tech paper - I sent the demographic data with the performance data included to Sean. I'll start working on the reviews as well. I looked at the demographic data and found a significant advantage of the MF's over the other two conditions for GPA. I also found a significant disadvantage of the Explain Steps compared to the other two for verbal ability (VSAT, vocab? I'll have to check again). At any rate, I think I already did the overall performance analyses that I am reporting in the manuscript trying the Vocab scores and the MSAT and VSAT as covariates and found that they did not make enough of a difference to change the results. But I'll have to check that, and I'll have to check the effect of using the GPA's as a covariate. I don't think I did that before (although maybe so - I can check through the old analyses to see).

Sean's future plans - Will not defer graduation. If no acceptances, then some post BA courses. Keep doing research. Also, will look for a job just for money.

Next meeting - Will check his schedule and get back to me.

Spring 2013 (2132)

First Week of Fifteen, Mon. 01-28-13

Thurs. 01/31/13

Met with Sean. He knows of a Fall 2012 KU grad Rachael Lerch who wants to get experience CP5 and Sean's independent study. He will bring her to meet me tomorrow.

Has about 3/4 of the IRB for the independent study (CP 4.5 - two conditions - eight versions with conterbalancing - one manipulation with everything but the kitchen sink in it) done. Corrections from last version of materials are made.

DRM - (H vs L is between - the counterbalancing is between - S vs W - within)
             (So there are 12 between cells, with a plan of 10 SSs per cell and 2 observations per subject)

                                    A        B       C          D             E             F
High Authority     Strong/
                              Weak

Low Authority

Will sent me .pdf's of all the materials and a list of how many copies you need. Just need copies and time slots and we can begin running. From me he will need one hour sessions - six sessions to start - recruiting 10 subjects per session. Each session will be the High Authority. There will be Strong suggestions and No suggestions within the subjects. The six sessions will be needed to cover the counterbalancing. If the balance is off, then more sessions will be run to fill in.

We will meet on a Saturday in a few weeks to work on the Cog Tech paper (after Sean gets his job commitment out of the way). He has already read the ms twice and the reviews several times, so he is already ahead of me on this project.

He will email the CP5 people to get them back on the lit search.

| back to top | next |

Spring 2013 (2132)

Second Week of Fifteen, Mon. 02-04-13

| previous | back to top | next |

Spring 2013 (2132)

Third Week of Fifteen, Mon. 02-11-13

Thurs. 02/14/13
Met with Sean and Rachel. Looked at the independent study materials (CP 4.5 - Backwards fading vs no backwards fading and a different method of presenting the materials). The materials are almost ready to go. Sean will make a couple of small changes - mostly adding a few instructions to be sure subjects in the no fading condition understand what to do. Also, the IRB for that is almost ready to go. It needs the informed consent and the debriefing.

Sean will try to get the DRM study data gathered, and analyzed in time to submit it to the Psi Chi symposium this year.

Also, Sean will send me info on how to send a letter of rec. for a Who's Who, and I will be hearing from Dr. Meehan regarding nominating Sean for the Chambliss award.

Sat. 02/16/13

Met with Dave Kile regarding the Statistics Motivation study from Spring 2012. After some digging around, I found the files for the study.
They were at "/home/bob/Documents/Current/Research/Ideas for 2118 (Fall 2011)/collaboration with Michele Baranczyk on goal setting study"

Dave had organized all of the data into the most usable, and the less usable. The most usable is data from subjects in which we have both a pretest and a posttest at the very least. The section number should tell us which data is in the experimental condition and which was in the control. That is, once Dave finds any subject in a given section who has enough data between the pretest and the posttest to identify that subject as being in the experimental or control condition, then that whole section of Statistics was in that condition.

We started an Excel spreadsheet into which Dave will enter the pre and post data. Dave will print out a copy of the tests to use as a key. The first set of items can be just numbered. They are simple rating items about attitudes towards Statistics with some items having the rating scale flipped. In the key, Dave can just number those items and indicate which ones are flipped. Then, in the Excel sheet for the raw data, he can just enter the rating for each numbered item. He will flip the ratings for the flipped items before entering them so that they all mean higher numbers are a more positive attitude towards statistics. Thus, we can later use Excel to calculate a sum or average for all those items.

Other parts of the pre and post tests have other kinds of items, such as attitudes towards math in general, and demographic info. Dave will figure out how best to name the variables to code those items and put the names in the key.

Dave will have Dan help him on coding the data. He will let me know if he would like some more manpower and I'll try to get it for him.

Later, we will have to figure out how to deal with the other data provided by the subjects (their responses to the value(experimental)/historical(control) vignettes and their responses to the expectancy(experimental)/practice(control) math items. And we will have to obtain their course grades from their professors, as approved by the IRB.

| previous | back to top | next |

Spring 2013 (2132)

Fourth Week of Fifteen, Mon. 02-18-13

| previous | back to top | next |

Spring 2013 (2132)

Fifth Week of Fifteen, Mon. 02-25-13

| previous | back to top | next |

Spring 2013 (2132)

Sixth Week of Fifteen, Mon. 03-04-13

| previous | back to top | next |

Spring 2013 (2132)

Seventh Week of Fifteen, Mon. 03-11-13

| previous | back to top | next |

Spring 2013 (2132)

Eighth Week of Fifteen, Mon. 03-18-13

| previous | back to top | next |

Spring 2013 (2132)

Ninth Week of Fifteen, Mon. 03-25-13

Thurs. 03/28/13

Met with Sean. He has done some more editing CP4.5. Still need to finish the debriefing. Then can submit the IRB and start running subjects. Then we talked about his philosophy paper. He gave me an update on where he is with revising the Cog Tech paper. He will see where his is over the weekend and then let me know when he can meet regarding that paper. He wants to have a decent amount done before he sets up the meeting.

| previous | back to top | next |

Spring 2013 (2132)

Tenth Week of Fifteen, Mon. 04-01-13

Fri. 04/05/13

Met with Sean. Checked our progress on the Cog Tech paper. So far so good. We swapped versions so that we each had both, but did not change anything yet.

He also reported on the number of subjects run in the DRM study. Only about half way to what we planned. Some imbalance in the counterbalancing. So he will finish coding all the data collected so far, analyze it, and decide whether or not to proceed.

For CP4.5, we made one final little edit (in the debriefing we named the conditions "Self-explaining the matching of features with/without backwards fading)". I signed the IRB application. He will turn in the IRB application next.

| previous | back to top | next |

Spring 2013 (2132)

Eleventh Week of Fifteen, Mon. 04-08-13

| previous | back to top | next |

Spring 2013 (2132)

Twelfth Week of Fifteen, Mon. 04-15-13

| previous | back to top | next |

Spring 2013 (2132)

Thirteenth Week of Fifteen, Mon. 04-22-13

Fri. 04/26/13

Met with Sean re- the Cog Tech paper. He has added some items, and will add more, so we can meet next Friday. He will also work on the response to reviewers. I gave him the response to reviewers as it exists so far. I also gave him a paper to cite for "how to make matching features into a meaningful instructional method". Sean suggests - give them a grade (from his experience with the motivation or lack thereof of students he has run as subjects).

The paper I gave him to cite was Liu, Bridgeman, & Adler 2012
It is among my files at /home/bob/Documents/Current/Research/Papers/My Papers/for paper on comparing lab and web studies/paper/Liu, Bridgeman, & Adler 2012 Motivation affects perf on assessment.doc

| previous | back to top | next |

Spring 2013 (2132)

Fourteenth Week of Fifteen, Mon. 04-29-13

| previous | back to top |

Spring 2013 (2132)

Fifteenth Week of Fifteen, Mon. 05-06-13

Spring 2013 (2132)

After the semester ended

Mon., 05/20/13

Met with Sean. We compared my latest version of the Cog Tech paper and his. There was nothing in Sean's version that was not in mine, although there were some changes in mine that were not in his. I made sure that Sean saw all of those changes. We also worked on editing his changes a little bit. The most recent version is now "R. S. Ryan - Feature Matching - Revision 1 - combined 2013_05_20.doc". I will continue working on that version. The changes that need to be made from here on are basically mechanical. Then I will send my latest version to him for review.