Ryan Lab Group Meetings

Spring 2022 (2222)

Prior to start of semester

Spring 2022 (2222)

First Week of Fifteen, Mon. 01-24-22

| back to top | next |

Spring 2022 (2222)

Second Week of Fifteen, Mon. 01-31-22

| previous | back to top | next |

Spring 2022 (2222)

Third Week of Fifteen, Mon. 02-07-22

| previous | back to top | next |

Spring 2022 (2222)

Fourth Week of Fifteen, Mon. 02-14-22

| previous | back to top | next |

Spring 2022 (2222)

Fifth Week of Fifteen, Mon. 02-21-22

| previous | back to top | next |

Spring 2022 (2222)

Sixth Week of Fifteen, Mon. 02-28-22

| previous | back to top | next |

Spring 2022 (2222)

Seventh Week of Fifteen, Mon. 03-07-22

| previous | back to top | next |

Spring 2022 (2222)

Eighth Week of Fifteen, Mon. 03-14-22 (spring break)

| previous | back to top | next |

Spring 2022 (2222)

Ninth Week of Fifteen, Mon. 03-21-22

| previous | back to top | next |

Spring 2022 (2222)

Tenth Week of Fifteen, Mon. 03-28-22

Friday, 04/01/22 - 10am - OM 385

Met with and Gabby (Gabby Maramag), and Ryann Thomson both new Spring 2022 - not here yet - Nicky (Nicayris Ulerio-Fernandez)

Problems to deal with for a followup experiment to StatsSE:
How to run enough subjects
How to make it more in depth material
How to get cooperation of faculty in different classes

Would try to get it as a registered report (get the pre-registration accepted for conditional acceptance)

One idea would be to try to get faculty from other PASSHE schools to participate
It could be a checkerboard design in order to do it in the classroom and with important material. That design has the advantage that one module could replicate our materials from last time. But is we can't get the participation of lots of faculty, then we end up running subjects for a long time in order to get a large enough sample. Also, the retention test has to be given in the Experimental course to those students who participated in the first part, and are taking Experimental the semester after Stats.

The COS, back in 2016 tried to interest Applied Cognitive Psychology, Teaching of Psychology, and Instructional Science in registered reports. I don't think TOP does them yet. The other journals need to be checked on the COS website.

Gabby will email Ryann the KLI paper and the StatsSE
Gabby suggested Matched Pairs and using a video to present the instructions to the subject.

Gabby and Ryann will work on creating several possible designs and we will meet again next Friday, 4/8/22, same time, same place

| previous | back to top | next |

Spring 2022 (2222)

Eleventh Week of Fifteen, Mon. 04-04-22

Fri. 04/08/22 - 10:30 - OM 385

Met with Gabby and Ryann

Will start a project on the OSF and make the reseach assistants contributors.

Ryann suggests manipulating Practicing Retrieval along with manipulating Testing
Gabby suggests if we use matched pairs, then we can use a control group in which the condition is "use your own methods". But that would risk carry over effects. But we still get the benefit of controlling for things like aptitude and motivation. AND we could use two different classes each of which is in a different university. So we have to have a structured activity as control - it should be "like" what students usually do.

We also discussed the possibility of doing retention in Experimental. But it doesn't have to be in Experimental class, it just has to be the next semester. And we could argue for a higher amount than $15.

Next time we will nail down the exact to-be-learned material.

Teaching Stats Fall 2022 Stoffey (2) Sunsay (2) Ryan (1)

So next we nail down the to-be-learned material and more about the method (D2L?)

| previous | back to top | next |

Spring 2022 (2222)

Twelfth Week of Fifteen, Mon. 04-11-22

Fri. 04/15/22 - 10:30am - OM 385

	Experimental	Control
	Pretest - Normal Dist and z scores	Pretest - Normal Dist and z scores

	Read Lesson	Read Lesson
	Study by Retrieving	Study by re-reading
	Study by Retrieving	Study by re-reading
	Study by Retrieving	Study by re-reading

	Posttest - Normal Dist and z scores	Posttest - Normal Dist and z scores

So, the Research Assts. will come up with the lesson and two equivalent tests. The tests will consist of 4 scenarios with 3 MC questions about each.

NEXT MEETING - With Neal and Nicky -Thurs. 4/21/22 at 11:30 am in OM 385

NEXT MEETING - With Gabby and Ryann - Fri 4/22/22 at 10:30am in OM 385

| previous | back to top | next |

Spring 2022 (2222)

Thirteenth Week of Fifteen, Mon. 04-18-22

	Experimental	Control
	Pretest - Normal Dist and z scores	Pretest - Normal Dist and z scores

	Read Lesson	Read Lesson
	Study by Retrieving (followed by seeing the lesson again as feedback?)	Study by re-reading
	Study by Retrieving	Study by re-reading
	Study by Retrieving	Study by re-reading

	Posttest - Normal Dist and z scores	Posttest - Normal Dist and z scores

If we put the lesson on D2L, how do we implement the retrieving? How do we know they weren't looking at the lesson while retrieving? How do we know the control subjects are doing the re-reading. How about if the lesson presents an example of a normally distributed population, giving the mean and standard deviation. Then there are a series of questions about z scores in that population along with the answers. Then for the practice sessions, the control subjects would see similar lessons with questions and answers, but just with different numbers. But for the experimental subjects, in their practice sessions they would see the same lessons that the control subjects see, but they have to "fill in the blank" to answer the questions. Maybe the questions could be conceptual, not computational.

Example lesson:

There is a normally distributed population with a mean of 50 and a standard deviation of 10. A z-score is a way of measuring how far an individual score is above or below the mean. The distance from the mean uses the standard deviation as a unit of measure. So in this population, a score of 60, is 10 points, that is one standard deviation, above the mean. So its z-score is +1. A score of 30 is 20 points, that is, two standard deviations, below the mean. So its z-score is -2.

Using z-scores enables one to have a "standard" way of expressing how high or low an individual score is in a normally distributed population. For example, if two individual scores, each from a different normally distributed population have a z-score of +2, they can be considered equally "high" scores, even if the distributions have different means and standard deviations. Suppose there is an individual score of 40 in a normally distributed population that has a mean of 30 and a standard deviation of 5. Then suppose there is a score of 74 in a normally distributed population with a mean of 70 and a standard deviation of 2. The score of 74 is not really "higher" than the score of 40, because both are two standard deviations above the mean in their own populations. They both have a z-score of +2.

Here's another example. Suppose a student gets a score of 86 points on a math test that has a mean of 80 and a standard deviation of 3. The same student gets a score of 90 points on a history test with a mean of 75 and a standard deviation of 15. In terms of z-scores (standardized scores) that student did better on the math test than on the history test because the 86 on the math test has a z-score of +2, but the 90 on the history test only has a z-score of + 1.

Example first practice:

There is a normally distributed population with a mean of 50 and a standard deviation of 10. A z-score is a way of ___________________________________________________. The distance from the mean uses _________________________________. So in this population, a score of 40, is 10 points, that is ___ standard deviation(s), ______ the mean. So its z-score is -1. A score of 70 is 20 points, that is, ___ standard deviation(s), _____ the mean. So its z-score is +2.

Using z-scores enables one to have a "_______" way of expressing _____________ an individual score is in a normally distributed population. For example, if two individual scores, each from a different normally distributed population have a z-score of +2, they can be considered _______ "high" scores, even if the distributions have different ______________________. Suppose there is an individual score of 400 in a normally distributed population that has a mean of 300 and a standard deviation of 50. Then suppose there is a score of 740 in a normally distributed population with a mean of 700 and a standard deviation of 20. The score of 740 is ____________ than the score of 400, because both are two standard deviations above the mean in their own populations. They both have a z-score of +2.

Here's another example. Suppose a student gets a score of 172 points on a math test that has a mean of 160 and a standard deviation of 6. The same student gets a score of 180 points on a history test with a mean of 150 and a standard deviation of 30. In terms of z-scores (standardized scores) that student did ______ on the math test than on the history test because _____________________________________________________________________.

Thurs., 04/21/22 at 11:30 am in OM 385

Met with Nicky and Neal.
Thought of the idea of using the Stat lab to run multiple subjects at a time, but proctored. Maybe touch base with stats profs to elicit their co-operation with proctoring the sessions in their classes.

Nicky will look into other schools that use D2L and even possibly non-PASSHE that use other comparable LMS's.
I'll contact the stat profs here to see what they think. We'll come up with a task for Neal probably next meeting.

We will meet next week, same time, same place

Fri., 04/22/22 at 10:30 am on OM 385

Met with Gabby and Nicky.

Prior to the meeting, Gabby sent a draft of an example lesson on Statistical Significance.
Also, Ryan sent a draft of a draft example lesson on normal dist. and z-scores

For ethical reasons - are these two topics (z-scores; stat sig) of equal difficulty?

Regarding equivalent tasks for the two groups if there is feedback for the Exp group:
In each Study session (after the first one in which both groups just read the same initial lesson) these are the tasks

Experimental:

Read lesson with blanks to be filled in. Then get feedback consisting of the lesson filled in, but they have to check their work.
Read lesson with no blanks. Then read the same lesson again, but with errors (but only spelling, grammar, and such proof-reading lessons) to be corrected.

I will work on editing Ryann's example - at least the first two or three paragraphs. I can look at the MC questions, but we won't be able to finalize the tests until we finalize the lessons.

I made Ryann and Gabby Guest Instructors on my Sandbox course.
Ryann and Gabby will work on how to get D2L to do what we need.

Next week we'll meet on Fri. the 04/29/22 at 11:45 in OM 385.

Email sent to statistics faculty at Kutztown, Alvernia, and Albright
===================================================
Subject: Help with experiment on retention of Statistics
From: Bob Ryan <cogprofessor@gmail.com>

4:08 PM 4/22/22

to (Kutztown faculty): Ron, Jay, Cara, Michele, Derek, Gregory, Jason, Steve, Meghan, (Alvernia faculty): scott.ballantyne, john.gieringer, matthew.prudente, john.rochowicz, david.shoup, soo.sun, (Albright faculty): bhearon, gseidman, hlee, jheberle, jcouchman, kfeigenson, shughes, (cc to research assistants): Gabby, Ryann, Neal, Nicayris

Hello all,

I'm sending this to you because either I know you teach Statistics, or I think that you might. I recently had a manuscript accepted for publication in Teaching of Psychology that showed that Self-Explanation was beneficial for initial acquisition of some basic statistics concepts, but there was no benefit for retention a semester later. Now I'm developing a new experiment to examine the effects of practicing retrieval on those outcomes. This is just an introductory email to see how much interest there is in collaborating on this project. I'll give you a few details of our plans, but it is still a work in progress.

1.) Before collecting the data, I will submit a pre-registration to Applied Cognitive Psychology as a Registered Report in hopes of obtaining their conditional advance acceptance for publication.
2.) As per the advice from reviewers of my previous study, this experiment will be an in-class experiment using important concepts in statistics as the to-be-learned material.
3.) To provide control over participants' adherence to instructions, while keeping the burden on the professor to a minimum, the participants' tasks will be a series of short sessions including a pre-test, several practice sessions, a posttest, and a delayed retention test, all administered through an LMS such as Blackboard (here at Kutztown we use Brightspace Desire 2 Learn - D2L.)

I would be very appreciative if you would respond with your level of interest in collaborating. Granted you may need more information. However, because we are just in the process of developing the plan, even if you might be interested given certain conditions, I'd love to hear from you and hear what conditions would work for you. Also, I welcome any of your thoughts on the design of the experiment.

Thanks,

--- Bob Ryan
===============================================

| previous | back to top | next |

Spring 2022 (2222)

Fourteenth Week of Fifteen, Mon. 04-25-22

Tues. 4/26/22 Prior to lab meetings

Since Dr. Sun (Soo Y. Sun) of Alvernia expressed interest, I'm looking into using Moodle as the LMS. I can have Neal check it out to see if it would work for us.

For example, I assume I, as a faculty member, will have to make an account as something like a course administrator. Then, the question is, could students who were not from Kutztown be enrolled in the "course?"He might need to contact Moodle and explain what we'd like to do and see what they could do for us.

Thurs. 4/28/22 - 11:30am OM 385

Had planned to meet with Nicky and Neal. Neither showed up. Perhaps I didn't communicate that we would make these meetings weekly.

Fri. 4/29/22 - 11:45am OM 385

Met with Ryann. Gabby had a Dr. apptmt. so couldn't make the meeting. Ryann has been working on how to use D2L for a fill-in-the-blank test.
Ryann created some fill in the blank quizzes on D2L. To make the Exp and Control tasks as similar as possible, the blanks for the Exp would be to complete a sentence about a concept. The blanks for the Control would be at various points in the text of the lesson, where in some places, but not others, there were mechanical mistakes. The intructions to the Control would be to fill in the blanks that followed a mistake, but to put nothing in the blanks following no mechanical mistakes.

We also looked at Moodle. I requested certification as an instructor, and sent a message describing what we want to do with Moodle. I expect to get a response from them. We will try to use the open source version not the paid version.

| previous | back to top |

Spring 2022 (2222)

Fifteenth Week of Fifteen, Mon. 05-02-22

Thurs. 5/5/22 - 10:30am OM 385

Gabby stopped in and gave me a tutorial on Google Docs to use instead of an LMS.

Thurs. 5/5/22 - 11:30am OM 385

Met with Ryann. We talked about the contact list. In some cases we will be sending our request to someone that we know teaches Statistics, in other cases it will go to a person who might teach statistics in some semesters, and in yet other cases we may need to send it to the Chair of a "Social Sciences" department (Cheney). So Ryann will create the contact list and another draft introduction/request to collaborate email. It will be on Google Docs.

We quickly checked out whether Google docs forms could be used to present our materials. It looks like it might be possible because a "form" can be a quiz. Gabby and I will check it out. The thing we're looking for is automatic "grading" of the "tasks/quizzes."

Thurs. 5/5/22 - 8:28pm At Home

See what I can do on Google Docs. Figured out that Docs is like Productivity Software, but Drive is cloud storage. So If I can selectively share things (folders, files), I could allow a participant to do a task and save it, then I would have access to it.

There's lots more for me to learn.

Spring 2022 (2222)

After the semester ended

Thurs., 5/12/22

Gabby sent her summer availability: MWF 5:30; T H All day.

Wed., 5/18/22

Ryann sent an email saying she had completed the email list for the Psych profs at PASSHE schools. She asked if she should just send the request for collaboration to the list or wait until she head from me. I responded that I'd like to see the proposed text for the request for collaboration first, to see if I thought it needed any editing. I also asked her to send me the email list.

Fri, 5/20/22

Got response from Ryann. I edited the letter to ask the respondents to "reply all." I asked Ryann to go ahead and send the request as I edited it, and to CC me, Neal, Nicky, and Gabby. The letter and the email list is in "2022_05_20-PASSHE list and Psy Faculty_Lab 2022.docx" in the folder for "Statistics - Practicing Retrieval/Responses re collaboration." Also, I updated the files on the OSF project"Statistics - Practicing Retrieval" to match the files among my files.

Mon., 5/23/22

Got response from Ryann. She sent the email request for collaboration. She said many faculty were on vacation and sent "out of office - will respond when returned." Some specified dates of return and some were not until July. Advised her to try sending it to the whole email list at once to avoid multiple sends.

Tues., 5/31/22

Sent the following email to Ryann, Gabby, Neal, and Nicky:

Subject: Progress on Dr. Ryan's research

Hello all,

I've drafted a more detailed plan for our Statistics-Practicing Retrieval experiment. I'd like to send those plans to our email list of potential collaborators. Ryann, how did you go about sending the initial email? That is, was there one email addressed to everyone on the list? Or did you send it to each school separately? I've attached the second email I'd like to send, but I want to use the best procedure to get it to everyone. Didn't you say that some schools didn't receive the initial email because it was marked as spam? As soon as I hear back from Ryann on how to best send that second email, I'll do so.

Meanwhile, if all of you would concentrate on a test to use as a pretest, posttest, and retention test, that would be most helpful. I think just one test, with the same test used at all three times, is the best we can do. If we try to make three "equivalent" tests, we have to complicate the design by randomly assigning every participant not only to Group A or B, but also, we'd have to randomly assign each participant within the groups to one of the six possible orderings of the three tests, as I did in my previous experiment. But in that experiment, we ran subjects face to face, one at a time. So we could establish in advance which subject numbers would be in which of the twelve groups. Since, in this experiment, we will be using students from different classes and universities, all of whom will be doing the tasks roughly at the same time, I don't think we can feasibly do that.

To help you with developing testing materials, I've attached the learning objectives and lessons to be used as the to-be-learned materials. I also attached the first lesson and study examples (both conditions) for the first module.

--- Dr. Ryan

Second email to recruit

Learning Objectives

Materials for Mod 1 - Use of effect

Thurs., 6/2/22

I did hear back from Ryann. She didn't say who all the emails on her list were (just Psych faculty who teach Stats, just Psych faculty, or the whole faculty). She did say that she got an email back from Shippensburg that no one received her email because it was marked as spam. She also created a better list that only included email addresses (no names) so she could just copy and paste a whole list next time instead of doing each address individually. I used that list to make an Excel spreadsheet and added Millersville, Alvernia, and Kutztown. However, I also emailed Ryann back to get an answer about who all those people were on the lists (some were long enough that they couldn't be just stats profs). I also have updated the Second email to recruit. I'll wait until Ryann tries to cut down the lists to just stats profs (I suggested she email chairs of departments). So I haven't sent the Second email to recruit yet. I'll send it when I get a new email list - hopefully shorter.

Mon., 6/6/22

Heard back from Ryann that she will work on cutting down the email list. She then sent a draft of the email to send to the chairs. I edited it a bit and emailed her back.

Thurs., 6/9/22

Finished my first demo experiments. Emailed the research assistants to register for PsyToolkit so I can send the demos to them.

Hello all,

I have finished creating an example of one trial of one learning objective for the first module. I created an example of what the experimental condition subjects (red group for the first module) and the control subjects (blue group) will see and do. In order for you to run a demo of them, you need to have an account on PsyToolkit. It's free and easy to work with.

To set up the account, go to "https://www.psytoolkit.org." On the left, click on "Web based / login." At the top of the next page, click on "Click here" to enter (or register for) the free web-based environment...." At the top of the yellow box, where it says "You are based in North America. Why not (also) register with the American server," click on "American server". Follow the instructions in the red box to register. Be sure to use the same email that I use for you.

Let me know when you have registered. I should be able to email the experiments to you so you can try them.

Thanks,

--- Dr. Ryan

Mon., 6/14/22

I had emailed the links to the res. assts. asking them to try the pretest. I don't think I sent a link for the study session (survey with embedded experiment). At any rate, apparently they don't need an account on PsyToolkit - I can just email them the link. Ryann got back to me saying she tried the pretest and just wanted to know what the to-be-learned materials that would be tested would be - just materials from the two modules or from the whole stats curriculum. I responded, just the modules. I also told her that she could, if she wished, start writing questions now, but that I would, as soon as possible, send out the review materials which would contain the learning objectives.

Tues., 07/05/22

In an exchange of emails with Ryann, I checked on the status of a list of just Psych profs. who teach stats. She said she had emailed the chairs back on 6/6, but with no responses. I asked her for a list of the chairs, which she sent today. So, I'll try emailing the chairs myself. Perhaps I'll include links to the example of the first part of the first session.. I'll also send those links to the research assistants, as well as a list of the learning objectives, and a request to work on more examples. Then we can work on the test to use as pre-post-retention.