[ED 257A Week 10]: Assessment of Learning Outcomes; Adaptive Learning; Learning Analytics

Merging data science and education

This week's reading is based off of:
----------------------------------------------------------------------
Cheng, et al., 2013 “Reconsidering assessment in online/hybrid courses: Knowing versus learning” 

van der Kleij et al. (2012) "Effects of feedback in a computer-based assessment for learning" 
----------------------------------------------------------------------

Data science is a big field these days. Technology has enabled the ability to collect swathes of information on just about anything, from what you (will) shop for  to how long you spend on a particular webpage. But it can also apply to education, which is what we learned with the guest speaker of the last lecture of the course.

As a graduate student training to become a computational materials scientist, this is pretty fascinating. The idea of finding patterns and trends, and then using those to predict behavior resonates with many people across disciplines. In materials, high throughput computations are being used to accelerate the process of finding new materials. Similar skills are used wherever data science can be applied.

For those interested in getting their feet wet with data science in education, I recommend checking out Kaggle, a web platform where data scientists can gather to solve data science related problems that are proposed from various companies, organizations, and non-profits. What is great about Kaggle is that anyone can choose to take on the challenge; you are only measured based on how well your code meets the task requirements. Particularly relevant is a Kaggle competition challenge about the College Scorecard (screenshot below), a measurement system from the U.S. Department of Education to elucidate the relationship between future income and university attendance. Check it out!

 

ED 257A Week 9: Discovery/project-based learning; learning in virtual environments; game-based learning

Learning in Virtual Reality

This week's reading is based off of:
---------------------------------------------------------------------------------
Gonzalez-Gonzalez & Blanco-Izquierdo (2012) "Designing social videogames for educational uses" - Caitlin McDowell & Sahar Sajadieh

Vander Ark (2014) "Eight principles of productive gamification" - Caitlin McDowell & Sahar Sajadieh
----------------------------------------------------------------------------------

Week 9 class was conducted a little differently than usual. Instead of meeting in-person in the Education building, we all participated in a Zoom video conference to talk about the gamification in the classroom. In the Zoom interface, there is a gallery of participants, all of who can participate in the video and audio. Whenever someone speaks, their video screen is highlighted in green in gallery mode or becomes the largest screen in speaker mode. While I thought the Zoom interface and session itself went well, I still prefer the face-to-face in-class discussion we have on a regular basis. Often times, it was difficult to focus on the speaker because so much else was also happening on the screen. In contrast, I often physically turn towards whoever is speaking at the moment and very little enters my periphery. There was also an electronic chat that was happening simultaneously. Although it is a great way for people to contribute to the discussion (who may normally be reluctant to speak, like myself), I had to constantly switch between the video and the chat, which made the discussion more mentally taxing. I can see the value in Zoom being useful for at least establishing face-to-face time for teams where that is difficult (e.g., international collaboration, across large geographical distances). At the end of that day, I still find in-class time to be more valuable and organic. However, I'm glad we went through this experience with video conferencing. It was very informative, and I think I may use it in (not future classroom sessions but rather) future collaborations in my research.

 

Source: http://zoom.us/subscribe (top), https://maroonloon.files.wordpress.com/2014/04/zoom-presence.png (bottom)

We also discussed how gamification would fit into our education curriculum. The tutorial on Edorble given by founder Gabe Baker was particularly interesting. Edorble is a web platform to hold virtual classrooms. It is similar to Second Life in that you interact with a virtual world through a customizable avatar. Edorble is more specific to the classroom setting. There is a lecture space in which a video screen is shared among users and also additional space (e.g., to use for break-out discussions).

Source: http://www.edorble.com/

I found it interesting that both Zoom and Edorble aim to recreate human interaction, but do so differently. Zoom does so using a collection of video screens and audio. Unlike Zoom, each participant in Edorble has an avatar that can interact with other avatars and the environment. Yet both aim to facilitate face-to-face discussion and collaboration. Zoom trades off the idea of a discussion space for more direct dialogue between participants while Edorble trades off that direct dialogue to be mediated among avatars. It would be interesting to see and reflect how differently people (including myself) would act in each of these scenarios, and how that compares to the traditional face-to-face dialogue. I'm curious to see how Edorble will implement other features into their virtual classroom, and whether they plan on including the ability to write on virtual blackboards and whatnot.

Some final thoughts I have on virtual reality is more on the aspect of gamification. There are certainly many benefits that gamification can achieve if designed well. As enumerated by Vander Ark, gamification allows for productive failure, boosts persistence and intrinsic motivation, can lead to deeper learning but needs conceptual challenges and careful calibration. I tend to think more about learning in higher education, so a hesitation I have using gamification in the classroom is that it may detract from the core content of the lecture. Another is that the material that would be used in such lectures would need to be self-generated, since it is currently not common to use gamification to teach STEM subjects. Perhaps gamification can help make STEM subjects more accessible and less textbook dry or maybe could be a more subtle incorporation into the course. The classic is a game of Jeopardy or Bingo, but I would be interested in hearing other people's thoughts on the subject.

[ED257A] Week 8: Feedback: from instructors, from peers; Learner engagement

Assessment and Feedback in the Classroom

Source: http://businesssolutions.it/en/assessment-center-2/

This week's reading is based off of:
---------------------------------------------------------------------------------
M. Brady, H. Seli, J. Rosenthal. "Metacognition and the influence of polling systems: how do clickers compare with low technology systems." Education Tech Research Dev (2013) 61:885–902

F. Dochy , M. Segers & D. Sluijsmans (1999) "The use of self-, peer and coassessment in higher education: A review," Studies in Higher Education, 24:3, 331-350,
DOI: 10.1080/03075079912331379935
----------------------------------------------------------------------------------

This week in class we went over learner engagement and feedback in the classroom as part of assessment and feedback in the classroom. As one of the discussion leaders, I experimented with in-class polling technologies (namely PollEverywhere) and had a discussion of the merits of clicker technology. I think this was a useful experience personally given my previous experience with clickers when it was a more recent technology.

When I was taking introductory electromagnetism, the professors were experimenting with a new program on campus called TEAL (Technology Enabled Active Learning), in which clickers were a part of the curriculum and grade. We had the TurningPoint clickers that could take in alphanumeric inputs (shown below).

We had to bring these clickers every class session for attendance and participation. The inconvenience of these clickers were that they were pricey, really only used in this one course, the buttons mechanically wore out quickly, and if you forgot your clicker, you lost points for that day. Of course, for students who didn't want to show up to a lecture, they could just give their clicker to a friend for the day. Perhaps having clickers did help in the engagement of the material, but the last thing I remember from this experience is wondering how I could sell off my clicker and recover a decent amount of what I paid for it. 

But I think clicker technology has come a ways from model. We saw in class that new clickers of the TurningPoint type have improved design. One technology I thought was particularly promising was PollEverywhere. The main advantages of PollEverywhere are:

• you need very little extra hardware or software to use it. The poll creation, administration, and response collection are all mediated using their website or a plug-in (e.g., to Google slides). Students can submit responses either by visiting a unique url or by texting. 
• a variety of questions/responses are possible- multiple choice, clickable images, free-form text. It can also handle LaTeX notation (++ for doing equations). 
• the user interface is simple, clean, and easy to use. I found that I spent more time thinking of poll questions than figuring out how to use the tool. 

During the course of the class, I wanted to experiment with several things that were covered in the Brady, et al. paper. Brady, et al. did a study on undergraduate psychology students and compared clicker (high-tech) with paddles (low-tech). They postulated that having that in-class polling motivates students to reflect on their progress towards their learning goals by improving their metacognition (i.e., their awareness of what and how they are learning). Overall, they found clickers are useful for this, particularly for low- to mid-performing students. I found the comparison to low-tech technology to be interesting, because both methods provide the same instant feedback. However, clickers provide a way to accurately tally answers and keep record of it and avoid the group mentality of polling the class with paddles.

In the experiment, we compared the experience between PollEverywhere and using American Sign Language to answer what city had the largest population density (ans: Mumbai). We first started with American Sign Language where everyone signed the letter corresponding to the answer. Several interesting observations are worth noting. When the real-time tracking of answers was hidden, I noticed the majority of answers were answering Tokyo. When using sign language, it was harder to distinguish which was the majority answer. At the third time when real-time responses were displayed (shown below), there was less of a spread, and a clear winner of the possible answers. While we observed the clear influence of group mentality among the three trials, it was interesting when someone remarked that in seeing other people submit an answer they were thinking about made them more confident about their answer, even if it were incorrect.

PollEverywhere interface for in-class discussion and results the second round

There were several lessons I learned in doing this exercise:

• Always make time for technical glitches. The internet is not always reliable for these types of things. Conducting such an in-class poll for a large class may not be feasible with the current network infrastructure of the campus.
• Be sure not to give away the answer before revealing the answer. I accidentally slipped out a hint of the correct answer, and some picked up on it. 
• I think it would be interesting to have someone purposefully submit and incorrect answer to encourage people to choose the answer they would have originally chosen. This would be a great way to start a dialogue on why the answer is incorrect, or go over the thought processes that went into the choice.
• Polling on how confident people feel about their answer would also be a good learning experience, because it's telling of what and how they are thinking about the problem. 

Overall, despite the not-super-steller experience I had with clickers before, I would use an in-class polling technology like PollEverywhere in a future class. I like the possibilities that it offers to really engage students and become a tool to start a conversation about the material. If paired with a learning (and social) activity, such as think-pair-share, the full potential of in-class polling could be realized. 

[ED257A] Week 6: Flipped Classroom and Distance/Online Learning

The Flipped Classroom

Source: https://21centuryedtech.files.wordpress.com/2012/07/flipped.jpg

This week's reading is based off of:
---------------------------------------------------------------------------------
Miller, 2015 Ch. 9 "Putting it all together"

Min Kyu Kim, So Mi Kim, Otto Khera, Joan Getman. "The experience of three flipped classrooms in an urban university: an exploration of design principles." Internet and Higher Education 22 (2014) 37–50
----------------------------------------------------------------------------------

The flipped classroom introduces a new way of instruction in which the student partakes in lecture outside of the classroom and problem solving during class time, and the teacher's role is more facilitative. The surge in the flipped classroom concept has gained momentum with the burgeoning of online technologies like Khan Academy.

I personally am excited about this new model for education. The flipped classroom allows students to pace and personalize their learning. If there is a concept they are unfamiliar with, they can spend the time to understand it without worrying that the instructor has moved on to a new concept. Having the teacher become more of a facilitator of activities during classroom improves face-to-face interaction, as students are directly interacting with the instructor instead of (passively) absorbing information. Additionally, through the usage of digital technologies, the course content may be made available to anyone else with access to internet. I think this idea is particularly powerful. People in regions with fewer educational resources or privilege can freely access this information.

As a student, I appreciated the flipped classroom model. In my undergraduate studies, I took a course on electronic devices that borrowed from the flipped classroom. We watched lectures outside of classroom and submitted questions we had on the lecture. During class, our professor would address all of our questions and go over other interesting aspects about electronic devices. We had homework assignments outside of lecture as well. There were of course some tradeoffs. In exchange for more interactivity during lecture, we had less mathematical rigor in our learning of electronic devices. However, as an introductory class for learning the conceptual operation of electronic devices, I found the flipped classroom format to be well-suited to the task. Of course, the flipped classroom is not the panacea to all challenges in instruction. I think the traditional lecture style is still appropriate for certain subjects, such as those that are discussion based or math heavy. Nevertheless, the flipped classroom offers another tool with which to instruct.

The techniques and tools presented in Miller and Kim, et al. were applicable to both the flipped classroom and instruction in general. Kim, et al. list useful principles for designing a class, which are reiterated here, and paralleled with ideas I have had or discussed with others. Many are being implemented in the course I am currently helping to teach (MAT 188). To reiterate, MAT 188 is an overview course about materials science in the context of energy technologies. The goal of the course is to expose non-materials scientists to how materials scientists think about the world around them. Although MAT 188 is not structured to be a flipped class, I think many of the same design principles listed in Kim, et al. apply here. Please let me know your thoughts and suggestions!

• Provide an opportunity for students to gain first exposure prior to class
• HW questions to preview next week's lecture, going over the class syllabus, presenting a course map at the beginning of each lecture
• Provide an incentive for students to prepare for class
• extra credit on HW (e.g., choose 3 of the 4 questions to complete; the fourth problem is graded optionally for extra credit)
• Provide a mechanism to assess student understanding, e.g., low stakes quizzes
• weekly quizzes to gauge immediate comprehension of the lecture (e.g., main idea, muddiest point) and weekly HW, doing a lecture recap
• Provide clear connections between in-class and out-of-class activities
• present course objectives and agenda in syllabus, inviting guest lecturers to discuss a topic related to the course (e.g., their career path, current/active research field, where they envision the field going next, where the research is found in everyday applications)
• Provide clearly defined and well-structured guidance
• having a lecture outline at the beginning of each lecture, having a defined format of the lecture, offering extensive office hours
• Provide facilitation for building a learning community
• have the students be responsible for a topic for a final review session where it is presented small groups
• Provide enough time for students to carry out the assignments-
• introducing final projects weeks in advance (i.e., first heads-up, then more formal choice of group/topics, and periodic check-ups)
• Provide prompt/adaptive feedback on individual or group works
• mid-term evaluations, weekly quizzes for determining comprehension right after lecture
• Provide technologies familiar and easy to access
• currently not as applicable, as we mostly use video, powerpoint, and in-class demonstrations of technologies

I am particularly excited about the potential of Gauchocast, 

a tool for recording online lectures. Below is a screenshot for the Fall 2015 version of Quantum Mechanics II. Not shown are the note taking abilities (text-based only; eventually adding in equation editing, like based on the syntax of LaTeX, would be a cool addition). 

Gauchocast allows you to see the headshot of the instructor, a set of slides and the timing of each slide (which is done automatically, based on the title of each slide). Although a transcript option isn't available yet, there is still much potential in this tool. While this doesn't replace actual face-to-face interaction with the instructor, it still offers the ability to see instructor as s/he is speaking about the subject matter. My favorite part is the fact that you basically have a full record of the lecture and is text searchable later on. This makes studying more streamlined for students. But this also applies to instructors as well. A well-made video lecture would save much time in the future for preparing for lectures and would be easy to keep track of what was covered in previous years. I hope that more professors choose to use this tool in the coming years and see if it's a suitable course to what they are teaching. 

[ED257A] Week 5: Collaborative Learning with Digital Media

To Wiki or not to Wiki?

Source: http://i.livescience.com/images/i/000/036/864/original/Internet-IPv6.jpg?1361298852

This week's reading is based off of:
---------------------------------------------------------------------------------
Ioannou, Brown, & Artino. "Wikis and forums for collaborative problem-based activity: A systematic comparison of learners' interactions" Internet and Higher Education 24 (2015) 35–45

Cacciamani, et al. "Influence of participation, facilitator styles, and metacognitive reflection on knowledge building in online university courses" Computers & Education 58 (2012) 874–884
----------------------------------------------------------------------------------

Technology has closed geographical, cultural, language distances and time with increased connectivity and in ubiquity of information. The complexity of the challenges we face as a society necessitates the ability to collaborate in a variety of settings. This week's discussion on using technology to facilitate collaboration in the students thus sets the stage for gaining that skill set. Of the technologies we discussed this week, I found wikis to be the most fascinating. Wikis are websites that allows its users to contribute to its content and structure, the most famous being Wikipedia.

Ioannou, et al found that wikis foster more collaborative (rather than cooperative) when compared to participants using MS Word attachments to accomplish the task. I found this to be an interesting result. Students discussed with each other parts of the wiki more so than that of the MS Word documents, which presumably entails exchange of ideas and feedback. However, as briefly discussed in class, the social dynamics of having to review a peer's work underly this kind of collaboration. For instance, for fear of offending or slighting the other, there is a tendency to not delete but rather to add or comment upon. Perhaps it is also a cultural thing- the generalization being that Americans tend to sugar coat things whereas foreigners do not. That is, collaboration is inherently a social activity. The major question is, do wikis facilitate easier collaboration and social interaction? Have you observed similarly or differently in your own experience when collaborating with others (e.g., using a wiki or other technology-based tool)?

This is indeed something I do myself. At times, I feel like the edits I make are easily interchangeable with the existing content, but is maybe more in line with how I interpreted the text; or that the wording the author chooses had a hidden intention not apparent to me as a reader; or the general habit that it's always good to keep previous versions of documents so that you can go back to them. I have personally been on both sides of the study that Ioannou, et al. use. In the course I am teaching this quarter, the structure of how my co-instructors and I have followed naturally lends itself to a wiki-like collaboration. All the material content we generate is hosted on Google drive. Each week we meet to discuss the following week's lecture, homework, and logistics so that at each step we exchange information and feedback. I have found this to be immensely useful as my co-instructors have varied experiences and insights. While there is not single aspect of the course that I could assign as being self-authored, I feel the style of wiki-like collaboration has been an extremely useful method towards forming a course.

At the same time, I have done the more cooperative-style of work in groups. I am currently organizing a conference symposium on diversity in STEM fields. One aspect of this organization was writing a blurb on what our symposium was about and why attendees to go to this symposium. My co-organizers and I first wrote blurbs independently and then merged the documents at the end. While there were sentences I wrote that I felt were well-written but left out, the final product overall was well polished and I was happy with it. This may be part of the negotiation, not only with your teammates but also with yourself, of what is presented as the final product and what is not. Perhaps the more wiki-styled collaboration would have done better, the same, or worse. Can you think of a situation in which both a collaboration and cooperative-style teamwork would do well?

Because you can track a wiki's history in detail, it lends itself to collaboration. Or maybe it doesn't, since everything you do is recorded, and so people may choose not to comment on certain things versus others.

Regardless, I think wikis have great potential, particularly in the STEM fields where the passing down of knowledge is critical (e.g., using wikis as an archive of techniques and useful knowledge in a lab group as students graduate from or join the research group). Wikis are also natural bridge between knowledge acquisition and knowledge building (see Cacciamani, et al.) in the summarization of knowledge from various sources into a digestible format. And because it can be published on the web, it has peer-review built in. Here, I am interested in the role of the teacher if one were to run a wiki-based project. I'm particularly interested in implementing this as a final project/review of sorts in my class MAT 188: Materials in Energy Technologies the next time it is offered. It's a direct application of what they have learned and it can benefit someone else interested in material science in energy technologies. How could instructor present themselves as supportive or oppositional in this context in your field? Through a guided formulation of the wiki? By constantly monitoring wiki activity and giving feedback? What are your thoughts?

[ED257A] Week 4: Social Constructivist and Sociocultural Theories of Learning

#LearningUsingSocialMedia

Adapted from: http://cristianaziraldo.altervista.org/debating-social-media/

This week's reading is based off of:
---------------------------------------------------------------------------------
Bennet, et al. 2012 "Implementing Web 2.0 technologies in higher education: a collective case study" Computers & Education 59 (2012) 524–534

Manca & Ranieri, "Is it a tool suitable for learning? A critical review of the literature on Facebook as a technology-enhanced learning environment." Journal of Computer Assisted Learning (2013), 29, 487–504

Warschauer & Grimes "Audience, Authorship, and Artifact: The Emergent Semiotics of Web 2.0"
Annual Review of Applied Linguistics (2007) 27, 1–23.
----------------------------------------------------------------------------------

Our class discussion on the usage of social media (e.g., Facebook, Twitter, Instagram) in the classroom greatly highlighted the importance of understanding the appropriateness of a tool to a task.
We learned from Bennet, et al. that even using the same activity (e.g., posting photos on Flickr as a class exercise or writing a blog) can lead to varying degrees of success. The same can be said of other technologies that have emerged from the Web 2.0 with the rise of user-generated content. The theme of a "square peg into a round hole" appeared several times (i.e., forcing the usage of tool for a task it was not designed for), and highlights the need that as instructors, we need to critically think about how to use the plethora of technology now available.

Manca & Ranieri propose the possibility of Facebook as a technology-enhanced learning environment. My knee-jerk reaction upon reading this was that this was not a good idea. It would mix people's social and academic lives, and it would introduce so much uncertainty in the student-teacher relationship (e.g., do you friend the student or not?). As someone with more STEM leanings, I was also hard pressed to think of times at which Facebook would be appropriate. However, I thought the in-class example of using Facebook to facilitate intercultural learning between students was fantastic. Because Facebook facilitates more informal social networking, it was well-suited to facilitating a more natural interaction between students of different cultures. This was a great example to enforce that while you need to think critically about the appropriateness of a tool to a task, it is also important to keep an open mind.

Although this was not discussed in class, I was also intrigued by the article from Warschauer and Grimes about the underlying social context behind social media. For example, several studies found a correlation between cultures with high power differentials and low frequency of editing of Wikipedia. That is, if a hierarchy of authority was more ingrained in a culture, people tended to edit Wikipedia less. I thought this was a striking observation- it implies that in-class activities cannot be treated as closed systems between student and teacher, but that other complex, confounding variables are at play. I wonder, have you as the reader observed this to occur in your experience?

I was really excited about the emergence of Web 3.0. Web 1.0 was mostly focused on building the internet (e.g., building the protocols, interface, formatting) and generating content. Web 2.0 shifted towards user-generated content with the expansion of the internet to mobile devices. Now, it is predicted, the internet will evolve again into Web 3.0, a more adaptive and intelligent form of the the internet, which includes but is not limited to: semantic web, microformats, natural language search, data-mining, machine learning, recommendation agents. These all (to varying degrees) stem from the manipulation of huge amounts of data and extracting useful information. While the technical details would be impossible to cover in the length of the class period, I found this upcoming stage of the web to be exciting. What do you think? How do you think it will affect the classroom? Will this redefine how we teach or what the role of the teacher is?

[ED 257A] Week 3: Cognitive Theories and Multimedia

Learning how to learn

Source: http://www.digitalistmag.com/files/2015/10/Prospect-Mortgage.jpg

This post is based off of the following readings:
---------------------------------------------------------------------------------------
Miller, 2014, Ch. 7 "Incorporating multimedia effectively"

Ocepek, et al. 2013. "Exploring the relation between learning style models and preferred multimedia."
---------------------------------------------------------------------------------------

This week was quite enlightening for myself as a student and instructor. We learned this past class of the different learning styles that have been proposed (I'm experimenting with media type 3 from Ocepek. Please let me know it's going :) ): 

• Kolb's learning style: 4 modes of learning, most commonly applied in e-learning, which can be combined into dominant modes
• 4 modes: abstract conceptualization (AC), concrete experience (CE), active experimentation (AE), and reflective observation (RO)
• combinations of dominant modes: assimilator (dominant modes AC and RO), converger (dominant modes AC and AE), accommodator (dominant modes CE and AE), and diverger (dominant modes CE and RO)
•  Rancourt’s learning styles model (RLS): 
• “a characteristic mode of way of manifesting cognitive and/or affective phenomena”
• rational (R), empirical (E), and noetic (N) style (subjective insights)
• Hemispheric dominance and learning styles (HLS)
• right-hemispheric (simulations, experiments, intuitive reasoning, creative thinking), left-hemispheric (facts, logical reasoning, working with numbers), and integrated learning style
• this is the left-brain, right-brain model
•  VAK learning style model (VLS) (reduced VARK, with reading)
• visual, audio, kinesthetic learning modes

I was most familiar with the VAK learning style model, and used to identify as a primarily kinesthetic learner. Because of this, I always wrote things down and did loads of practice problems in order to make up for the PowerPoint presentation heavy lectures (even for derivations!) in school. Perhaps this ended up being the self-fulfilling prophecy we discussed, in which if you believe you cannot do something, you won't. As the reader, have you ever found yourself in a similar position? Last week's class was a great clarification of what it means to learn, i.e., learning how to learn.

There were particular insights from our small group and class discussions that I think are worth mentioning. The first is the difficulty in measuring the effectiveness of media in the classroom. Not only are there huge numbers of variable to control (e.g., time of measurement, demographic, finding a representative form of multimedia, discipline-specific challengest), but also confounding variables to account for (i.e., correlation, not causation). This is a common trait among all fields of research, but is a good reminder that just because the results say something, it does not necessarily reflect reality. That is, even if there is research that says one particular form of media is good or bad, this does not mean it is the panacea or poison of education. We always need to be vigilant of our results.

A second insight is to focus less on learning styles and more on learning preferences. I would like to add, perhaps it also makes sense to focus more on understanding the learning preferences each form of media enables. In other words, it is not necessary for a specific form of media (e.g., simulations) to fit in a specific learning style (e.g., right-hemispheric thinking). Of course, this complicates the picture, but it also opens some opportunities to think of forms of media in a new light (e.g., using simulations to go through a thought experiment, which requires logical thinking). Along the same vein is how to measure the successful implementation of media in the classroom. The biggest realization is the lack of congruence between teaching with media and testing without media. The example given in class I think illustrated this point well: for a foreign language class, changing the test to include more contextual and picture-based questions as practiced led to better retention. 

This led me to rethink the class on Materials in Energy Technologies course I'm helping to teach this quarter. Our goal in this course was to make materials science accessible to the non-materials scientist and to make connections of energy-related technologies with concepts of materials scientist. In order to do so, we've included many interactive activities (e.g., think-pair-share) to encourage our students to think about how materials science impact their everyday lives (e.g., how does understanding materials in extreme environments impact our ability to provide the electricity you use everyday? ans: it lets the turbines work at the extreme temperatures and pressures needed to transform the mechanical rotating energy from pushing steam into electrical energy). We've also included many guest lectures to show that all the theory we talk about in class has real-life applications and are still open-research questions. However, we still have a final at the end of the course planned. I feel we could very easily fall into same trap of testing in a different mode than our teaching.  Do you have recommendations/suggestions? Have you had a similar experience where the test and teaching methods didn't match up? I'd love to hear about your experiences!