In this video, I'm going to talk about assessment issues in Content and Language Integrated

Learning, CLEW.

Existing research on CLEW has focused more on students' learning outcomes and classroom interaction processes.

It is noticed that assessment has so far been something of a blind spot in many CLEW programs and research.

But why is assessment important and worth examining?

First, we are all familiar with the potential backwash effect of assessment, which means that assessment will affect teaching and learning behaviours.

Hence, it is important to examine the nature of CLEW assessment and see the implications for teaching and learning.

Second, how students are assessed directly affects the evaluation of students' learning outcomes in CLEW programs.

In other words, if we don't critically examine the assessment in CLEW, how do we know that we are evaluating students' learning outcomes accurately and conducting fair comparison between CLEW and non-CLEW students?

This is especially important in contexts where CLEW students are compared with non-CLEW counterparts in high-stakes public examination.

Now, let's turn to the importance of assessment in the teaching and learning process.

We usually start with the curriculum goals and learning objectives, which refer to what we want our students to learn.

Based on the stated learning objectives, teachers design instructional activities that will help students learn.

Then, assessment allows teachers to know whether students have achieved the learning objectives.

The assessment data then reflects students' strengths and weaknesses, which in turn inform the next teaching and learning cycle.

The diagram here underscores the importance of aligning the three components to promote effective teaching and learning.

When the alignment of objectives, instruction, and assessment is applied to CLEW, it becomes a bit more complicated.

For the learning objectives, teachers need to pay attention to both content and language learning objectives.

For instruction, teachers need to integrate or counterbalance content and language teaching so as to help students to achieve the dual goal.

When assessing students, teachers also need to pay attention to the content, cognitive, and language dimensions, but many teachers may not be aware of these.

These are some issues associated with assessment in CLEW.

First, do we assess language or content first?

Second, do we sometimes assess one and not the other?

If so, which one do we assess, and when, why, and how?

Third, when do we assess, and then how do we assess?

Let's try to explore these questions.

In the what question, we should assess what we set as the learning objectives so that the assessment is valid.

Therefore, if CLEW pays attention to both content and language objectives, both should be assessed.

In many CLEW programs, teachers pay more attention to content knowledge, but we would like to emphasize that the language demands involved in assessment should not be ignored.

I will come back to this point later.

In terms of content, teachers should assess students' content knowledge and cognitive skills.

For example, whether students can record the knowledge, apply what they have learned, and demonstrate higher-order thinking skills such as analyzing and evaluating.

Regarding assessment of language, teachers need to be aware of the language demands that assessment questions impose on students.

For example, when attempting assessment, students usually need to read and understand the questions, which involve receptive language skills.

At the same time, students are expected to demonstrate their content knowledge through sentences, words, or text.

These involve productive language skills.

In CLEW, content subject teachers tend to focus more on content knowledge and cognitive skills involved in assessment, but they should also be aware of the language demands involved.

Regarding when to assess, teachers may focus more on summative assessments, which are usually conducted at the end of a unit, topic, term, or academic year.

However, to have continuous assessment and provide immediate feedback for students, teachers also need to pay more attention to the quest to formative assessment, including questioning in lessons and also homework.

There are also different forms or different ways of assessment.

I have just mentioned formative assessment and summative assessment.

Meanwhile, teachers can also think about whether students are assessed individually or in groups, in oral or in written form.

We encourage teachers to employ different forms of assessment, which can better cater for students' diverse needs and different learning styles.

When teachers are assessing students, they are actually challenging them.

Here, I would like to illustrate the relationship between challenge and support and their implications.

In most cases, teachers are giving high challenges to students.

If teachers do not provide sufficient support, students have limited chance to success, and they will be frustrated.

On the other hand, if teachers provide sufficient scaffolding, student learning occurs in the zone of proximal development.

That should be what teachers aim at.

In CLEAR programs, students are learning content knowledge and language at the same time, and hence they are facing high challenges.

Providing sufficient support is then very important to ensure that learning takes place.

But how can teachers know the demands they are imposing on students so that they know how to provide support?

To address this question, I would like to share three frameworks for designing and analyzing

CLEAR assessment.

The first one is proposed by Schott, who examined the English language learners in the US.

Actually, there's been rather extensive research on how to accommodate the needs of English language learners in assessment so that they can be assessed in a more vetted way.

This figure is a matrix of what and how.

The what dimension highlights the focus of assessment, for example, whether it's on content knowledge, skills, or language.

The how dimension focuses on the format of assessment.

Instead of having written assessment all the time, we can consider other formats such as portfolios, oral reports, videos, etc., so that students with lower English proficiency can still demonstrate their content knowledge in some formats.

I think this matrix is useful for reminding teachers of different ways of assessing students.

This is especially useful for formative assessment and school-based assessment.

However, in contexts where the high-stakes examination is still dominated by written assessment, we may need to refer to other frameworks which focus on the combination of content and language skills.

So this framework, developed by CLEAR researchers in Europe, highlights the integration of content knowledge, cognitive thinking skills, and also L2 competence.

I think this framework is useful for teachers to be aware of what they are assessing students and what may be the demands imposed on the students.

But this framework looks a bit descriptive.

The last framework from our research team is similar to the second framework, and it is developed based on the English medium education in Hong Kong.

This framework combines different aspects of skills, language and cognitive.

To make it more generic, we focus on the generic cognitive demands and linguistic demands, and we divide them into three different levels.

On the cognitive dimensions, we condense the six levels of the Bloom's taxonomy to three levels, capturing the low, medium, and high-order thinking skills.

On the linguistic dimension, we also have three levels in general, vocabulary, sentence patterns, and text.

And for each level, we can also consider receptive or productive skills.

We believe this framework can generate more fine-grained analysis of assessment questions.

In a recent study, we have further revised this framework, which is more complicated.

You may refer to Lo and Fung, 2018, for more details.

But for the time being, this 3x3 matrix would help teachers understand the different demands of assessment questions in CLUE.

Let me further illustrate the framework with some examples.

With our framework, each question can be coded according to the cognitive and linguistic demand.

For example, this is a typical diagram labelling question in science.

Students need to read the sentence or instruction in the question, and then they need to write down the names of different parts.

So we will code this question as Productive Vocabulary and Cognitive Recall, as students are expected to recall the location of different organs.

This is a multiple-choice question.

Typically, students need to read the question and choices, which are often written in sentences.

So the linguistic demand is at the sentence level.

For this particular question, it asks students to recall the functions of mineral phosphorus for the human body, so it requires low-level thinking skills.

This is a structural or short question.

It usually asks the question in sentences, and students are also expected to express their answers in sentences.

In this question, the students are asked to apply Newton's first law of motion to a new scenario, so the cognitive skills required is at the application level.

In this last example, students are required to design an experiment to test the amount of vitamin C in different objects.

They are asked to carry out the experiment and write the report and observations.

In this case, students are expected to produce a text, that is, lab report, and to demonstrate high-order thinking skills.

With such a framework, we could analyze the cognitive and linguistic demands of questions in assessment, and how these different levels are distributed.

This is important.

If we want to better diagnose our students' learning progress, including their strengths and weaknesses, we may need to include questions at different cognitive and linguistic levels, so that students with lower English proficiency can still express their content knowledge in some questions.

If all the questions target at a high level of cognitive and linguistic skills, when students cannot answer the questions, we won't be able to know whether the students do not know the content or they have language barriers.

That's why we believe that this framework could help teachers to design assessment tasks with enhanced validity.

In one of our projects, we analyzed the assessment questions in textbooks and public examination at different grade levels in English-medium education in Hong Kong.

We coded the questions with our framework, and we observed that the cognitive and linguistic demands increased significantly from junior to senior forms, especially when students sit for the public examination.

This has important implications for pedagogy.

If teachers understand the demands imposed on students in assessment, they can prepare students for the assessment.

For example, in another study we conducted in EMI schools in Hong Kong, we observed that some teachers are more aware of the language demands in assessment, and in the lessons, they provided explicit guidance and instruction for students regarding how to answer a particular type of question in assessment.

Students' sample work collected showed that such scaffolding did help students perform in assessment.

In this video, I have highlighted some issues related to assessment in CLEW.

I hope teachers can pay more attention to the issues so that students can be assessed in a valid and fair way.

These are some key references if you would like to know more about the topic.