Despite the widespread popularity of criteria-based assessment methods, there are some who are disappointed with them. The complexity involved in understanding the principles, evaluating, and interpreting makes it no less formal than traditional non-criterion-based methods. This means that, despite the significant increase in labor costs, it is equally ineffective. As an expert in HR technology, “Assessment Centers”, which are considered very expensive by the “conduits of working souls”, do not become less expensive in the educational system without a discussion on the justification for this transfer of technology.
There are two main approaches to reducing the teacher’s assessment workload – to automate this task using digital intelligence (QI) or to change the assessment parameters/method. Here, I will not discuss QI – this is the domain of young and enthusiastic developers seeking to harness the power of numbers. Instead, I am referring to the approach we have been exploring recently – assessment through qualitative indicators, by identifying distinctive features.
Traditionally, assessment has been based on measuring the difference between a solution and a model solution. The goal of such assessment is to identify proximity zones to the model solution and provide feedback to the assessed individual on how close they are to reaching the model (this can be read about in the article “Figure Enlivens the Mark”).
The advantage of this approach is its ability to track progress – whether it was far from the model at the beginning and then began to approach, or whether it did not begin at all, or started to diverge.
The disadvantage of this approach is that, firstly, the dynamics are not essential or relevant for everyone, and secondly, measuring the magnitude of short flights can be time-consuming. If there is an understanding that “first” is not required, it may be unnecessary to expend effort on “second”, as the need for such effort is a source of frustration.
In order to make changes to the assessment process, it is important to determine the parameters of the assessment – what will be evaluated. Discussions about evaluation results have taken place for a long time without resolution. From this, I conclude that education is infinite, and therefore it is unproductive to argue about results: each individual must decide for themselves what they want to focus on during the assessment process.
Ideally, students should report their performance, and teachers (schools) should assess them. However, this model is rarely feasible, as only a professional can clearly identify the relevant parameters. It is more realistic if schools (teachers) declare what they are guided by, and students who are satisfied with this approach will follow their guidance.
However, students (parents) who seek a “maximum USE score” may argue with me successfully. Although such an educational goal is understandable, many schools (teachers) are already established to achieve this for a long time. I am not referring to this specific goal, but to a broader request for developing a worldview and personal growth, where the USE is just a stage for obtaining formal documents. This request may not be the most common, but it is still significant.
We found it interesting to assess two parameters: the level of proficiency and the level of subjectivity. Qualitatively, we assessed these parameters without using terms such as “more” or “less”, or “better” or “worse”. The level of proficiency (or practical experience) is similar to the traditional focus on this aspect. The level of subjectivity, on the other hand, is atypical. We are not the only ones trying to assess this aspect now, as there is a growing demand for the development of students’ personalities. Visually, our model resembles an octahedron – a diamond-shaped crystal, hence the name “Personality Diamond.”
I will not go into detail here – for those who interested, at the end of this post, you will find a selection of updated links to relevant publications. This post is about the initial stage of our journey in this direction. It is about the iSPEC model, which originated from Guli Bazarova’s project for the Skolkovo School. Unfortunately, this model was not widely adopted for a long period of time. Therefore, with Guli’s permission, I have begun to develop it independently and incorporate it into other projects. Here, I want to highlight a particular aspect of the evaluation process that I found useful in the context of the logic presented.
One of the key factors that has proven to be beneficial for schools seeking to foster subjectivity is the differentiation of educational tasks into two categories: typical and atypical.
- Typical tasks are those, the meaning and methods of solution of which were explicitly explained in class. Students are expected to practice and demonstrate their ability to solve these tasks.
- We consider tasks that require a student to use personal initiative in finding a solution to be non-standard.
In the traditional educational process, there are both standard and non-standard tasks, but, first, they are not specifically identified in the journal and are not recorded separately, and, second, non-standard tasks are rare in the traditional teaching approach. If non-standard tasks were recorded in an electronic journal, the teacher’s workload would be negligible, but a significant amount of previously hidden data would become visible: it could be presented in the form of different charts in the context of the student, class, or teacher to compare the number of standard and non-standard assignments, which ones are completed by students better/more often, and which assignments are assigned by a teacher. It is also possible to present this information in the form of Yandex-tables, but this approach is less flexible and more time-consuming, although this depends on individual preferences.
If it is important for a teacher or school to understand how a student approaches and solves atypical problems – and this student can either find a solution by using available resources, or come up with their own solution – then, in addition to assessing the solution itself, the teacher could also provide a mark for creativity in finding solutions. Without a teacher, it is difficult to distinguish between a resourceful and an inventive solution – this additional step would be a burden for the examiner, but would significantly simplify the process of assessing the student’s proximity to the desired outcome. While there is a risk of error, this is an integral part of the learning process and allows for further development.
However, for the time being, in order to track subjective development, the school has not been interested in details more complex than the “active/passive” dichotomy. To promote subjective development, it is more relevant to focus on reflective practices, which present a number of challenges. Therefore, this year, we have shifted from the “Personality Diamond” model to procedural formats for reflective practices, as the iSPEC framework (in Russian YARILO notation) alone was not sufficient for schools to focus attention on reflection. We have examined various existing practices and, based on these, developed our own, which we believe may cause less resistance to regular implementation.
HyperMethod Partners have partially implemented our grading system logic on their distance learning eLearning Server platform. This is in addition to traditional grading models – it is sufficient to specify a threshold for the successful completion of a task. If users take advantage of this feature, the platform will provide interfaces with simple statistical charts. If there is demand, the platform’s capabilities can be expanded to provide more detailed monitoring of grading and subjectivity levels.
Continuing with the grading logic, it is possible to note at the task declaration stage the direction of the task:
- Informational (understanding the educational context and reasoning on the topic)
- Applied/Qualification (ability to solve typical problems)
These data are also not differentiated in any way in the journal. Although statistical processing provides useful information about the direction of the teacher’s work and the direction of student development, it is not necessary to compare “more/less” or “better/worse” when using this approach. Instead, it is sufficient to determine whether a task has been completed or not. This logic is reflected in the “Diamond of Personality” model. However, we have not found interest in conducting a qualitative assessment of the level of development/practice, as everything is measured in various ways using the same sample.
