Concurrent Validity of a Virtual Version of Box and Block Test for Patients with Neurological Disorders ()
1. Introduction
Affectation on manipulative dexterity and loss of motor coordination are some of the consequences of neurological diseases, greatly conditioning the autonomy and level of independence and social participation of people who suffer from them. Thus, the quality of movement and performance of daily life activities is heavily influenced by hand function and manipulative dexterity [1].
One of the most important goals of the spinal cord injured patients’ rehabilitation program is to recover and maintain the maximum possible level of upper extremity (UE) function and manual dexterity allowed and conditioned by each particular spinal cord injury level and severity. Patients often cannot successfully perform manual or finger fine dexterity tests, such as the Purdue Pegboard Test [2] or the Crawford Small Parts Dexterity Test. Other manual dexterity tests such as the Minnesota Rate of Manipulation Test [3] have limited clinical application because they require a lot of administration time, a standardized starting position, and also use normality values representing in a very poor way the extent patients’ range. For these reasons, clinicians often subjectively assess UE skills [4].
This is where the Box and Block scale emerged, as a way to solve the limitations detected in other measurement tests. The Box and Block (BBT) test was designed as a method for evaluating and measuring dexterity and grip, handling, and releasing functions. It was designed to be durable and simple so that it could be used for the assessment of patients with severe dexterity deficits. It is also validated by showing normality values by age range [4].
There is evidence from several studies that have applied BBT as one of the methods to evaluate manual dexterity in patients with central nervous system disorders. These studies have been analyzed suggesting that BBT is a fast, simple and safe method for measuring manual dexterity with good applicability in adults and children with neurofunctional disorders [1].
Currently, there is a growing interest in the use of technology in the Neurorehabilitation field with the aim of quantifying and generating knowledge about sensorimotor disorders after neurological diseases, understanding that the technology has a high potential for using as therapeutic tools [5]. Thus, at present, there are several works that propose virtual versions of this test, in principle designed for stroke patients and using low-cost technology to allow the patient interaction with the virtual application, such as Microsoft Kinect [6] and Leap Motion Controller [7]. However, no evidence has been found with respect to the analysis of the relation between the virtual developments and the real BBT in Spinal Cord Injury, mostly.
Therefore, the objective of this work is to present the virtual application of the Box and Block scale, as well as details about its design and development for manipulation through Leap Motion Controller and to analyze the concurrent validity of this virtual application in comparison to the real test. The relationship between the results obtained in the actual test and in the virtual application was analyzed in healthy subjects and neurologically affected patients, mostly patients with cervical Spinal Cord Injury (SCI), obtaining a high correlation between both tests performance. The overall objective is that this virtual test will serve as an element to evaluate in the future the effectiveness of a prototype based on virtual applications equipped with a therapeutic sense that, manipulated from Leap Motion allow work the manipulative dexterity patients with neurological conditions [8].
2. Methodology
2.1. Box and Block Test Description
The case consists of wooden boards 1 cm thick. The dimensions of the case are 53.7 cm long × 25.4 cm wide. The long sides of the case are 53.7 cm by 8.5 cm and 1 cm thick. The short sides are 7.5 cm by 25.4 cm. The box is divided into two square compartments by inserting a split board in the center. This board is 25.4 cm long by 15.2 cm high and 1 cm thick. At the top of this division there are two 2.5 cm holes to facilitate the transport of the box. A layer of varnish is then applied and then a fabric is placed on the base to minimize the noise of the blocks during the test (Figure 1).
On the real scale 100 wooden blocks are used, they are cubes of 2.5 cm. 150 [4] were used on the original scale. If you look at the results section, healthy subjects were left at a yield of approximately 80 blocks, which is probably why the current actual scale has 100 blocks.
The test instructions are a slight modification of the original test. The box is located along the edge of a table standardized in height. Each subject sits in a chair standardized in height in front of the table and box. The blocks are in the box compartment that corresponds to the subject’s dominant hand (in healthy people). The test examiner or observer sits opposite so that he or she can see the blocks being transported. A trial period of 15 sec. precedes the test. The blocks are then returned to their compartment. Immediately afterwards, the test begins and before starting, the subject puts his hands on both sides of the box. At the signal, the subject takes only one block at a time with the dominant hand, carries the block over the partition and releases it into the opposite compartment. Time is timed and stopped after 1 min. Then the procedure is repeated with the
![]()
Figure 1. Box and Block test (real scale).
non-dominant hand. After completing the test, the examiner counts the blocks before returning them to the compartment. If the subject transports two or more blocks at the same time, only one is counted. If the block bounces and falls to the ground or table, it counts as success. If the block and fingers do not exceed the partition, that block does not count as successful.
2.2. Development of the Virtual BBT
The design of the virtual version of the scale was defined by clinicians, paying special attention to those elements that allowed a better adaptation of the application to the patients. The software Unity3D, as video game engine, was used for the development, which offers enough features to meet our goal and has the advantage of having a large community of developers.
The game consists of various scenes and game modes. On the one hand the scale has been implemented following the same guidelines of the article, first a 15-second test, without a score, and then one minute, counting the cubes that pass from side to side taking into account the restrictions described in the scale. On the other hand, the game mode has been designed and developed, proposing several difficulty levels in function of the cubes’ sizes.
Calibration allows detect if a patient’s gripping ability is below a threshold, and in that case activate the autograsping function, so that the cube adheres to the hand just by touching it.
The Game scene is the main game and is composed of several elements (Figure 2(a)):
· SceneElements: includes the scene’s main camera and lighting.
· LeapElements: elements that allow the capture of hand movements using the Leap Motion GUI sensor: user interface.
· WoodBox: The box where the cubes will be placed. It has several triggers used in the game.
· PlayManager: is an object whose behavior is defined in a script with the same name, is responsible for controlling the game in a general way, launching the necessary events to give you the game, as well as managing changes such as game completion. This includes updating the new data of the level to be played or the graphical interface.
The main object within the scene is the wood box, designed by using Blender, following the real test characteristics to maintain the semantic of the real scale [4], providing a more adaptative version that those presented in a previous study [9].
Once the virtual test elements were developed, all of them were embedded into a graphical user interface composed of different elements which deliver visual feedback to the patient about different aspects (Figure 2(b)): difficulty level of the application, instructions panel, a timer with a countdown, and the points in function of the positive objectives reached. Moreover, previously, five healthy people performed the different difficulty levels and a progression bar relative to the healthy performance was included.
![]()
Figure 2. Box and Block test: (a) model of the wooden box and (b) virtual version developed in Unity.
2.3. Study Population
The sample analyzed and included in the study was of 24 subjects, divided into two groups: a healthy subjects (HS) group (n = 12) and a group of patients (n = 12) with neurological diseases and as consequence UE motor function impairments. Most of patients suffered a cervical SCI (n = 10) with metameric level between C4 and C7, and AIS grade between A and D (as defined by the International Standards for the Neurological Classification of Spinal Cord injury [10]); one patient suffered Guillain-Barré syndrome and another one suffered sequelae of an infectious meningen cephalomeningitis. Patients who presented any vertebral deformity, joint restriction, surgery on any of the upper limbs, balance disorders, dysmetria due to associated neurologic or orthopedic disorders, visual acuity defects were excluded. The upper limb Motor Index was obtained, with the assessment of the strength of five muscle groups of the right upper limb evaluated by the clinical staff. Each muscle group can be evaluated between 0 (no function)-5 (normal function) with a total of 25 points. All patients signed an informed consent form before the study. The guidelines of the declaration of Helsinki were followed in every case and the study design was approved by the local ethics committee. Subject demographics are provided in Table 1.
2.4. Experimental Setup
The study is carried out in a single experimental session by the performance of both BBT, real and virtual, with the dominant arm. The real Box and Block is performed firstly. The subject was seated in front of an adaptable table in height, and in front of him stands the wooden box with all the blocks located on the side of the dominant or less affected hand, which is with which it will be performed in the first place. The time counter starts, and for 15 seconds the subject performs a test that allows him to practice with the test, when this time ends, the cubes are returned at the starting position and the timer starts again, this time for 60 seconds, counting all cubes that pass correctly to the other compartment, taking into account the above restrictions.
Once the test is finished, the participants took a five-minute rest period between the two tests. Then, the virtual test is performed. Before starting, there is an initial practical period, in which the subject has 2 one-minute tests to make him feel comfortable in the virtual environment. The 15-second trial period is
![]()
Table 1. Demographic characteristics of the sample analyzed.
a(p < 0.01); *categorical variables are expressed as frequency and percentage; +continuous variables are expressed as mean and standard deviation.
then started with the dominant hand or less affected with the virtual scale, followed by the 60 seconds with an automatic count of the cubes or blocks correctly carried.
2.5. Data Analysis
To determine the correlation index between the number of cubes moved in the real BBT and that in the virtual BBT, Pearson’s correlation coefficient was used. Correlations between 0 and 0.25 were considered low, those between 0.25 and 0.5 were considered fair, those between 0.5 and 0.75 moderate; and those greater than 0.75 were considered strong. Moreover, the possible differences in the two test performance between the healthy and patients groups were analyzed by using the U Mann-Whitney non-parametric test and the possible differences between the two BBT performances were analyzed by the application of Wilcoxon test.
The data analysis was done on the software SPSS for Windows version 17.0 (SPSS, Chicago, IL, USA).
3. Results
The sample analyzed was matched in age and gender between the two groups analyzed, healthy and patients with neurological disorders (Table 1). Therefore, no statistically significant differences were found between both groups in these demographical variables. However, in relation to the Motor Index variable (dominant arm), significant differences were found between both populations (p < 0.01).
In relation to the motor performance in the Box and Block Test (real and virtual versions), all the results are shown in Table 2. Taking into account the real test, the motor UE performance was statistically higher in healthy population (73.50 (9.81)) than in patients (36.58 (15.20)) (p < 0.01). The same behavior was observed in the virtual version of the BBT (25.16 (6.19) > 13.16 (5.45), p < 0.01).
Moreover, within each group analyzed, the performance in the real test was higher than those in the virtual version (p < 0.01). For the healthy group, 73.50 cubes were obtained in the real test, versus 25.16 in the virtual version. In the case of patients group, 36.58 cubes were obtained in the real test, versus 13.16 in the virtual version.
![]()
Table 2. Performance obtained in the BBT (real and virtual versions) for the two groups analyzed.
Results are expressed as mean and standard deviation. a,b,c,d(p < 0.01); a,b(application U Mann-Whitney Test); c,d(application Wilcoxon test).
These results were observed graphically in the box plots (Figure 3). The statistical significances obtained were reflected in the corresponding diagram, taking into account that there is no overlap between any box plot. Moreover, graphically, it can be observed that the difference between both tests (real and virtual version) is minor in patients group than in the healthy population.
The results in relation to the dispersion analysis are shown in Figure 4. So, a higher performance in the real test corresponds to a higher performance in the virtual BBT with a correlation coefficient of 0.801. So, there is a high trend between both tests the real and the virtual version (Figure 4).
4. Discussion
In this work a virtual application of the Box and Block test has been presented and developed to analyze the relationship between the performance in the real BBT and this virtual version in a group of healthy people and patients who had suffered neurological diseases. Our results obtained suggest that the trend between the results of both tests is maintained, so that a higher performance in the real test corresponds to a higher performance in the virtual test.
The loss of skills achieved after intensive rehabilitation programs in patients with neurological sequelae if some training is not maintained is well established. Hence, the need to develop low-cost home devices for training and maintenance of manual dexterity that include assessment tools such as the one presented here [11]. In relation to the neurological pathologies we used for validation purposes, 10 patients suffered a cervical SCI, one patient suffered a Guillain-Barré syndrome and another suffered sequelae of an infectious meningen cephalomeningitis. Our patient series included mostly SCIs, but regardless of the etiology, all of the patients included had sensorimotor upper limb deficits and, because of them, lack of ability in reaching and grasping with the hand selected.
Interestingly, the significantly smaller difference in the virtual performance between healthy and patients groups compared to the real one, could be due to the amount of sensory information linked to the touch sense, weight of the cubes and joint displacement of the hand supporting them throughout the task with which healthy individuals count, but which patients lack due to the sensory deficits depending of their respective injuries. If our interpretation is true, this
![]()
Figure 3. Box plots related to the motor performance in both tests for the healthy group and patients group.
![]()
Figure 4. Dispersion diagram obtained between the performance in both tests for the healthy and patients population in conjunction.
would mean that virtual BBT might be able to measure motor aspects detached from most of the sensory feedback, but it would also mean that to properly train sensitive remnants in each particular patient, it would be necessary to incorporate information from other sensory modalities, mainly haptic components.
This study has a high potential application taking into account the clinical experiences published by using the real BBT as a tool for the assessment of manual dexterity in several neurological pathologies such as cerebral palsy, and more frequently stroke [1]. Previous studies that have also presented a virtualization of BBT, were mostly focused on stroke pathology [6] [7] [12] and have not shown a dispersion analysis with the actual test. Therefore, the results obtained in the correlation analysis carried out between both the actual and virtual tests could not be contrasted with other studies. For example, Oña et al. developed a virtual version of BBT providing an automatic count of the blocks by using Microsoft Kinect [12]. Other previous study proposed a virtualization of the BBT to compare the participants’ performance and acceptability level between the real and the virtual test. However, this study based on Leap Motion Controller was only conducted in 12 healthy subjects [13].
In addition, the objective application of this work incorporates the necessary sources of feedback so that the patient is aware of his own improvement in consecutive sessions that he can perform. The progression bar indicates the patient’s performance relative to a reference pattern made up of a group of healthy people. Thus, 100% of the task is calculated as the average of cubes that passed a group of healthy people at that time and using a specific type of clamp. In order to avoid the ceiling effect it is given the option to obtain a result superior to the reference pattern, thus being able to exceed 100%. Other sources of visual feedback are the number of cubes that pass to the other side of the partition and the time counter that controls the task in countdown.
However, this paper also has limitations. Some of them are inherent to the technology used, the Leap Motion, whose performance is strongly conditioned by the conditions of the environment and by the performance of the computer on which it runs. The other limitations are due to the development of the virtual environment itself. An example is the virtual object that corresponds to the wooden box. Initially it was wanted for the wooden box to be controlled by the physics engine, like the cubes, so that the virtual hand model would find opposition to the box, that is, that it cannot pass through it and that it grabs the cubes as physical objects, but, after contacting the developers of Leap Motion to discuss the possibility of putting obstacles in the scene that the hand can’t pass through we informed us that it is not designed for it because it would break the presence of the hand, so the alternative was to make the box kinematic. With this type of objects, the virtual hand was enabled to interact with the box, so the hand could pass through it. It was chosen to develop this alternative, since the cubes will have physical properties and that forces users to perform the complete movement, because otherwise, the cube will not pass through the walls of the box. Finally, the second option was implemented.
5. Conclusion
The virtual version of the BBT developed in this work has shown high consistency in its application to a sample of healthy people and patients with neurological diseases. The following step is that this virtual test will serve as an element to evaluate in the future the effectiveness of the RehabHand prototype, based on a set of virtual applications with a therapeutic sense that, manipulated through the Leap motion device, will allow training manipulative dexterity in patients with neurological diseases.
Acknowledgements
This research has been funded by grant from the Spanish Ministry of Economy and Competitivity and cofunded from FEDER, National Plan for Scientific and Technological Research and Innovation. Project RehabHand (Plataforma de bajo coste para rehabilitación del miembro superior basado en Realidad Virtual, ref. DPI2016-77167-R) and from MINECO and the Fondo Europeo de Desarrollo Regional (MAT2016-78857-R, AEI/FEDER, UE).