This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on //www.mybigtv.com/, as well as this copyright and license information must be included.
An established and well-known method for usability assessment of various human-computer interaction technologies is called heuristic evaluation (HE). HE has been adopted for evaluations in a wide variety of specialized contexts and with objectives that go beyond usability. A set of heuristics to evaluate how health information technologies (HITs) incorporate features that enable effective patient use of person-generated health data (PGHD) is needed in an era where there is a growing demand and variety of PGHD-enabled technologies in health care and where a number of remote patient-monitoring technologies do not yet enable patient use of PGHD. Such a set of heuristics would improve the likelihood of positive effects from patients’ use of PGHD and lower the risk of negative effects.
This study aims to describe the development of a set of heuristics for the design and evaluation of how well remote patient therapeutic technologies enable patients to use PGHD (PGHD enablement). We used the case of Kinect-based stroke rehabilitation systems (K-SRS) in this study.
The development of a set of heuristics to enable better use of PGHD was primarily guided by the R3C methodology. Closer inspection of the methodology reveals that neither its development nor its application to a case study were described in detail. Thus, where relevant, each step was grounded through best practice activities in the literature and by using Nielsen’s heuristics as a basis for determining the new set of heuristics. As such, this study builds on the R3C methodology, and the implementation of a mixed process is intended to result in a robust and credible set of heuristics.
A total of 8 new heuristics for PGHD enablement in K-SRS were created. A systematic and detailed process was applied in each step of heuristic development, which bridged the gaps described earlier. It is hoped that this would aid future developers of specialized heuristics, who could apply the detailed process of heuristic development for other domains of technology, and additionally for the case of PGHD enablement for other health conditions. The R3C methodology was also augmented through the use of qualitative studies with target users and domain experts, and it is intended to result in a robust and credible set of heuristics, before validation and refinement.
This study is the first to develop a new set of specialized heuristics to evaluate how HITs incorporate features that enable effective patient use of PGHD, with K-SRS as a key case study. In addition, it is the first to describe how the identification of initial HIT features and concepts to enable PGHD could lead to the development of a specialized set of heuristics.
Person-generated health data (PGHD) are produced from technologies that allow people to access and utilize health data that they themselves generate outside of a health clinic setting, and to share these data with health care providers and others, typically via the internet. These technologies are designed to generate data about a range of health conditions and in pursuit of a range of health outcomes for remote patient monitoring [
It is possible for PGHD utilization to have positive, negative, or nil effects on patients who use these technologies. PGHD use has been reported to provide patients with a range of benefits, such as increasing interest in their own health care processes [
The varying effects of PGHD use on patients necessitates the development of a patient-reported outcome measure (PROM) of utilizing PGHD, or PROM-PGHD [
The authors previously developed a PROM-PGHD that entailed establishing health outcomes that should be measured for a specific technology category and a health condition [
Using a list of features based on predetermined criteria to assess technology categories is called heuristic evaluation (HE). First introduced by Nielsen and Molich [
HE has since been adopted for evaluations in a wide variety of specialized contexts and with objectives that go beyond usability. This has resulted in specialized heuristics, which could be used to direct evaluators to focus on assessing technologies according to specific concepts of interest [
Developing a set of heuristics for use in the formative stages of development of patient and consumer health technologies could ensure more deliberate PGHD-enabling designs. This would improve the likelihood of positive effects from patients’ use of PGHD and lower the risk of negative effects. Such heuristics would also offer health care providers a standard way to evaluate technologies as part of selecting and implementing PGHD programs with their patients [
In this study, we describe the development of a set of heuristics for the design and evaluation of how well remote patient therapeutic technologies enable patients to use PGHD (PGHD enablement). We used the case of Kinect-based stroke rehabilitation systems (K-SRS) to illustrate how the development of PROM-PGHD has led to the creation of these heuristics.
An important use case for evaluating PGHD effects is in simulated rehabilitation technologies for stroke, in particular those using body-tracking technology Kinect (Microsoft) [
K-SRS offer a more convenient and cost-effective option [
A number of heuristic development processes have been employed in the literature. This includes literature reviews, analysis of usability problems, mixed processes, use of guidelines, interviews, and analysis of theories related to a specific domain [
The development of a set of heuristics for PGHD enablement was primarily guided by the R3C methodology most utilized and applied in the literature [
Thus, the development of a specialized set of heuristics for K-SRS technologies was guided by the R3C [
The steps of R3C are described in
Steps of the R3C methodology to develop a specialized set of heuristics [
Step 1: Exploratory stage
Examine literature related to the main topics or technology domain of the research.
Step 2: Descriptive stage
Highlight key concepts from step 1. Connect the information, and assign weights.
Step 3: Correlational stage
Identify the characteristics that the heuristics should have. Original heuristics [
Step 4: Specification stage
Formally specify the set of proposed specialized heuristics, using a standard template. Prioritize heuristics, and report any missing elements that need to be added.
Step 5: Validation or experimental stage
The new set of heuristics is tested on the target domain, and usability problems or issues found are compared with those that would be found using traditional heuristics.
Step 6: Refinement stage
The new set of heuristics is refined or improved following analysis of the results of step 5. However, this occurs iteratively, that is, if no more changes are recommended after step 5 then the development ends there.
A literature review was conducted as part of the PROM-PGHD development process to understand how K-SRS have been designed to enable PGHD and what benefits, if any, were recorded [
在上下文中描述的文献综述which PGHD is produced and used, relevant for step 4 when the heuristics are specified. As part of this step, factors that may influence the outcomes resulting from accessing and using PGHD were analyzed. The review has shown that PGHD was given to patients to guide them, for example, change movement behavior or actions to perform correct exercise movements. PGHD were also provided to help them achieve their short- and long-term goals [
The review also highlighted the lack of patient-centered design in the development of K-SRS, given that the focus of providing data back to patients was for prescribing tasks and not to allow patients to access and make sense of their PGHD themselves. This is a missed opportunity to encourage patients to be more involved in their own health care [
This step is described as having 3 tasks: highlighting, connecting, and assigning weights to the concepts found in step 1. Step 2 is applied for this study as follows. First, concepts found through the literature review that could enable the effective patient use of PGHD were identified. Second, the concepts were categorized based on their inherent meaning according to 5 different reported effects of PGHD use thematically derived from the literature. Articles from a significant journal special issue on PGHD were inductively analyzed as an efficient and targeted way to identify reported effects of PGHD utilization from a range of HITs for different health conditions [
In the third activity of this stage, the concepts were assigned weights through qualitative studies. As part of the PROM-PGHD development process, the input of target users (stroke survivors) and experts (physiotherapists) was elicited through focus groups and interviews [
The qualitative studies enabled the authors to receive direct feedback from PGHD users to assign weights to the concepts found from step 1. In addition, the studies also bridged any relevant gaps between the concepts described in the literature, those perceived by clinicians, and those reported as important by stroke survivors themselves [
Concepts identified through the literature review but not described by either stroke survivors or clinicians were assigned a weight of 0, concepts described by clinicians only were assigned a weight of 1, concepts described by stroke survivors only were assigned a weight of 2, and concepts described by both stroke survivors and clinicians were assigned a weight of 3.
The categories were initially applied by GD and then independently reviewed by the coauthors. Any disagreements in the categories applied were discussed in a meeting until agreement was reached for each concept. More example quotes are shown in
Weighted key concepts identified from step 1: exploratory stage.
Categories | Key concepts | Weight |
Stroke survivor health-related behaviors | PGHDacan guide stroke survivors to make appropriate movement behavior or action changes to perform an exercise correctly |
2—stroke survivors: “I automatically adjust what I was doing, if I couldn't burst every balloon I, I had to adjust… To work out what I was doing wrong” (FG1_STC2-3) |
Stroke survivor personal care goals | PGHD can help stroke survivors achieve their short- and long-term goals | 0—Not described |
Functional effectiveness of therapy | PGHD can help inform clinicians, to analyze the functional effects of simulated rehabilitation therapy, and tailor programs for stroke survivors | 0—Not described |
Evaluation of the PGHD-enabled K-SRSb | PGHD can be used to assess the effectiveness and reliability of a K-SRS, compared with other types of simulated rehabilitation technologies | 0—Not described |
Stroke survivor interest in care processes |
PGHD may be remembered by stroke survivors over time and can provide them with more motivation to improve their therapy performance PGHD needs to be accessible to the stroke survivors who produce them to allow them to be more involved in their own health care |
3—stroke survivors and clinicians: PGHD可能箴言我de “an extra percentage of motivation” (FG1_STC2-1) and an “incentive to do better” (FG1_STC2-3) 医生:“继续努力”(FG3_AHC1_3) 3—stroke survivors and clinicians: “it helped me understand my rehabilitation progress” (INT1_HWC1) “quite often you learn more when you've done something wrong” (FG2_AHP_2) Clinician: “[survivors] would understand better after a second session where they could compare their results...it's good for them to have a comparison, to see how they've changed from one session to another” (FG3_AHC1_5) |
aPGHD: person-generated health data.
bK-SRS: kinect中风康复系统。
Weighted new concepts identified from qualitative studies.
Categories | Key concepts | Weight |
Stroke survivor health-related behaviors |
PGHDacan encourage stroke survivors to do more exercises related to their therapy PGHD can discourage stroke survivors from doing more exercises if it is negative or low |
3—stroke survivors and clinicians: “if I can see the improvement I'm making then it would...encourage me to maybe have more of those sessions” (INT1_STC3) Clinician: “that might stimulate them to be more compliant” (FG3_AHC1_5) 3—stroke survivors and clinicians: “probably lose faith in the system” (INT2_AHP) [unless there is] “some explanation [...] that you need to consider looking” (INT2_AHP) Clinician: “They might get over it, or might not be willing to participate” (FG3_AHC1_3) |
Stroke survivor personal care goals |
PGHD can demotivate stroke survivors if it is negative |
2—stroke survivors: “that can be a positive motivator, but can also (be) a negative one” (INT2_STC1) “I'm thinking well, are normal people at a hundred percent? And I'm only at 60?” (FG1_AHP_2) |
Feelings about health status |
PGHD can cause positive or negative emotions, correlated with whether their PGHD is positive or negative PGHD can make stroke survivors feel confused about their health progress PGHD can make stroke survivors feel more self-aware about their health care |
3—stroke survivors and clinicians: “going backwards, that would be a little bit depressing” (INT1_AHP) seeing “yourself gradually making improvements, it just makes you feel so much better. Okay, I'm achieving something” (INT1_AHP) Clinician: “a score can motivate you or please you” (FG3_AHC1_5) 3—stroke survivors and clinicians: “the percentages to me is more difficult to understand [...] are we trying to be a hundred percent at these things?” (FG1_AHP_2) Clinician: “scope to put in something about, I didn't really understand” (FG3_AHC1_4) 2—stroke survivors: “to measure your improvement...or measure your, deterioration...You could see it in cold hard figures.” (FG2_AHP_2) |
Stroke survivor interest in care processes |
PGHD can interest stroke survivors in how their exercises are contributing to their activities of daily living PGHD can help stroke survivors to self-manage their energy while undergoing therapy PGHD can affect stroke survivors’ perception about their therapy |
1—clinicians: “any relevance to a functional activity. You know like washing the dishes, or hanging the washing out or...climbing a flight of stairs” (FG3_AHC1_4) 2—stroke survivors: “It was certainly something that I watched, to see where I was at. 'Cause you need to think about this, we have some sort of a budget of energy that you have to manage yourself, and you can't afford to get to empty” (FG1_STC2-3) 3—stroke survivors and clinicians: Mismatch between PGHD and feeling of performance: “you probably think oh, the system's not doing its job” (INT2_AHP) “nothing going on in the background about anyone judging me...that there isn't anything that's being kept from me” (FG1_AHP_2) Clinician: if you just did the computer thing and they...just stopped without data they'd be like, why do I do it?” (FG3_AHC1_2) |
Stroke survivor relationship with care provider(s) |
PGHD can prompt stroke survivors to contact their therapists about their therapy performance PGHD can make stroke survivors be more conscious of the exercises prescribed by their clinicians |
3—stroke survivors and clinicians: “If they were always bad then I would need more assistance and even if they [were] good, they [are] not perfect, right so I would want to have more, more assistance to improve” (INT2_STC3) Clinician: how can I get a better score...why did I...not do very well” (FG3_AHC1_4) 1—clinicians: “maybe if they're not being compliant, they might get a phone call from the therapist” (FG3_AHC1_5) |
Relationship with family and carers |
PGHD can assist stroke survivors in communicating their rehabilitation progress with their loved ones |
3—stroke survivors and clinicians: “you could show them something, it's easier for them to visualize” (INT1_AHP) Clinician: “share it with family...it's that...bragging power as well, perhaps” (FG3_AHC1_4) |
aPGHD: person-generated health data.
In this stage, the characteristics that the new set of heuristics should have are defined, which are later specified in step 4.
The key concepts identified through the first 2 steps are used as the basis to define the characteristics. As opposed to concepts, characteristics read as features that K-SRS should have to enable stroke survivors to use PGHD. These characteristics are also matched with the original heuristics [
The characteristics were initially defined by GD and then independently reviewed by the coauthors. Any disagreements were discussed in a meeting until agreement was reached for each characteristic.
Characteristics defined from the key concepts identified from steps 1 and 2—that match an original heuristic.
Number | Characteristics | Reworded using the original heuristic as a guide |
1 |
PGHDa-enabled systems should assist users in performing more exercises or actions, in a correct way. Matched with original heuristic 9. |
Help stroke survivors in performing more exercises or actions and to recognize, understand, and recover from errors they make. Guidance or error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution. |
2 |
PGHD-enabled systems should provide PGHD to stroke survivors for increased understanding about their rehabilitation or therapy process. Matched with original heuristic 1. |
PGHD-enabled systems should always provide PGHD to stroke survivors to keep them informed about what is going on with their health status, through appropriate feedback within reasonable time. This would increase their understanding about their rehabilitation or therapy process. |
3 |
PGHD-enabled systems should avoid formatting PGHD through a scale that represents a completeness or an endpoint, for example, 100% as much as possible, as it would likely represent failure. Instead, PGHD should resemble the ongoing functional therapeutic progress of stroke survivors. Matched with original heuristic 2. |
PGHD-enabled systems should present PGHD in a format that matches the real-world context, therapy progress, and goals of the stroke survivors. |
4 |
PGHD-enabled systems should ensure that PGHD is, or could be, presented in a way that is clearly understandable to a stroke survivor. Matched with original heuristic 2. |
PGHD-enabled systems should ensure that PGHD is clearly understandable to stroke survivors. The system should speak their language, with words, phrases, and concepts familiar to them, rather than system-oriented terms. PGHD-enabled systems should follow real-world conventions, making information appear in a natural and logical order. |
5 |
PGHD-enabled systems should provide PGHD to stroke survivors for increased self-awareness about their health care. Matched with original heuristic 1. |
PGHD-enabled systems should always provide PGHD to stroke survivors to keep them informed about what is going on with their health status, through appropriate feedback within reasonable time. This would increase their self-awareness about their health care. |
6 |
PGHD-enabled systems should provide patients real-time PGHD that allow them to self-manage their energy while performing therapy exercises. Matched with original heuristic 1. |
PGHD-enabled systems should always provide PGHD to stroke survivors to keep them informed about what is going on with their health status, through appropriate feedback within reasonable time. This would allow them to self-manage their energy while performing therapy exercises. |
7 |
Notwithstanding the need for PGHD to be as accurate as possible, PGHD-enabled systems should inform the patients of its limitations or potential inaccuracies in the PGHD produced by stroke survivors. PGHD-enabled systems should also provide PGHD to stroke survivors to foster an increased sense of trust about their rehabilitation or therapy process. Matched with original heuristic 1. |
PGHD-enabled systems should always provide PGHD to stroke survivors to keep them informed about what is going on with their health status, through appropriate feedback within reasonable time. This would foster an increased sense of trust about their rehabilitation or therapy process. |
8 |
PGHD-enabled systems should allow stroke survivors to contact their/a clinician about their PGHD or at least provide survivors with the option of viewing functional, action-based suggestions for them to improve their performance. Matched with original heuristic 10. |
Even though it is better if the system can be used without additional help or documentation, it may be necessary to provide them. PGHD-enabled systems should provide stroke survivors the option to contact a clinician about their PGHD and vice versa or at least provide survivors with the option of viewing functional, action-based suggestions for them to improve their performance. Any such information should be easy to search, focused on the survivors’ exercises, list concrete steps to be carried out, and not be too lengthy. |
9 |
survivo PGHD-enabled系统应该提供中风rs the option of allowing their clinicians to contact them based on the progress of their PGHD. Matched with original heuristic 10. |
Even though it is better if the system can be used without additional help or documentation, it may be necessary to provide them. PGHD-enabled systems should provide stroke survivors the option to contact a clinician about their PGHD and vice versa or at least provide survivors with the option of viewing functional, action-based suggestions for them to improve their performance. Any such information should be easy to search, focused on the survivors’ exercises, list concrete steps to be carried out, and not be too lengthy. |
aPGHD: person-generated health data.
10
Person-generated health data (PGHD)–enabled systems should present PGHD that indicate negative or decreasing therapy progress carefully and in a form that elicits a stroke survivors’ competitiveness with the self.
11
PGHD-enabled systems should highlight PGHD indicating positive or improving therapy progress more and providing them with more frequency than negative or decreasing progress.
12
PGHD-enabled systems should provide PGHD to stroke survivors for increased understanding about how their rehabilitation is contributing to their functional ability.
13
survivo PGHD-enabled系统应该提供中风rs with the option to share their PGHD with loved ones in a secure manner.
In this stage, the characteristics defined in step 3 are specified as a heuristic, following a structured format: ID, name, definition, explanation of how the heuristic was developed, example(s) of when a system being evaluated complies with or violates the heuristic, expected benefits if the system complies with the heuristic, and anticipated problems of heuristic misunderstanding [
The ID applied to the heuristics was structured as [Number]-PGHD-W[Weight]. The weights were indicated to provide implementers an idea of the process that went through developing the heuristics. The heuristic names were written succinctly, in a similar fashion to the original heuristics [
A number of characteristics were either similar or tightly complemented each other and were therefore combined into one heuristic. Characteristics 3 and 12 described the need for K-SRS to match the health and therapy context of the stroke survivors. Characteristics 5, 2, 6, and 7 described the need for K-SRS to always provide PGHD to stroke survivors. Finally, characteristics 8 and 9 both described the need for K-SRS to provide options for stroke survivors to seek more information, contact their clinicians, and allow their clinician to contact them.
这个步骤进行必要的扩充. In combining the characteristics, considerations had to be taken with the individual weights and the categories of those characteristics. Where the weights of the characteristics being combined were different, the highest weight was indicated for the resulting heuristic’s ID. Meanwhile, the categories were added as tag(s) under each heuristic and reworded to indicate how the heuristic informs implementers to enable PGHD use. This preserves the categories identified in step 2 as metadata that could aid implementers in understanding the concepts underlying each heuristic. Interestingly, the R3C methodology did not describe how the categories were relevant to the resulting heuristics [
The heuristics were initially specified by GD and then independently reviewed by the coauthors. Any disagreements were discussed in a meeting until agreement was reached for each specified heuristic. Moreover, steps 5 and 6 are expected to refine the example, expected benefits, and anticipated problems of each heuristic. Therefore, the heuristics presented here may still be revised later on [
The next section shows the new, initial set of heuristics for PGHD enablement.
八个新的启发式K-SRS PGHD支持were created. Heuristics with higher weights are presented first, to highlight heuristics that should be prioritized. The first 6 heuristics have a weight of 3, and the last 2 heuristics have a weight of 2. There are 6 tags used: improve health-related behaviors, increase positive feelings about health status, facilitate positive personal care goals, increase interest in care processes, improve relationships with care providers, and improve relationships with family and caregivers.
For brevity, only the ID, name, tag(s), definition, and explanation are presented in
1-PGHD-W3: Encouraging person-generated health data (PGHD)
Tag(s): Improve health-related behaviors; increase positive feelings about health status
Definition: The system should highlight PGHD indicating positive or improving therapy progress and provide them with more frequency than negative or decreasing progress.
Explanation: This heuristic was formed from 3 concepts identified through a literature review and confirmed by stroke survivors and clinicians. Stroke survivors and clinicians commented on how PGHD can cause positive or negative emotions, correlated with whether the PGHD is positive or negative, for example, when survivors frequently see their PGHD improve, it encourages them to do more of the exercises. A clinician described how seeing PGHD could help survivors be more compliant. On the other hand, if survivors see their PGHD decline, they might lose faith in the system and not perform their exercises.
2-PGHD-W3: Evoking competitiveness with self
Tag(s): Facilitate positive personal care goals
Definition: The system should present PGHD that indicates negative or decreasing therapy progress carefully and in a form that elicits a stroke survivors’ competitiveness with the self.
Explanation: This was a concept identified through a literature review and confirmed by stroke survivors and clinicians. Stroke survivors described how seeing their PGHD can motivate them to try harder, sometimes even when their PGHD indicates that they are not doing very well with their therapy exercises, because it can feel like a competition with themselves. Clinicians agree that it can give survivors something to keep them striving to be better.
3-PGHD-W3: Understandable health data
Tag(s): Increase positive feelings about health status
Definition: PGHD-enabled systems should ensure that PGHD is clearly understandable to stroke survivors. The system should speak their language, with words, phrases, and concepts familiar to them, rather than system-oriented terms. They should follow real-world conventions, making information appear in a natural and logical order.
Explanation: This was a concept identified through a literature review and confirmed by stroke survivors and clinicians. A stroke survivor found that simply having percentages as feedback for performing an exercise could be confusing, as it could be misunderstood as being compared with people who have not had stroke. Clinicians also agreed that PGHD may not be understood by survivors.
4-PGHD-W3: Visibility of health progress
Tags: Increase positive feelings about health status; increase interest in care processes
Definition: PGHD-enabled systems should always provide PGHD to stroke survivors to keep them informed about what is going on with their health status, through appropriate feedback within reasonable time.
Explanation: This heuristic was formed from 4 concepts identified through a literature review and confirmed by stroke survivors and clinicians. Stroke survivors described how PGHD could help them understand their rehabilitation progress, to determine whether their functional progress was headed in the right direction and learn when they have made a mistake. PGHD could also help them self-manage their energy as they perform their exercises and increase their trust in their therapy process.
5-PGHD-W3: Help and support
Tag(s): Improve relationship with care providers
Definition: Even though it is better if the system can be used without additional help or documentation, it may be necessary to provide them. PGHD-enabled systems should provide stroke survivors the option to contact a clinician about their PGHD; allow their clinicians to contact them; or at least provide survivors with the option of viewing functional, action-based suggestions for them to achieve their therapy goals. Any such information should be easy to search, focused on the survivors’ exercises, list concrete steps to be carried out, and not be too lengthy.
解释:这种启发式方法是由2浓缩的epts identified through a literature review and confirmed by stroke survivors and clinicians. Stroke survivors described how they would want to contact their therapists, particularly if they have not been doing well to ask for assistance and ask for advice on what they could do better. Clinicians also described how it would be good if therapists could contact survivors when they are not being compliant. They also described how stroke survivors would likely ask for more explanation and for advice on how they can do things differently to improve their PGHD.
6-PGHD-W3: Communication of health data
Tag(s): Improve relationship with family and carers
定义:PGHD-enabled系统应该提供圣roke survivors with the option to share their PGHD with concerned parties, for example, loved ones, in a secure manner.
Explanation: This was a concept identified through a literature review and confirmed by stroke survivors and clinicians. Stroke survivors described how it could help them communicate their health status to people they would like to share it with, particularly when they are tired or when they “run out of words.” Clinicians described how survivors might want to share their PGHD with family, particularly with the younger generation who are more technologically inclined.
7-PGHD-W2:指南正确运动或锻炼actions
Tag(s): Improve health-related behaviors
Definition: The system should help stroke survivors in performing more exercises or actions and to recognize, understand, and recover from errors they make. Guidance or error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
Explanation: This was a concept identified through a literature review and confirmed by stroke survivors. A stroke survivor commented on how PGHD can help them work out what they were doing wrong and adjust accordingly.
8-PGHD-W2: Match between system PGHD and real-world context of stroke survivors
标签:促进积极的个人护理目标;我ncrease interest in care processes
Definition: PGHD-enabled systems should present PGHD in a format that matches the real-world context, therapy progress, and goals of the stroke survivors. When presenting PGHD, for example, as a score, percentage, or a graph, they should be matched to the therapy goals of the survivors. These goals may change over the course of a survivors' therapy, from gaining functional progress in the first few months or years to maintaining function to perform ADL when their progress starts to plateau [
解释:这种启发式方法是由2浓缩的epts identified through a literature review and confirmed by stroke survivors and clinicians. Stroke survivors described how receiving a percent score that did not indicate a 100% could be a negative motivator. Achieving 100% given their condition could be very difficult or even impossible. Clinicians also described how PGHD should help survivors understand how their therapy is contributing to improved performance of ADL.
This study is the first to develop a new set of specialized heuristics to evaluate how HITs incorporate features that enable effective patient use of PGHD, with K-SRS as a key case study. In addition, it is the first to describe how the identification of initial HIT features and concepts to enable PGHD could lead to the development of a specialized set of heuristics. As such, it uncovers a valuable dimension to the PROM-PGHD development method, which considers the sociotechnical context of HITs [
Subsequently, this study highlights the value of considering the sociotechnical context of HITs in their design and evaluation. Although unexpected consequences and even failures can occur from design flaws and technical limitations, they are also often the result of sociotechnical factors [
The R3C methodology [
A step-by-step implementation of the R3C methodology was presented. A systematic and detailed process was applied in each step of heuristic development, which bridged the gaps described earlier. It is hoped that this would aid future developers of specialized heuristics, who could apply the detailed process of heuristic development for other domains of technology, and additionally for the case of PGHD enablement for other health conditions. Throughout the process, it was observed that the concepts and characteristics may need to be combined. This study presents a way to retain the weights and categories applied to them as heuristic metadata to aid implementers in understanding their underlying concepts. Although the last 2 heuristics have a weight of 2, implementers should still aim to conduct the evaluation using all 8 heuristics. However, in case of any resource constraint, the first 6 heuristics should be prioritized.
The R3C methodology [
Data gathered from the qualitative studies were used as an important augmentation in the heuristic development process. However, as those studies were primarily meant to support the development of a PROM-PGHD, it is possible that participants were not asked all the relevant questions around PGHD enablement features. In the event, many concepts found through the literature review were supported by qualitative studies, and indeed, most of the resulting heuristics were formed from new concepts found in those studies. Therefore, although it might appear to be a limiting factor, it actually indicates the robustness of the process followed and highlights the importance of this suggested augmentation. Moreover, the validation stage (step 5) still allows for possible additional heuristics to be identified.
The flexibility of the R3C methodology [
The new set of heuristics for PGHD enablement, following a detailed, systematic development process augmented from best practice that we have presented, could serve as a guide for future developers of specialized heuristics in general and specifically for developers of heuristics for PGHD enablement of a variety of technology domains and health conditions. The new set of heuristics is needed in a period of rising demand for, supply of, and variety of PGHD-enabled technologies in health care. It offers health care providers a standard way to evaluate technologies as part of selecting and implementing PGHD programs with their patients [
Steps 1-4, transforming concepts to heuristics.
activities of daily living
heuristic evaluation
health information technology
Kinect-based stroke rehabilitation system
person-generated health data
patient-reported outcome measure
patient-reported outcome measure of utilizing person-generated health data
GD would like to acknowledge the Melbourne School of Engineering through which his research scholarship is provided and his organizational sponsor—Newman College (University of Melbourne).
None declared.