User Testing of an Adaptation of Fishbone Diagrams to Depict Results of Systematic Reviews
Transcription
Gartlehner et al. BMC Medical Research Methodology (2017) 17:169DOI 10.1186/s12874-017-0452-zRESEARCH ARTICLEOpen AccessUser testing of an adaptation of fishbonediagrams to describe results of systematicreviewsGerald Gartlehner1,2*, Marie-Therese Schultes2,3, Viktoria Titscher2, Laura C. Morgan1, Georgiy Five. Bobashev1,Peyton Williams1 and Suzanne L. West1AbstractBackground: Summary of findings tables in systematic reviews are highly informative but require epidemiologicaltraining to be interpreted correctly. The usage of fishbone diagrams as graphical displays could offer researchers aneffective approach to simplify content for readers with limited epidemiological training. In this newspaper nosotros demonstratehow fishbone diagrams can be applied to systematic reviews and nowadays the results of an initial user testing.Methods: Findings from two systematic reviews were graphically depicted in the class of the fishbone diagram. Totest the utility of fishbone diagrams compared with summary of findings tables, we developed and pilot-tested anonline survey using Qualtrics. Respondents were randomized to the fishbone diagram or a summary of findings tablepresenting the same body of prove. They answered questions in both open-ended and closed-answer formats; allresponses were anonymous. Measures of involvement focused on first and second impressions, the power to observe andinterpret critical information, likewise as user experience with both displays. We asked respondents nearly the perceivedutility of fishbone diagrams compared to summary of findings tables. We analyzed quantitative data by conducting ttests and comparing descriptive statistics.Results: Based on real earth systematic reviews, we provide two different fishbone diagrams to show how they mightbe used to display complex data in a clear and succinct manner. User testing on 77 students with basicepidemiological preparation revealed that participants preferred summary of findings tables over fishbone diagrams.Significantly more than participants liked the summary of findings tabular array than the fishbone diagram (71.8% vs. 44.8%; p .01);significantly more participants establish the fishbone diagram confusing (63.2% vs. 35.9%, p .05) or indicated that it wasdifficult to find information (65.eight% vs. 45%; p .01). Yet, more than half of the participants in both groups wereunable to find critical information and answer three corresponding questions correctly (52.vi% in the fishbone group; 51.iii%in the summary of findings grouping).Conclusions: Fishbone diagrams are meaty visualizations that, theoretically, may prove useful for summarizing thefindings of systematic reviews. Initial user testing, however, did not support the utility of such graphical displays.Keywords: Evidence summary, Fishbone diagram, Systematic review, Visualization, Summary of findings, User testing* Correspondence: [email protected] International, 3040 Due east Cornwallis Rd, Inquiry Triangle Park, Durham,NC 27709, USA2Department for Evidence-based Medicine and Clinical Epidemiology, 3500Krems, AustriaFull list of writer information is available at the end of the article The Author(s). 2017 Open up Admission This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/past/4.0/), which permits unrestricted apply, distribution, andreproduction in any medium, provided you requite appropriate credit to the original author(southward) and the source, provide a link tothe Artistic Commons license, and indicate if changes were made. The Artistic Commons Public Domain Dedication o/1.0/) applies to the data fabricated available in this article, unless otherwise stated.
Gartlehner et al. BMC Medical Research Methodology (2017) 17:169BackgroundSystematic reviews assemble, describe, synthesize, andevaluate testify on a broad range of topics in healthcare, many of which are complex and multi-faceted. Thetraditional approach to presenting such complexity is todevelop multiple summary tables that describe the pattern of the studies, present results, and assess the qualityof evidence. Such tables are often dense and do notallow readers to grasp the findings "at a glance" simply instead require review of numerous pages of summary tables and large parts of the total testify report [ane].Summary of findings tables such as those proposed bythe Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Grouping [ii], havebecome of import tools to summarize results of systematic reviews and present the certainty of findings. Theysignificantly improve readers' overall understanding andtheir ability to find critical data compared withhaving data only in text [3].Even so, to be interpreted correctly, summary offindings tables require familiarity with some concepts ofclinical epidemiology and with grading the certainty ofthe evidence. To address such limitations, a simplification of the content using graphical displays could provide an fantabulous arroyo for enhancing presentation toreaders with limited epidemiological grooming [4].Fishbone diagrams were outset proposed by Kaoro Ishikawa [5] in the 1960s to display crusade and effect in thecontext of continuous improvement of industrial processes [6]. The diagram was offset termed an "Ishikawadiagram" merely was afterward dubbed "fishbone diagram" because of its resemblance to the skeleton of a fish—ahorizontal spine with the "caput" representing a problemor result with the "basic" emanating at acute anglesrepresenting causes. Software for amalgam fishbonediagrams is available within general graphics packagesPage 2 of 9such as Microsoft Visio (Microsoft Corporation, Redmond, WA, The states) or in specific packages, such every bit SmartDraw (https://www.smartdraw.com), which providesmore flexibility for drawing, modifying, and annotatingfishbone diagrams. Figure ane presents a generic fishbonediagram.Fishbone diagrams also have been widely used inhealth treat cause and effect analysis related to patient safe [vii], and for evaluating quality managementprocesses [8, 9].To our noesis, fishbone diagrams have not beenused to summarize the findings of health intendance research.Currently no presentation tool exists that offers a lesscomplex option than summary of findings tables yetmore in-depth methodological information than factboxes [10] that summarize the all-time available evidenceon the benefits and harms of treatments but are targetedtowards patients. Fishbone diagrams could mayhap fulfillthis need.In this manuscript, we explore the utility of fishbonediagrams to display the totality of a torso of evidencewith multiple outcomes. We envision that fishbone diagrams could exist effective tools for improving informeddecision making amidst readers of systematic reviewswith little epidemiological training. Theoretically, such agraphical technique could assist readers to comprehendthe conceptualization of the review and complex findings through a unproblematic, succinct, and understandable format. Determining the best format of communicatingrisks and benefits of healthcare interventions is of import because the correct interpretation of benign andharmful treatment effects is a prerequisite for informeddecision making.In the following sections, we first discuss the adaptation of the fishbone diagram using two real-world examples to show how systematic reviewers and others canFig. one Case of a generic Ishikawa fishbone diagram https://eatables.wikimedia.org/west/index.php?curid 6444290 (by Fabian Lange)
Gartlehner et al. BMC Medical Inquiry Methodology (2017) 17:169use this graphical technique. 2d, we summarize theresults of a user testing practise comparing a fishbonediagram with a Form summary of findings table.MethodsConcept of the fishbone diagram to summarize evidenceof systematic reviewsIn our adaptation of the fishbone graphic, the "head"represents the overall balance between benefits andharms of an intervention or of competing interventions.The bones of the fish stand for individual outcomes thatare critical or important to residuum the overall benefitsand harms of the interventions. The proximity of thebones to the head reflects the importance of outcomesfor decision making. GRADE, for example, recommendsranking the relative importance of outcomes when developing guidelines. Each os (representing an outcome) can include additional explanatory factors such asa plainly linguistic communication summary of the issue, the number ofstudies, the magnitude of handling effects, or other factors that influence the certainty (strength) of a body ofevidence.Nosotros applied the basic fishbone schematic to ii systematic reviews [11, 12] that differed in primal ways, so thatwe could illustrate the applications for various kinds oftopics, patient populations, interventions, and outcomes.To sympathize whether the fishbone could usefullysummarize comparative treatments that might form thebasis of a clinical exercise guideline, we took the findingsfrom a systematic review of randomized controlled trials(RCTs) of preoperative anemia direction versus usualcare ("anemia fishbone", Fig. two) [xi]. To evaluate the applicability and utility of fishbone graphics for preventiveservices, nosotros selected an umbrella systematic review onthe benefits and risks of mammography screening("screening fishbone", Fig. three) [12].For each fishbone diagram, we used and simplified information from the summary of findings table in eachsystematic review to populate the bones of the fish. Oneresearcher populated each fishbone diagram initially; asecond researcher evaluated the accurateness of the summary data. If discrepancies arose, the investigators arrived at a terminal version through consensus discussion oradjudication by a 3rd researcher.Target audienceWe envision equally the target audition for fishbone diagrams, health professionals with some epidemiologicaltraining just who are not facile with comprehension ofcomplex summary of findings tables.User testingThe goal of user testing was to compare the utility of afishbone diagram to convey results of a systematicPage iii of 9review with that of a Class summary of findings table.Nosotros conducted the user testing in English language in January2017 using Qualtrics, an electronic web-based surveytool. All answers were bearding. Additional File 1 presents the construction of the survey every bit a flow diagram.We airplane pilot-tested the survey on health services researchers and revised the final version accordingly; thepilot data were not included in the final analyses. Exceptfor initial impressions (see below) and demographicdata, all items in the survey were in a forced-choice format. To let for a mixed-methods approach in dataanalysis, we used both open-ended and closed answeringformats. The Danube University Institutional ReviewBoard determined that ethics blessing was not requiredfor anonymous user testing.We used the "anemia fishbone" described in a higher place andpresented in Fig. 2 every bit the example in the survey. TheGRADE summary of findings tabular array (Table 1) presentedmore detailed information than the fishbone diagram. Boosted File 2 presents the concluding version of the survey.SampleWe used a nonrandom, purposive sample of studentswho were enrolled in a health sciences available programor in a health management main's programme. All students had a basic understanding of clinical epidemiology.Because they had not been previously introduced to fishbone diagrams or summary of findings tables, nosotros provided a cursory introduction to the main concepts at thebeginning of user testing. Respondents completed usertesting via an online survey during regular course hours;however, participation was voluntary.MeasuresWe were interested in 4 singled-out measures for comparing the fishbone and the summary of findings table(Table 2): 1) Initial impressions; 2) Ability to discover and interpret disquisitional data; 3) Perceived utility; and four)2d impression later using either the fishbone diagram or the summary of findings table.To gather participants' starting time impressions, we askedthem to comment on their initial reactions when firstviewing the two displays by writing down iii spontaneous thoughts for the fishbone diagram and the summary of findings table. In add-on, we had participantsuse a visual analog calibration with a slider bar to compare thetwo displays based on five attributes ('easier to use', 'easier to understand', 'ameliorate designed', 'preferable', 'more disruptive'). The middle position indicated a neutral attitude.To continue with the questionnaire, the slider bars hadto be clicked on by the participants and either moved orleft in the neutral position.To assess the ability to find and interpret critical information, we randomized participants to work either with
Gartlehner et al. BMC Medical Research Methodology (2017) 17:169Page 4 of 9Fig. 2 Fishbone diagram of benefits and risks of preoperative anemia managementthe fishbone diagram or the summary of findings tabular array.The participants were given a brief summary text thatintroduced the topic (preoperative anemia direction)represented in the 2 displays. In a first pace, participants had to choose the correct conclusion that couldbe drawn from the display (meet Additional File ii for details). In a 2d stride, they had to reply threemultiple-choice questions that required finding andinterpreting information presented in the displays; only i reply per item was right. Participants could alsochoose an "I don't know" option. Because nosotros were alsointerested in the participants' speed of navigating thedisplays, nosotros tracked the time from loading the page toproviding the concluding answer for this section.Fig. three Fishbone diagram of benefits and risks of mammography screening vs. no screening
Gartlehner et al. BMC Medical Enquiry Methodology (2017) 17:169Page 5 of 9Table ane Summary of findings table of benefits and risks of preoperative anemia direction ofparticipants(studies)Follow-upOutcomesCertainty ofthe show(Grade)Reduction of Mortality follow up: range7 days to 30 days210 (iii RCTs) LOWQuality of Life(0 studies)Need for Blood Transfusions follow up: range7 days to 30 days304 (5 RCTs) MODERATEDuration of Hospital Stay follow upwardly: mean30 days74 (1 RCT)Thromboembolic Events follow up: range7 days to 30 days421 (4 RCTs) LOWRate of Infections(0 studies)Relativeeffect(95% CI)1–2 VERYLOW 1,3–1Anticipated absolute effectsRisk with no interventionRisk difference withpreoperative treatmentfor anemiaRR ane.5938 per g(0.48 to 5.31)23 more per thou(20 fewer to 160 more)notestimable0 fewer per 1000(0 fewer to 0 fewer)0 per 1000RR 0.78532 per 1000(0.61 to 1.02)117 fewer per chiliad(207 fewer to xi more)notestimable11.3 vs. thirteen.5 days(departure not statisticallysignificant)The mean duration ofhospital stay wasRR 1.710 per 1000(0.41 to seven.08)0 fewer per yard(2 fewer to forty more)notestimable0 fewer per grand(0 fewer to 0 fewer)0 per 1000*The run a risk in the intervention group (and its 95% conviction interval) is based on the assumed risk in the comparison grouping and the relative result of theintervention (and its 95% CI)CI: Conviction interval; RR: Take a chance ratioGRADE Working Grouping grades of evidenceHigh certainty: Nosotros are very confident that the true effect lies close to that of the estimate of the effectModerate certainty: Nosotros are moderately confident in the effect estimate: The true effect is likely to be close to the guess of the result, but in that location is a possibilitythat it is essentially differentLow certainty: Our confidence in the effect judge is express: The true outcome may exist substantially unlike from the estimate of the effectVery low certainty: We have very fiddling conviction in the effect guess: The true effect is probable to be essentially different from the guess of effect1Few events, studies practise not see optimal data size, confidence intervals encompass clinically important differences2Studies practice non run into optimal information size3High risk of bias of included trialAfter having worked with either the fishbone diagramor the summary of findings table, participants wereasked to evaluate the perceived utility of the respectivedisplay on several dimensions (e.g. 'By using the diagram, I can easily describe the risks and benefits of anintervention.') employing a half-dozen-point Likert calibration from'strongly disagree' to 'strongly agree'.To assess their "2nd impression" subsequently working withone of the displays, participants again used a slider barto compare the two displays based on the same v attributes as in the beginning of the survey ('easier to use','easier to empathise', 'better designed', 'preferable', 'moreconfusing'). We also asked participants which of the twodisplays they would recommend to a colleague, show toa patient, or propose to researchers to summarize the results of systematic reviews.Data analysisData were stored securely and were protected fromunauthorized access. Nosotros analyzed quantitative data byconducting t-tests and comparison descriptive statistics inIBM SPSS Statistics Insert for Windows version 24 (IBMCorp. Armonk, New York, USA). To analyze qualitativedata (participants' first impressions), we first translatedTable two The ten well-nigh oftentimes named associations at get-go sight of the fishbone diagram and the summary of findings tableFishbone DiagramSummary of Findings TableCommentRelative FrequencyCommentRelative %clear16.04%Clear5.99%lots of lear2.83%unstructured4.15%understandable2.83%
Gartlehner et al. BMC Medical Research Methodology (2017) 17:169comments expressed in German language into English and listedall data in Microsoft Excel, version 15.32 (MicrosoftCorporation, Redmond Washington, Usa). Then, westandardized synonyms and similar associations (east.g.'easy agreement' and 'easy to understand' to 'understandable'). Two raters carried out the translation andstandardization of data consecutively and controlledeach other'south work. Finally, nosotros calculated the relative frequencies of associations.ResultsWe get-go illustrate the utilise of fishbone diagrams throughthe two topics described in a higher place. We so present resultsof the user testing based on the anemia fishbone.Case 1: Systematic review on preoperative anemiamanagementA professional person club deputed this systematic review as groundwork for a panel developing clinical exercise guidelines [xi]. The anemia fishbone (Fig. two) depictsthe unabridged body of evidence from this review. The headof the fish represents the comparison of interest, namelythe residue of benefits and risks of preoperative anemiamanagement versus no management. Post-obit theGRADE approach, the guideline panel selected vi outcomes that they accounted critical or of import for clinicaldecisionmaking; each bone represents an issue. Disquisitional outcomes are closer to the head than less importantoutcomes. A solid line between the fish spine and theoutcome indicates that the review identified eligible evidence; dotted lines symbolize the lack of evidence (i.east.,the review detected no eligible studies).Each os representing an result as well provides abrief plain-linguistic communication summary of the results and presentskey characteristics of the evidence for that outcome: thenumber of trials, the full number of participants inthese trials, the magnitude of issue in accented and relative risks, and the certainty of evidence according toGRADE methods. For example, the panel ranked reduction of bloodshed and quality of life equally critical outcomes,then both are close to the head. The testify for reducedmortality consisted of three RCTs with a total of 210participants. The pooled result of the three trials yieldedan indeterminate effect guess with wide confidenceintervals including both benefits and harms (relative risk[RR] 1.59; 95% confidence interval [CI] 0.48, five.31). Theteam that conducted the review graded the certainty ofevidence equally depression. Because no evidence was available forquality of life, a dotted line is used. Past contrast, theguideline panel rated the rate of infections and durationof hospital stay as important (but not critical) outcomes.Consequently, they are located close to the tail.Page 6 of 9Example 2: Umbrella systematic review onmammography screeningThis example attempts to estimate the procedure of aninternational organization developing recommendationsfor mammography screening. Because of applicabilityconcerns when relying on results only from RCTs, theguideline panel had commissioned a systematic reviewof systematic reviews (sometimes called an umbrella review) of both RCTs and observational studies [13].The guideline panel had selected viii outcomes theydeemed disquisitional or of import for decisionmaking. In ourfishbone diagram, we depict the evidence for highresource settings from both RCTs and observationalstudies in a parallel manner. By drawing the diagram thisway, nosotros tin can help users not simply view of import characteristics of each upshot but also compare findings onthe aforementioned outcomes based on each torso of prove—RCTs and observational studies. Additional File 3presents the corresponding summary of findings tabular array.Results of user testingThe sample for user testing comprised 77 students. Most(75%) of the participants were female and were anile betwixt 18 and 40 years (median 22 years). All participants completed the survey. Additional File 4 presentssupplementary information on user testing.Beginning impressionParticipants' get-go impressions of the displays favored summary of findings tables over fishbone diagrams. Out of217 comments initially noted by participants, the mostfrequent associated with the fishbone diagram were'unclear' (11.52%), 'confusing' (8.78%), and 'clear' (v.99%).From the 212 initial reactions for the summary of findingstable, the most frequent ones were 'structured' (xviii.iv%),'articulate' (16.04%), and 'lots of information' (10.38%). Table 2summarizes the ten near frequent comments for eachdisplay.Effigy iv presents ratings of pre-determined attributesdirectly comparing the two displays afterward getting a firstimpression (light blueish) and afterward having worked with either of the two displays (dark blue). Rectangles representmean ratings. The middle position (0) indicates a neutralattitude. Overall, results favor the summary of findingstable over the fishbone diagram and did not modify substantially subsequently having worked with either display. Additional File 4 presents box plots depicting thedistribution of answers for each attribute.Ability to notice disquisitional informationAfter participants had been randomized to either thefishbone diagram or the summary of findings table, theywere asked to option an overall determination and to answerthree questions that required them to observe and interpret
Gartlehner et al. BMC Medical Inquiry Methodology (2017) 17:169Page 7 of 9Fig. 4 Direct comparison of fishbone diagrams with summary of findings tables before and afterward working with one of themfacts from the displays (eastward.g., Preoperative anemia direction is favored the almost by outcomes concerning: a)reduction of mortality, b) quality of life, c) demand for bloodtransfusions, d) duration of hospital stay). When askedto pick a correct overall conclusion, 68.iv% of participants from the fishbone group and 71.8% from the summary of findings group provided the right answer. Ofthose who picked the right conclusion, participants inthe fishbone group were on average 26 s faster in determining the correct respond (154.5 s vs. 180.viii due south; differencenot statistically meaning). However, more than half ofthe participants in each group were unable to answerthe three questions correctly that required finding andinterpreting facts (52.6% in the fishbone group; 51.three% inthe summary of findings group). Of those who pickedthe right options, participants in the fishbone groupwere on boilerplate 26 s faster in deciding on the answers(123.2 s vs. 149.three s) than those using the summary offindings table. The difference, however, was notstatistically significant. More detailed results on the correct selection of options in each grouping tin exist found inAdditional File 4.Perceived utilityAfter working with the fishbone diagram or the summary of findings tabular array, participants' assessments of theperceived utility differed significantly between the twodisplays for some of the items. More participants usingthe summary of findings tables than the fishbone diagrams agreed with the argument 'Overall, I liked thediagram/tabular array' (p .01; combining strongly hold/concord/slightly concord options: 71.8% vs. 44.8%). By comparison,significantly more participants using the fishbone diagrams agreed with the statements 'It was difficult to findthe information I was interested in' (p .01; combiningstrongly agree/agree/slightly concur options: 65.viii% vs.45%;) and 'The information in the diagram/tabular array wasconfusing' (p .05; combining strongly agree/agree/slightly agree options: 63.2% vs. 35.ix%).Additional File 4 summarizes the perceived utilitycomparing fishbone diagrams with summary of findingstables. Overall, the perceived utility was better for summary of findings tables. Consequently, more participantswould recommend summary of findings tables to a colleague (53.2% vs. xi.vii%) or to a systematic reviewer(74.0% vs. 7.viii%). Fewer participants, however, would recommend summary of findings tables than fishbone diagrams to patients (23.4% vs. 31.2%).DiscussionThe concept of fishbone diagrams has been used in healthcare for decades for crusade and outcome analysis related to patient prophylactic and for evaluating quality management processes. We presented two adaptations of a fishbonediagram to display big volumes of evidence in systematicreviews. The diagrams provide data-rich graphicsthat can supplement the circuitous and detailed tables thatusually summarize the findings of systematic reviews. Weenvisioned fishbone diagrams every bit a useful tool for peoplewith some epidemiological preparation but not enough familiarity with complex summary of findings tables.User testing, all the same, revealed that students withbasic training in clinical epidemiology found fishbone diagrams complicated and hard to sympathize. Participants rated the perceived utility of summary of findingstables every bit better than that of fishbone diagrams. A somewhat surprising result, however, was that more than halfof the participants in each group were unable to answerall iii factual questions correctly. This finding confirms that summary of findings tables, despite theirwidespread utilize, are not an platonic style to convey keypoints well-nigh evidence to readers of systematic reviewswith little epidemiological grooming.An of import reason for the unfavorable results concerning fishbone diagrams is probably that most
Gartlehner et al. BMC Medical Research Methodology (2017) 17:169participants were confronted with this concept for the firsttime. Nosotros briefly explained the bones idea of fishbone diagrams earlier user testing only this could probably notovercome the fact that participants were more familiarwith tables. The unfavorable ratings, nevertheless, pertain tofirst impressions and the perceived utility only. When wetested how well participants can detect and interpret factspresented in fishbone diagrams or summary of findings tables, no substantial differences were identified.Our user testing has several limitations. Kickoff, past surveying students unfamiliar with the concept of fishbonediagrams, we probably placed fishbone diagrams at a disadvantage compared with summary of findings tables.Had participants received some training in the usage offishbone diagrams, the perceived utility might have beenbetter. We did not provide training and so as to emulateusual situations in which systematic reviews are used.The average reader who might face a fishbone diagram in a systematic review would likely lack training inthe diagram usage. 2d, the participants of user testing comprised of students who had at to the lowest degree one coursein clinical epidemiology and prove-based medicine.Clinicians and decision-makers who read systematic reviews might non have this background knowledge. Theapplicability of our findings to other populations withdifferent educational backgrounds, therefore, is unclear.Third, we conducted the pilot test in wellness services researchers, some of whom had many years of experiencewith conducting and interpreting evidence syntheses. Thefact that the airplane pilot population had more experience withevidence syntheses than the actual population in the usertests could have contributed to a potential lack of agreement of the purpose of the user test. Nosotros cannot ruleout a certain extent of measurement mistake because participants may have misunderstood our questions.Despite the unfavorable results of the user examination, nosotros stillbelieve that fishbone diagrams, in general, have the potential to convey the facts of systematic reviews in astructured and efficient manner. The fishbone diagramis a flexible tool; users can conform information technology in numerous waysbased on the details of the systematic review or theneeds of the users (e.yard., guideline developers, clinicians).For instance, by juxta-positioning results from RCTs andobservational studies in our mammography fishbonediagram, we permit for a quick comparison of effects fromRCTs and observational studies which can assistance readersgauge potential issues with applicability of results fromRCTs. With its easily modifiable structure, the number ofbones for any given systematic review topic can exist increased or reduced depending on whether the focus isnarrowed or expanded. Additionally, researchers canexperiment with and test other graphical elements tocharacterize numerous pieces of information, such asthe types of studies summarized, the quality of thePage 8 of 9evidence, or the magnitude of the overall effect. Suchelements tin include color, line width (varying thicknesses), and line blazon (eastward.m., solid or dot
Fishbone diagrams were starting time proposed by Kaoro Ishi-kawa [5] in the 1960s to display cause and effect in the context of continuous improvement of industrial pro-cesses [6]. The diagram was first termed an "Ishikawa diagram" simply was later dubbed "fishbone diagram
Source: https://fdokumen.id/dokumen/6634_user-testing-of-an-adaptation-of-fishbone-diagrams-to.html
0 Response to "User Testing of an Adaptation of Fishbone Diagrams to Depict Results of Systematic Reviews"
Post a Comment