Benefits of BYOD: BYOD is an effective approach in healthcare settings and clinical trials as it allows subjects to provide medical information via their own internet-enabled device. It’s not a secret that recruiting participants and collecting data are among the most challenging aspects of research. With numerous benefits, BYOD is preferred over traditional methods. For instance, users can either access an online platform or download a medical app. The BYOD approach has been implemented even in Phase II and Phase III clinical trials. Some of the major benefits include:

• Access to data in real-time: Just like with any eCOA and provisioned devices, BYOD ensures access to high-quality medical information. As data collection occurs in real time, 24/7, errors and bias are minimized. As a result, clinicians have access to accurate and valid data.
• High engagement: Studies show that BYOD boosts engagement and improves compliance. Via SMS, notifications, and emails, doctors can establish a good relationship with their patients and monitor non-compliance. In addition, up-to-date images and visuals can help people track their condition and progress over time.
• Usability: BYOD means accessibility. Since a vast majority of people own and use a mobile device on a daily basis, experts can reach a wide range of participants. According to data, there are approximately 2.5 billion smartphone users today; and these numbers are increasing. By not carrying an additional device and having optimal familiarity, training costs can only decrease.
• Better user experience: Customized options improve the user experience. mHealth apps, in particular, are gaining more and more popularity. Interestingly, statistics show that 105,912 apps in the Google Play store and 95,851 apps on the iTunes app store are marketed as health apps (Bol et al., 2018).
• Productivity: The implementation of BYOD in practice improves clinicians’ productivity. By having access to real-time data, experts can access reports 24/7, which benefits decision-making. In fact, electronic health records (EHRs) that contain data about patient health, demographics, medications, and lab tests, can improve medical workflow.
• Cost-effective: By implementing the BYOD approach in research, experts can reduce training costs and improve resource efficiency. When a patient brings their own device, there’s no need to store tech gadgets on-site or deal with logistics.
• Limited site involvement: Automatic updates and online platforms eliminate the need for site involvement and the burden of commuting. What’s more, with the integration of Help buttons, patients can find online support, which can boost participation and outcomes.
• Advanced features: Mobile devices are equipped with numerous advanced features (such as GPS, barcode scanning, etc.). For instance, GPS options can help researchers monitor a patient’s location and activities in a study in which activity levels are used as an endpoint. BYOD gives access to reports which are available in different formats (e.g., PDF) across different devices (e.g., Android). While clinicians can access biomedical research to provide support, users can connect their devices with other wellness and fitness wearables.

Challenges in the implementation of BYOD: Although BYOD is increasing in popularity, there are a few challenges researchers need to overcome in order to implement BYOD in clinical trials (Marshall, 2014). Researchers need to create a good study design, taking in account patient rapport, data accuracy, and technical aspects (e.g., screen size). Factors, such as lack of a mobile device, demographics, reimbursement, and IT use support, should also be considered. Note that one of the major concerns is data security and HIPAA regulations.

Risks associated with BYOD: Possible risks in clinical research are alarming. Data security and patient privacy are among the major concerns. Since medical data is sensitive, networks must be protected. Virtual sandboxes can be installed on a device to protect apps that deal with medical data. Thus, clinicians won’t breach HIPAA policies, support staff won’t access certain services, and patients will access a hospital’s patient portal only for relevant information.

BYOD in healthcare practice and aspects to consider: With its numerous benefits, BYOD is becoming one of the most effective approaches in research. When implementing BYOD in practice, clinicians and IT specialists should consider the following aspects in order to overcome challenges and possible risks (“9 Key Factors to Evaluate When Considering BYOD”):

• App-based and web-based BYOD: Experts must decide on either an app-based or web-based BYOD. mHealth apps, as stated above, are increasing in popularity. They allow patients to complete a wide variety of PRO’s, including diaries, reports, and reminders. App-based BYOD can benefit populations that use smartphones on a daily basis, as well as the administration of simple questionnaires. Note that unexpected events (e.g., changing phones) should be considered. Web-based BYOD, on the other hand, allows patients to enter data through a web browser (e.g., Chrome) via their own devices (e.g., PC). They are effective in Phase IV studies and in a large number of patients. Note that web-based questionnaires can automatically resize according to the screen size of any patient’s mobile device.
• Usability and availability: BYOD can improve usability. Nevertheless, although some patients prefer BYOD over provisioned devices, experts should consider patients who need additional training or have privacy concerns. Note that with the increasing range of mobile devices on the market, staff may also need additional training to provide support – regarding OS, brands, and study schedules. Also, although more and more people use technology on a daily basis, researchers need to make sure that enrollment is not biased by mobile device ownership (e.g., age and location). In fact, experts can employ a combined approach, and provide a provisioned device to subjects without a personal device. Note that to ensure accuracy and safety, provisioned devices have their calling and browsing options disabled and run only study software.
• Compliance: In research, eCOA reveal high levels of compliance (over 90%). Yet, when compared to other eCOA, BYOD can improve the user experience. To set an example, a study with 89% BYOD usage revealed compliance rates at 91.5% (“9 Key Factors to Evaluate When Considering BYOD”). Another study about the use of probiotic supplement showed that there’s a difference between provisioned devices and BYOD regarding compliance and engagement. Participants (n=87) were assigned to use a mobile application or no intervention. The mobile application subjects were additionally randomized into two groups: BYOD and a provided smartphone. Results revealed that BYOD is feasible in healthcare: the BYOD subgroup showed higher engagement, use of an application, and frequency. Nevertheless, when designing a clinical study, experts should consider the fact they can’t control or lock down personal devices (Pugliese et al.)
• Costs and training: BYOD can result in lower research costs. Although provisioned devices are widely used in research, experts agree that training, maintenance, and delivery can be costly. BYOD, on the other hand, can eliminate additional costs and logistic obstacles, as well as improve the user experience. Note that although BYOD can reduce costs associated with delivery and provisioned devices, sponsors still need to consider factors, such as training of staff, availability of support (e.g., support desk), and burden on participants. Since participants are using their own devices, sponsors should include reimbursements for all the data sending costs.
• Regulations and privacy risks: Before implementing BYOD in research, sponsors must ensure data consistency, quality, and transparency (Marshall, 2014). Note that even screen size may lead to bias. The stage of the study can also affect the implementation of BYOD. Therefore, experts must always consult regulatory bodies, follow existing regulations, and maintain clear documentation. Researchers should also establish a good relationship with the owners of the original questionnaires and get the owner’s permission to migrate the tool onto an electronic platform. Most of all, privacy concerns must be addressed. In fact, when it comes to privacy concerns, BYOD can ensure safety. Protection features (e.g., unique PIN) and encrypted data can improve the security of any software.

BYOD checklist: Creating a good study design is one of the major factors for scientific success. Apart from the aspects described above, researchers should consider the following checklist (“9 Key Factors to Evaluate When Considering BYOD”):

• The phase of the study
• Type of questionnaires (including items and images)
• Type of data (e.g., symptoms, data reported by patients)
• Frequency and duration of data collection
• Data collection (e.g. on-the-go)
• Characteristics of the population and the geographical situation (e.g., access to technology, shipping costs)

Clinical trials with BYOD: Research proves that BYOD is an effective approach in practice, considering patient management, compliance, and satisfaction. Based on Google Analytics, the findings from the randomized trial described earlier revealed that the BYOD subgroup showed higher engagement with the intervention, more application sessions per day, and longer sessions. Interestingly, the BYOD subgroup showed a significant effect of engagement on drug compliance in the end-line period of the trial. In fact, interviews revealed that BYOD users found it easy to integrate the mobile application into their daily routines (Pugliese et al., 2016).

With the effects of IT consumerization in real life, there’s no doubt that technology facilitates industries. Mobile devices have become an integrated part of people’s lives and business strategies. In healthcare, in particular, BYOD can lead to an increase in productivity, engagement, compliance, high-quality data, real-life communication, and cost-effective research. By considering data type, BYOD-related training, demographics, and privacy risks, experts can manage the successful implementation of BYOD in clinical trials in accordance with safety and ethical regulations.

Most of all, BYOD can improve user experience and well-being. Patients can bring their own device, which eliminates the need for carrying an additional device or training. Real-life data, smooth doctor-patient communication, and online support can only improve patient outcomes and quality of life. In the end, patients are the core of research and digital health.

Bol, N. Helberger, N., & Weert, J. (2018). Differences in mobile health app use: A source of new digital inequalities? The Information Society: An International Society, 34 (3).

Pugliese, L., Woodriff, M., Crowley, O., Lam, V., Sohn, J., & Bradley, S. (2016). Feasibility of the “Bring Your Own Device” Model in Clinical Research: Results from a Randomized Controlled Pilot Study of a Mobile Patient Engagement Tool. Cureus, 8 (3).

Marshall, S. (2014). IT Consumerization: A Case Study of BYOD in a Healthcare Setting. Retrieved from  https://timreview.ca/article/771

9 Key Factors to Evaluate When Considering BYOD. Retrieved from https://resources.crfhealth.com/ebooks/9-key-factors-to-evaluate-when-considering-byod

Real-world data is defined as data obtained from a heterogeneous population in real-world settings, with sources varying from administrative claims to health surveys. Although health care providers worldwide recognize the importance of real-world data, we should note that there’s no consensus about its definition. Interestingly, in a recent study, which included 53 documents and 20 interviews, 38 definitions of real-world data were identified (Makady et al., 2017). While most of the definitions categorized real-world data as information obtained from non-randomized controlled trials, many experts were unable to provide a clear institutional definition.

Therefore, to clarify the concept of real-world data, previous definitions given by major initiatives and healthcare bodies can be utilized and modified. According to the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), real-world data is “Data used for decision-making that are not collected in conventional RCTs.” The Association of the British Pharmaceutical Industry (ABPI) states, “For the purposes of this guidance, “RWD” will refer to data obtained by any non-interventional methodology that describe what is happening in normal clinical practice.” On the other hand, the official definition given by RAND goes, “RWD” is an umbrella term for different types of health care data that are not collected in conventional RCTs. RWD in the health care sector come from various sources and include patient data, data from clinicians, hospital data, data from payers, and social data.” Last but not least, the Innovative Medicines Initiative (IMI)-GetReal defines real-world data as “An umbrella term for data regarding the effects of health interventions (e.g., benefit, risk, and resource use) that are not collected in the context of conventional RCTs. Instead, RWD is collected both prospectively and retrospectively from observations of routine clinical practice. Data collected include, but are not limited to, clinical and economic outcomes, patient-reported outcomes, and health-related quality of life. RWD can be obtained from many sources including patient registries, electronic medical records, and observational studies.” (Makady et al., 2017).

Just like with the definition of real-world data, there’s a wide range of additional and non-randomized control sources that can provide real-world evidence in research and practice. Note that real-world information can be collected retrospectively and prospectively, with electronic platforms facilitating data collection. Some of the major sources of real-world data include:

• Supplementary data: Although randomized controlled trials provide valuable health-related information, real-world data from patient-reported outcomes and claims can complement findings and provide insights into the long-term effectiveness of the new treatment. Real-world data can also help experts generate hypotheses, design studies, test research questions, and recruit subjects.
• Pragmatic trials: While randomized controlled trials limit their findings to controlled environments and populations, pragmatic trials or large simple trials aim to test the effectiveness of any novel treatment in real settings. Popular study designs include population enrichment randomized controlled trials, comprehensive cohort studies, non-randomized controlled trials, and cluster studies (“Sources of Real-world Data”). Note that these studies can provide valuable prospective information.
• Observational studies: Observational studies are also a powerful source of real-world evidence, including cohort studies, case reports, and case-control studies. Health-related observations may include various aspects, such as health outcomes, social factors, patient well-being, and quality of life.
• Health surveys: Health surveys are a powerful source of health care information which can provide valuable insights into a patient’s subjective experiences and well-being. Note that health surveys provide medical information about the population of interest (e.g., a wide range of patients), not just the participants in the clinical trial (Dang & Vallish, 2016). In addition, patient-powered networks, which are online platforms run by patients to collect data on a disease or medications, can also be used as a rich source of real-world evidence.
• Administrative data: Administrative data provided by pharmacies and health insurance companies are another legitimate source of real-world data, as well as medical research. A recent review proved that claims-based non-randomized and randomized controlled trials could complement each other by creating an overlap of information (Najafzadeh et al., 2017). To be more precise, randomized controlled trials may focus only on a particular variable and miss long-term outcomes (e.g., effects on mortality). Insurance claims data and billing information, on the other hand, can provide long-term data and follow-ups at a lower cost. Note that claims provide retrospective or real-time data, particularly for reimbursement and economic outcomes. In addition, databases that provide health information outside of the trials (e.g., Medicare) can be used to deal with missing data and drop-outs.
• Electronic health records and patient registries: Electronic health records and other electronic databases can be used to provide information about laboratory tests and standard medical services related to health care treatments and interventions. Note that electronic records reduce costs and improve interoperability. Patient registries that contain information on a group of patients can also provide valuable data and benefit observations. Note that registries are prospective, with disease-based registries being the most powerful source of high-quality data (e.g., the Global Registry of Acute  Coronary  Events) (Dang & Vallish, 2016).
• Social media: With the increasing role of technology in health care settings, social media data is also beneficial. It can provide valuable information on additional factors that may affect a patient’s treatment, non-compliance, and emotional well-being (“Sources of Real-world Data”). Note that social media channels can improve doctor-patient communication, recruitment, and drug marketing.

Real-world data has numerous applications in research and practice. Real-life data can give valuable information about the effectiveness of treatment in real-world settings, and across diverse populations. Note that real-world data can be used as primary data from interventions or as secondary data from patient-reported outcomes and administrative sources. Interestingly, a recent review conducted by the IMI-GetReal revealed that usually, real-world data is utilized to provide insights before and after the market authorization of a new drug; assess the pharmacoeconomic properties of treatment; and explore effectiveness in conditional reimbursement schemes (Makady et al., 2017). Note that Makady and colleagues evaluated the existing policies of  six European agencies, as follows: “the Dental and Pharmaceutical Benefits Agency (Sweden), the National Institute for Health and Care Excellence (United Kingdom), the Institute for Quality and Efficiency in Health-care (Germany), the High Authority for Health (France), the Italian Medicines Agency (Italy), and the National Healthcare Institute (The Netherlands)” (Makady et al., 2017). Some of the major applications of real-world data include:

• Drug development: Drug development is a complex process, in which safety and efficacy come first. To support research and routine clinical practices, real-world data can be employed in drug development. Real-world data can be used to examine factors, such as the nature of a disease, existing clinical practices, and medical costs.
• Pre- and after-market authorization and decision-making: Randomized controlled trials, especially Phase III trials, are essential to provide information about the safety and efficacy of a new intervention. Nevertheless, real-world data has become a powerful source of information in research. Real-world data can be used to assess the generalizability of scientific results and real-world safety. As a result, more and more regulatory bodies, health technology assessment organizations, and patients are opting for robust real-world data to complement clinical findings.
• Pharmacoeconomic analyses: Since clinical trials can be lengthy and costly, real-world data can be used to provide robust information on the efficacy of a new drug and its pharmacoeconomic properties. Data can also be used to assess effectiveness across heterogeneous populations and between different products on the market.
• Reimbursement schemes: Since medical costs are increasing, patients, payers, and insurance companies are looking for reliable real-world data about the benefits of any novel treatment, including long-term effects and real-world safety. Real-world information can also provide valuable insights into the financial side of new interventions and healthcare services, which often obstructs health care practice.

Real-world data has numerous applications in research and practice. Real-life data can give valuable information about the effectiveness of treatment in real-world settings, and across diverse populations. Note that real-world data can be used as primary data from interventions or as secondary data from patient-reported outcomes and administrative sources. Interestingly, a recent review conducted by the IMI-GetReal revealed that usually, real-world data is utilized to provide insights before and after the market authorization of a new drug; assess the pharmacoeconomic properties of treatment; and explore effectiveness in conditional reimbursement schemes (Makady et al., 2017). Note that Makady and colleagues evaluated the existing policies of  six European agencies, as follows: “the Dental and Pharmaceutical Benefits Agency (Sweden), the National Institute for Health and Care Excellence (United Kingdom), the Institute for Quality and Efficiency in Health-care (Germany), the High Authority for Health (France), the Italian Medicines Agency (Italy), and the National Healthcare Institute (The Netherlands)” (Makady et al., 2017). Some of the major applications of real-world data include:

• Drug development: Drug development is a complex process, in which safety and efficacy come first. To support research and routine clinical practices, real-world data can be employed in drug development. Real-world data can be used to examine factors, such as the nature of a disease, existing clinical practices, and medical costs.
• Pre- and after-market authorization and decision-making: Randomized controlled trials, especially Phase III trials, are essential to provide information about the safety and efficacy of a new intervention. Nevertheless, real-world data has become a powerful source of information in research. Real-world data can be used to assess the generalizability of scientific results and real-world safety. As a result, more and more regulatory bodies, health technology assessment organizations, and patients are opting for robust real-world data to complement clinical findings.
• Pharmacoeconomic analyses: Since clinical trials can be lengthy and costly, real-world data can be used to provide robust information on the efficacy of a new drug and its pharmacoeconomic properties. Data can also be used to assess effectiveness across heterogeneous populations and between different products on the market.
• Reimbursement schemes: Since medical costs are increasing, patients, payers, and insurance companies are looking for reliable real-world data about the benefits of any novel treatment, including long-term effects and real-world safety. Real-world information can also provide valuable insights into the financial side of new interventions and healthcare services, which often obstructs health care practice.

Medical research and drug development can challenge standard healthcare practices. Although randomized controlled trials are the gold standard in research, real-world evidence becomes crucial to establish the effectiveness of a new intervention. To meet the need for real-world evidence, experts, payers, and patients must integrate real-world data (RWD) in practice.

To sum up, real-world data is vital in health care. As explained above, real-world data can be obtained from various sources, such as patient registries, administrative claims, and social media channels, in order to assess heterogeneous groups and settings. Findings can be applied to different real environments in order to provide insights into drug safety, in health and financial terms, and its long-term effects. Real-world data can improve decision-making, pre-authorization, and reimbursement of new drugs and treatments and benefit medical research and patient outcomes. Therefore, regulatory agencies and experts must reach a consensus and embrace real-world data in practice. In the end, patient well-being is the main focus of digital health research.

Dang, A., & Vallish, B. (2016). Real-world evidence: An Indian perspective. Perspectives in Clinical Research.

Makady, A., Ham, R., de Boer, A., Hillege, H., Klungel, O., Goettsch, W., on behalf of GetReal Workpackage (2017). Policies for Use of Real-World Data in Health Technology Assessment (HTA): A Comparative Study of Six HTA Agencies. Value in Health: The journal of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), 20 (4).

Makady, A., de Boer, A., Hillege, H., Klungel, O., Goettsch, W., on behalf of GetReal Work Package (2017). What Is Real-World Data? A Review of Definitions Based on Literature and Stakeholder Interviews. Value in Health: The journal of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), 20 (7).

Najafzadeh, M., Gagne, J., & Schneeweiss, S. (2017). Synergies from Integrating Randomized Controlled Trials and Real-World Data Analyses. Clinical Pharmacology and Therapeutics.

Singal, A., Higgins, P., & Waljee, A. (2014). A Primer on Effectiveness and Efficacy Trials. Clinical and Translational Gastroenterology, 5. Retrieved from https://www.nature.com/articles/ctg201313

Sources of Real-World Data. Retrieved from https://rwe-navigator.eu/use-real-world-evidence/sources-of-real-world-data/

Research in the field of oncology is sensitive and challenging. Note that there are various cancer treatments, with radiation therapy and chemotherapy being among the most common interventions (“Types of Cancer Treatments,” 2017). With the increasing use of digital health solutions and experimental cancer therapies, though, real-world data can support numerous aspects of medical research and routine clinical practice. Some of its applications include:

Parkinson’s disease is a complicated medical condition, defined as a central nervous system disorder. The wide range of symptoms and complications requires the use of different measurements for the effective management of the disease. Interestingly, a recent literature review explored the benefits of various Parkinson’s disease measurements, classified into the following dimensions of assessment (Tanguy, Jonsson & Ishihara, 2017).

• Motor and neurological function: Parkinson’s disease, as explained earlier, is a complex disorder of the central nervous system. Common symptoms include shaking, difficulty with walking, and rigidity (Opara et al., 2017). Patients are also prone to falls. Comparing motor tests is one of the most significant indicators employed in Parkinson’s research. There are various motor assessments (e.g. of hand function), such as the Hoehn and Yahr stages of progression of the disease, the Unified Parkinson’s Disease Rating Scale, and the Timed Up and Go Test. Note that the latest advancements in medical technology allow the three-dimensional analysis of gait and movement in participants. Since motor symptoms affect patients’ daily living and quality of life, real-world data is needed to improve Parkinson’s disease assessment and treatment options. Note that preventing falls is one of the leading goals in routine clinical practice.
• Cognition: Cognitive changes also occur in patients with Parkinson’s disease, especially in the late stages of the disease (Romann et al., 2012). Although there are various medications and existing treatments, delaying Parkinson’s symptoms is still a challenge in medical research. As mentioned earlier, there’s no cure for Parkinson’s; each patient receives treatment based on their individual symptoms. Note that in some cases, Parkinson’s treatments (e.g., deep brain stimulation) may worsen a patient’s cognitive functioning. Some of the common tools used to assess dementia in Parkinson’s patients include the Clock Drawing Test, Stroop Test, Digit Span, Trail Making Test, Wisconsin Card Sorting Test, and Verbal Fluency Test. In addition, real-world evidence can provide a comprehensive picture of one’s cognitive abilities and changes.
• Psychiatric symptoms: Psychiatric symptoms can also be observed in Parkinson’s disease. Neuropathological findings indicate that brain dysfunction, medications, and the impact on daily living can lead to psychotic episodes, shame, and depression. Common scales to assess this dimension in patients with Parkinson’s include the Beck Depression Inventory, Parkinson Psychosis Questionnaire, Clinician Global Impression of Change, and Geriatric Depression Scale. Interestingly, when it comes to depression, statistics show that 40% of Parkinson’s patients experience depression and distress. Real-life data can benefit Parkinson’s research, including patient health outcomes.
• Activities of daily living: Parkinson’s disease has a negative impact on daily living, especially in the late stages of the disease (Lee et al., 2016). Basic activities, such as feeding, bathing, and dressing, can be severely affected. Hence, the caregiver’s role is vital. Since Parkinson’s is a progressive neurological condition, real-world data is essential for the assessment of disease management and treatment. The Unified Parkinson’s Disease Rating Scale and self-reported outcomes are among the most popular tools used to study Parkinson’s disease and its impact on daily living.
• Sleep quality: Sleep quality in patients with Parkinson’s disease declines over time (“Sleep disorders in Parkinson’s disease: Diagnosis and management,” 2011). Note that the Epworth Sleepiness Scale Sleep, the Apnea Scale of Sleep Disorders Questionnaire, and the Parkinson’s disease sleep scale can be utilized to assess this domain in patients with Parkinson’s disease. In addition, polysomnography can provide valuable clinical insights into a patient’s sleep patterns and symptoms. Interestingly, practice shows that the bed partner can also be affected (e.g., due to limb movements that prevent them from sleeping). Thus, longitudinal real-world data is needed to benefit patients and families.
• Treatment: Although various treatments and medications (e.g., dopamine agonists) exist, Parkinson’s disease still challenges researchers and practitioners. Note that Levodopa is among the most effective medications to treat Parkinson’s. Research, however, shows that some medications can cause severe side effects. Therefore, Parkinson’s disease rehabilitation (including speech therapy, physiotherapy, and education) is becoming a leading approach in research. Real-world data, such as administrative claims, can provide beneficial information about treatment effectiveness, adherence, and long-term effects.
• Quality of life: Quality of life is one of the main areas of digital health research. Real-world data can be utilized to improve patients’ quality of life. Since Parkinson’s disease leads to numerous symptoms, such as shaking, depression, and dementia, patients report low levels of quality of life (Opara et al., 2012). We should note that the quality of life is a complex construct, which includes physical, mental, and social aspects. It’s not surprising that the quality of life is becoming one of the leading factors in digital health research. Satisfaction with treatment, self-image, and social support also influence a patient’s well-being. Subjective measures are usually employed to measure the quality of life in patients with Parkinson’s disease. Assessments such as the Quality of Life in Neurological Disorders, 39-item Parkinson’s Disease Quality of Life, and 36-item Short Form can be utilized to measure a patient’s well-being and treatment effectiveness.
• Autonomic symptoms: Research shows that autonomic symptoms in Parkinson’s disease include blood pressure, constipation, swallowing, sweating, and sexual dysfunction. Note that autonomic symptoms can affect a patient’s quality of life (Merola et al., 2018). Merola and colleagues revealed that autonomic symptoms could worsen by 20% over the course of 12 months. Therefore, scales for outcomes of Parkinson’s – Autonomic Symptoms should be employed to help researchers monitor patients. Sensors and wearable devices also provide valuable real-world evidence across a wide range of data points.
• Other: Research of Parkinson’s disease has identified another crucial dimension, defined as Other (Tanguy, Jonsson & Ishihara, 2017). This broader research area includes aspects, such as olfaction, emotional support, and caregivers’ burden. Some beneficial measurements include the 16-item sniffin’ Sticks Odor Identification test, the Social Readjustment Rating Scale, and the Multidimensional Caregiver Strain Index, respectively. Longitudinal real-world data is needed to explore all these medical and socioeconomic problems in the long-term. Data can benefit patients as well as caregivers.

Yet, this research classification should be further harmonized across local and international research and administrative bodies.

Parkinson’s is a progressive neurogenerative disease with severe complications. Since symptoms fluctuate on a daily basis and between patients, real-world data is needed to measure symptoms, assess treatments, benefit financial decisions, and improve patients’ quality of life. Note that a recent study showed that smartphone assessments could provide valuable clinical insights, including frequent real-world data (Zhan et al., 2018). Zhan and colleagues assessed 129 Parkinson’s patients who completed five tasks on a mobile app. The tasks measured finger tapping, balance, gait, voice, and reaction time, providing data from 6,148 smartphone activity assessments in total. mHealth apps can help patients with a chronic condition track their symptoms, manage medications, store insurance information, and find social support. Practice shows that mobile sensors which track symptoms and Parkinson’s apps which show exercises (e.g., to improve balance and posture) are highly beneficialwell-being. What’s more, Parkinson’s disease research shows that real-world measurements and remote monitoring can reduce research costs and improve health outcomes. Data collection in real time complements scientific findings: it provides vital information that annual medical visits are unable to capture. Last but not least, the mHealth approach in medicine is growing in popularity as it empowers and motivates patients. A recent initiative revealed that patients with Parkinson’s disease are willing to share their health-related data to contribute to research and routine clinical practice. Interestingly, 4,218 people from more than 50 countries provided a large source of medical information, including symptoms scores, patient-reported outcomes surveys, diary entries, and data from patients’ wearables.

With the transfer of medical information into electronic datasets, electronic health records and administrative claims also reveal some impressive benefits over standard clinical trials. In fact, although randomized controlled trials are still the gold standard in research, real-world data is vital. It can address some of the major obstacles in Parkinson’s disease research, such as strict inclusion criteria, poor generalizability, underrepresented populations, high costs, unexpected delays, and a lack of follow-up information (especially in non-pharmacological interventions). Medical claims databases, on the other hand, offer some impressive advantages over standard trials (Bloem et al., 2018). Administrative claims provide information about diverse populations, real-world settings, and co-morbid conditions. Investigating a medical claims dataset is more cost-effective than traditional clinical trials. Electronic health records also provide robust real-world data, including medical history and prescriptions, which can complement scientific findings.

In conclusion, real-world data plays an important role in research and practice. When it comes to Parkinson’s disease research, longitudinal real-world data benefit the evaluation of epidemiology, treatment management, and payment decisions. Parkinson’s disease, as explained above, is one of the most challenging neurological diseases, which affects millions of people worldwide. Since researchers and practitioners cannot capture all the varying symptoms and side effects associated with the disease, interventions are often individualized. Hence, longitudinal real-world data is essential to complement medical findings and improve clinical decisions.

With the increasing capabilities of today’s digital health solutions, electronic sources of real-world data are becoming more and more popular. Sources, such as administrative claims and mHealth apps, support data collection, and analysis across various domains of research (e.g., cognition, quality of life, etc.). Note that digital solutions facilitate data collection, increase user engagement, and empower patients. In the end, real-world evidence is reshaping the future of Parkinson’s research – improving patients health outcomes and quality of life.

Bloem, B., Ypinga, J., Willis, A., Canning, C., Barker, R., Munneke, M., & De Vriesa, N. (2018). Using Medical Claims Analyses to Understand Interventions for Parkinson Patients. Journal of Parkinson’s Disease, 8(1), p.45-58.

Lee, S., Kim, S., Cheon, S., Seo, J., Kim, M., & Kim, J. (2016). Activities of daily living questionnaire from patients’ perspectives in Parkinson’s disease: a cross-sectional study. BMC Neurology.

Merola, A., Romagnolo, A., Rosso, M., Suri, R., Berndt, Z., Maule, S., Lopiano, L., & Espay, A. (2018). Autonomic dysfunction in Parkinson’s disease: A prospective cohort study. Movement Disorders, 33(3), p. 391-397

Opara, J., Brola, W., Leonardi, M., & Blaszczyk, B. (2012). Quality of life in Parkinson`s Disease. Journal of Medicine and Life, 5(4), p. 375-381.

Opara, J., Malecki, A., Malecka, E., & Socha, T. (2017). Motor assessment in Parkinson`s disease. Annals of Agricultural and Environmental Medicine, 24(3), p. 411-415.

Romann, A., Dornelles, S., Maineri, N., Rieder, C., & Olchik, M. (2012). Cognitive assessment instruments in Parkinson’s disease patients undergoing deep brain stimulation. Dementia & Neuropsychologia.

Sleep disorders in Parkinson’s disease: Diagnosis and management (2011). Annals of Indian Academy of Neurology.

Tanguy, A., Jonsson, L., & Ishihara, L. (2017). Inventory of real-world data sources in Parkinson’s disease. BMC Neurology.

Zhan A, Mohan S, Tarolli C, Schneider R, Adams J, Sharma S, Elson M, Spear K, Glidden A, Little M, Terzis A, Dorsey E, Saria S. (2018). Using Smartphones and Machine Learning to Quantify Parkinson Disease Severity The Mobile Parkinson Disease Score. JAMA Neurol, 75(7), p.876-880.

Research in the field of oncology is sensitive and challenging. Note that there are various cancer treatments, with radiation therapy and chemotherapy being among the most common interventions (“Types of Cancer Treatments,” 2017). With the increasing use of digital health solutions and experimental cancer therapies, though, real-world data can support numerous aspects of medical research and routine clinical practice. Some of its applications include:

While clinical trials are the gold standard in medical research, real-world data can complement clinical findings. In fact, real-world experiences were the base of medical discoveries for hundreds of years (Corrigan-Curay, Sacks, & Woodcock, 2018). Real-world data can be used to explore biomarkers, validate surrogate endpoints, assess long-term effects, and evaluate treatments in diverse settings. Both prospective (e.g., self-reports) and retrospective (e.g., chart review) data can bring valuable insights into research. In addition, new metrics and technologies (e.g., wearables and apps) can enable data collection from almost every patient with cancer in real time. Such innovations can benefit oncology care and patient-centric methods. Digital solutions can also improve interoperability and research regulations.

In cancer treatments, pragmatic trials reveal some advantages over randomized clinical studies. Note that usually, the rates of participation in clinical trials are low (under 5%) and some groups might be excluded from research (e.g., elderly people) (Khozin, Blumenthal & Pazdur, 2017). Pragmatic trials, on the other hand, combine real-world evidence and information from randomized trials to support the effects of an existing treatment in practice. By including real-world data, community oncologists can get involved (especially in the late stages of the study) helping more patients access novel treatments. Electronic health records become an essential source of information. It’s not surprising that the use of electronic records in health care has increased. Note that digital solutions can facilitate data extraction and data management.

While well-designed observational studies reveal similarities with controlled trials, they generate real-world evidence. Such studies supply health insights without invading patient and clinical behavior. In fact, electronic health records and routine clinical practice observations can provide vital information in cancer research, suggest hypotheses for new clinical trials, and facilitate decision-making (Khozin, Blumenthal & Pazdur, 2017). This approach can benefit subjects with organ dysfunctions and other abnormalities. Note that usually, these patients are excluded from conventional cancer studies. Besides, real-world data can improve the generalizability of any clinical information.

Real-world data can benefit early discovery. Since various factors affect cancer (e.g., genomic characteristics), experts can employ genomic sequencing data to identify biomarkers of response and cohorts, which can improve drug development. Note that advanced data mining techniques can support and improve the validity of biomedical data, as well as benefit target identification (Hughes et al., 2011). To set an example, a pharmaceutical company used a genomic database (with information on tumor sequencing) from subjects with lung cancer and created valid genomic profiles.

Real-world data can support the entire drug development cycle and shorten development times and costs. Note that normally, drug development benefits from aspects, such as accidental discovery, experimentation on fungi and plants, and the profound study of cancer cells and drug targets. Technological solutions and sophisticated simulation techniques can show oncologists how a drug interacts with a target. After that, new drugs are tested on tumor cells and animals, and only after approval – on humans (“Drug Discovery and Development,” 2018). We should mention that when it comes to innovations, biotechnology and pharmaceutical companies are among the leading high-quality providers. Therefore, more and more companies have started to invest in real-world data approaches and electronic sources to improve data collection and analysis – with the sole purpose of benefiting patient outcomes.

Real-world sources, such as electronic health records, support the design and execution of clinical studies. Real-world data can improve the trial design, protocol, and feasibility. It can also improve trial execution, particularly by the inclusion of external control arms. Having an external control arm obtained from historical or present populations and real-world settings can improve recruitment and registration. This approach is essential for rare cancers. Note that recently, a drug developer designed a study protocol with high external validity to explore a metastatic cancer population.

Designing a study and executing a trial, though, rely on robust findings regarding the natural history of the disease, which encompasses the entire course of the disease (including the pre-symptomatic phase and the point where the patient is cured or not). Real-world data can assess real-life settings (e.g., community-based medical facilities) and examine early screening and assessment methods. The retrospective analysis of electronic health records can also provide valuable insights into risks, early intervention, drug safety, and outcomes.

Voluntary passive reporting and clinical reports are the most popular techniques employed to assess post-marketing pharmacovigilance. Nevertheless, these approaches are prone to bias, such as media attention. Real-world data, on the other hand, can benefit active reporting and detect early signals and risks. Apart from electronic health records, experts may employ sources, such as apps and Internet searches, to capture self-reported symptoms. Mining techniques, natural language processing, and Bayesian geometric methods can verify any safety signals (Khozin, Blumenthal & Pazdur, 2017).

Real-world evidence can benefit post-marketing research. Post-marketing studies are usually conducted after a product has been approved for marketing. When it comes to retrospective data (e.g., chart review), data can be obtained from searches across different electronic sources to improve cancer diagnoses. In fact, it can also help experts understand the effect of a treatment in underrepresented populations (e.g., presence of brain metastases).

Although novel cancer therapies reveal some promising findings, medical costs are the main burden which societies must overcome. With the increasing number of treatment pathways, payers need evidence of clinical value and generalizability. In fact, data shows that only one-third of launched drugs meet medical expectations. Therefore, real-world data can be employed to inform reimbursements schemes and help experts reach a balance between costs and benefits.

Real-world data may support vital aspects of oncology, such as the expansion of the indication of a drug. It can help biopharmaceutical companies reduce costs and eliminate the need for randomized trials. To set an example, an oncology drug developer used data from electronic health records to support the indication expansion across rare biomarker-defined populations. Note that electronic health records are among the most powerful sources of real-world data.

By utilizing real-world data, experts will be able to provide personalized care, which can be supported by the newest advancements in genomics. Note that in the US, in particular, electronic medical records and administrative claims are the main sources of real-world data. Routine clinical practice and a patient’s insights into their own condition and family history can only support health and treatment outcomes. After all, today’s digital health era has initiated a shift in perspectives – with patients being the center of health care.

Real-world data has numerous advantages over standard methods; real-world information complements scientific findings, improves external validity, and facilitates access to treatment. We should note that randomized controlled trials often compromise external validity due to rigorous research factors, eligibility criteria, and protocols. Real-world data sources and electronic medical records contain vital information, such as patient diagnoses, drug prescribing, and laboratory tests. Health outcomes, progression, remission, and complications can also be followed up, which can benefit oncology research. Administrative claims can provide a beneficial overlap of information (e.g., billing codes and codes for medical procedures) to investigate drug safety and effectiveness. Even records not specific to oncology centers may be employed to provide robust medical information.

Yet, there are several challenges oncologists face in the successful implementation of real-world data in practice. Experts should achieve a balance between internal and external validity, based on good clinical practices. Since there’s a wide range of sources of real-world data, technological solutions to extract relevant information (e.g., from notes) must be employed. Direct feeds, cloud-based datasets, cohort expansion tools, and visuals are some of the tools that support real-world data collection and analysis. Note that big data can benefit clinical investigation, interoperability, and patient-centric medicine. Nevertheless, since medical information is sensitive and prone to security risks, safety and privacy concerns should be considered. Often, modifications of consent forms have to be made. Experts should also tackle the problem of off-label prescribing, patient-by-patient compassionate access, and accelerated admission (Lewis, Kerridge & Lipworth, 2017). A lack of access to life-saving cancer therapies can be devastating, so real-world data should be utilized to speed up drug discovery, marketing authorization, and post-marketing studies.

Real-world data is becoming fundamental in oncology. Cancer is one of the most devastating diseases worldwide, with high mortality rates and poor quality of life. The rapid pace of drug development in oncology requires the use of real-world evidence to support drug safety and effectiveness. More and more biopharmaceutical companies have started to integrate real-world data and analytic models in practice and research. Real-world data sources vary from administrative claims to electronic health records. Such sources benefit scientific findings, post-marketing pharmacovigilance, and indication expansion. Information can also be employed to inform reimbursements. Note that digital solutions improve health care and interoperability as they provide diverse medical information 24/7.

Most of all, real-world data can lead to an increase in personalized care in oncology. Patients become active participants in research and routine clinical practices. Real-world data can also benefit people who are usually excluded from clinical studies (e.g., those with comorbidities or rare malignancies); it can improve their access to health care and life-saving therapies. In the end, patients’ well-being and emotional support become a focus of cancer care and real-world data.

Cancer prevention and control (2018). Retrieved from https://www.who.int/nmh/a5816/en/

Corrigan-Curay, J., Sacks, L., & Woodcock, J. (2018). Real-World Evidence and Real-World Data for Evaluating Drug Safety and Effectiveness. JAMA

Drug discovery and development (2018, August). Retrieved from https://www.cancer.net/research-and-advocacy/introduction-cancer-research/drug-discovery-and-development

Hughes, J., Rees, S., Kalindjian, S., & Philpott, K. (2011). Principles of early drug discovery. British Journal of Pharmacology

Lewis, J., Kerridge, I., & Lipworth, W. (2017). Use of Real-World Data for the Research, Development, and Evaluation of Oncology Precision Medicines. Precision Oncology

Khozin, S., Blumenthal, G., & Pazdur, R. (2017). Real-world Data for Clinical Evidence Generation in Oncology. JNCI: Journal of the National Cancer Institute, 109 (11)

Types of cancer treatments (2017, April 6). Retrieved from https://www.cancer.gov/about-cancer/treatment/types

To facilitate healthcare practices, COSMIN has emerged as an ambitious initiative in the development of core outcome sets (COS), the standardization of measures, and the evaluation of outcome measurement instruments. By developing tools for selecting research instruments, the COSMIN initiative aims to facilitate the selection of outcome measurements. Thus, the initial aim of the initiative was to create a user-friendly checklist for experts and novices. To achieve this goal, a Delphi study was performed to reach consensus between experts. The Delphi study consisted of four rounds which took place between 2006 and 2007. Note that 91 international experts with at least five publications on health measures published in PubMed were invited. It’s interesting to mention that 57 experts agreed to participate; 25 from North America, 29 from Europe, 2 from Australia, and 1 from Asia. Out of those 57 scientists, 20 completed all four rounds. These numbers corresponded with the initial expectations of the committee panel; usually, Delphi studies reveal that 70% of the people invited normally agree to participate (Mokkink et al., 2010).

As stated above, the Delphi study conducted by Mokkink and colleagues (2010) consisted of four rounds. The first round focused on questions about measurement properties and scientific definitions (e.g., “Which definition do you consider the best for internal consistency?”). The second stage included questions and ratings about standards, study designs, and statistical methods (e.g., using Cronbach’s alpha to asses internal consistency). In the third round, the panel was presented with the most chosen methods and asked to provide feedback (e.g., if the most chosen method is actually the most preferred and appropriate one). Note that feedback and ideas were collected throughout all four stages of the study to foster constructive discussions and further solutions. In the last stage of the Delphi study, the aspects which the panel agreed on were integrated. Note that consensus was achieved when a minimum 67% of the members “agreed” or “strongly agreed” on a 5-point scale. After the final stage of the study, the steering committee created an initial version of the COSMIN checklist, which now is an integrated part of good research practices.

The ambitious Delphi study led to the development of the preliminary version of the COSMIN checklist. The list contains standards for assessing the methodological quality of studies and tools on measurement properties. Note that when it comes to patient-reported outcome measures, three domains become fundamental: reliability, validity, and responsiveness. To be more precise, the checklist consists of 12 boxes; 10 can be used to assess if the study meets the criteria for methodological quality and 9 can provide standards for the measurement properties of the actual study. Note that these criteria are internal consistency, reliability, measurement error, content validity, structural validity, hypotheses testing, cross-cultural validity, criterion validity, and responsiveness, plus interpretability. The list contains two more boxes: a box for IRT methods and another one for generalizability. Here we should mention that internal consistency was defined as the interrelatedness among questions. Content validity, on the other hand, was defined as the extent to which the content of a tool is an adequate reflection of the construct to be measured, as well as the hypotheses tested. Note that content validity includes the structural validity, hypotheses testing, and cross-cultural validity presented above. Criterion validity was defined as the degree to which the scores of a health-related patient-reported outcome instrument are an adequate representation of a “gold standard.” It’s important to mention that in the case of patient-reported outcomes, we can talk about a “golden standard” only when a shorter version is compared to the original version of the test. Last but not least, responsiveness was defined as the ability of a patient-reported tool to detect differences over time (in any construct being assessed).

To complete the COSMIN checklist, there are four steps experts must follow:

• The first step is to determine which measurement properties are being assessed
• Then experts must decide if the statistical analyses used in the study of interest are based on Classical Test Theory (CTT) or on Item Response Theory (IRT)
• The third step is to complete the boxes of the checklist (with standards that accompany the properties chosen)
• Last, experts should assess the generalizability

The COSMIN Checklist has numerous applications in research and practice. It can be used to select adequate instruments, design a clinical study, report findings (based on good measurement properties), or review a scientific paper. In addition, the checklist can be used for educational purposes or research appraisal. It’s not a secret that health status measurements should be valid and reliable in order to provide precise results, which can support healthcare. Note that usually, health outcomes encompass different modes of data collection: there are tools administered by an interviewer, delivered by a computer, self-administered, and performance-based methods.

When it comes to subjective measures, such as patient-reported outcomes, standards on measurement properties become essential to choose the most appropriate instrument. Interestingly, Marshall was one of the first experts to report bias due to the nature of the actual measurements. He revealed that in schizophrenia clinical trials, scientists were more likely to report that treatment was superior, particularly when the research team utilized an unpublished measurement instrument rather than a published and validated tool (Mokkink et al., 2010). The COSMIN can also ensure the standardization of cross-cultural adaptations and comparisons. Such instruments are multidimensional and subjective, so evaluating measurement properties can only improve practice and patient outcomes.

With digital health empowering patients, patient-reported outcomes measures (PROMs) have become essential tools in research. It’s not a secret that some symptoms are known only to the patients themselves. Due to the subjective nature of these measures, though, reliability and validity may suffer (Prinsen et al., 2018). In addition, experts often conduct systematic reviews to choose a tool that suits their objectives and methodology, which is a complex process prone to unclarity and bias. Note that bias can lead to a waste of resources and unethical decisions.

Therefore, COSMIN standards can only facilitate practice and literature searches. It’s interesting to mention that the research committee, which conducted the initial Delphi study and created the COSMIN checklist, keeps updating the COSMIN guidance on a regular basis. To set an example, the research team formulated a COSMIN Risk of Bias Checklist and ten steps to support the actual process of performing systematic reviews of patient-reported outcome measures. These steps are:

• Formulate the aim of the review (with a focus on the quality of the tool)
• Define the eligibility criteria
• Conduct a literature search
• Select articles and abstracts
• Measure content validity
• Evaluate internal structure
• Assess other measurement qualities, such as reliability, measurement error, etc.
• Describe interpretability and feasibility
• Give recommendations for future improvement and standardization
• Report the findings of the systematic review

Moved by the lack of clarity around measurement properties, inconsistency in standards, and poor evidence in patient-reported outcomes, the COSMIN initiative developed clear guidance and an engaging checklist to support evidence-based research. As the number of patient-reported outcomes measures is increasing, experts need a good methodology to conduct systematic reviews and clinical studies. The initial COSMIN Checklist with its criteria (internal consistency, reliability, measurement error, content validity, structural validity, hypotheses testing, cross-cultural validity, criterion validity, responsiveness, interpretability, IRT methods, and generalizability) provides clear standards to assess the measurement properties of any subjective tool and the methodological quality of any study that employs such methods.

To sum up, the COSMIN checklist has become one of the main factors for scientific success. Standards are constantly being updated, embracing the newest advancement in research and digital health.

Mokkink, L., Terwee, C., Patrick, D., Alonson, J., Stratford, P., Knol, D., Bouter, L., & de Vet, H. (2010). The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.

Quality of Life Research, 19(4), p. 539-549.

Prinsen, C., Mokkink, L., Bouter, L., Alonson, J., Patrick, D., de Vet, H., & Terwee, C. (2018). COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27(5), p.1147-1157.

Real-world data can benefit numerous aspects of research, including drug discovery. Drug discovery can be defined as the process of identifying new compounds and medications. Once a novel compound has been identified, the process of drug development can continue to establish a new medication on the market. Note that the newest advancements in scientific research and health technology have shifted the focus of research. Scientists focus on how a certain disease can be controlled on a molecular level and employ real-world data to support continuing research. Getting a better understanding of any disease is essential. Therefore, it’s alarming that, according to recent findings, seven out of eight compounds used in the clinical testing pipeline might be unsuccessful. Although pharmaceutical companies tend to employ real-world data mainly in post-marketing research, real-world evidence becomes eminent. Real-world data can benefit the entire cycle of drug discovery and development, with the following areas of implementation  (Wise et al., 2018):

Clinical research is complex. Randomized clinical trials are still defined as the gold standard in research. Although controlled medical studies provide valuable insights into drug efficacy, data obtained from real-world settings and diverse populations becomes mandatory to assess drug effectiveness. When it comes to drug discovery, real-world data can complement scientific findings and help researchers generate hypotheses and recruit participants. Since pharmaceutical bodies need to accommodate real-world data, pragmatic trials (that assess drug effectiveness in routine clinical practice) become a valuable approach. Pragmatic trials can decrease costs and improve generalizability because the focus of research is no longer on highly selective samples or strict inclusion/exclusion criteria. Note that a recent study, which tested a new inhaled therapy for chronic obstructive pulmonary disease, assessed drug effectiveness in real settings, revealing that real-world data can decrease costs and improve health outcomes. In fact, real-world data can benefit both the pre- and post-approval process for new drugs. In the post-approval aspect, for instance, real-world evidence can help experts reach a balance between drug safety, governmental regulations, and patient outcomes.

Real-world data in drug discovery can benefit pharmacoepidemiology (Toh, 2017). Note that pharmacoepidemiology is an ambitious discipline that connects pharmacology and epidemiology. One of the main goals in pharmacoepidemiology is to assess the benefits of a new drug in large populations. It’s interesting to mention that pharmacovigilance is defined as a subdiscipline of  pharmacoepidemiologyp. Recent findings show that more and more regulatory bodies and research initiatives, such as the European Union-Adverse Drug Reaction project, integrate real-world data and mining of clinical databases in scientific knowledge. Such data can help experts explore drug safety and drug discovery. What’s more, with the increasing use of electronic health records and digital health tools in practice, smart real-world data becomes essential in medical research.

Real-world data can improve the entire nature of disease taxonomy or disease classification. With novel medical practices focusing on the molecular level of diseases, the current Classification of Diseases (ICD) has started incorporating molecular findings to improve disease classification and drug discovery. This approach will help researchers plan a clinical trial based on relevant characteristics and allow participants to access effective treatments. Since real-world data covers a wide variety of sources (e.g., medical history, patient outcomes), stakeholders, researchers, and patients are willing to share and employ real-world data in order to benefit disease taxonomy and patient care.

Real-world data in drug discovery has a wide range of benefits, particularly in the field of quantitative systems pharmacology (Geerts & Spiros, 2015). Note that quantitative systems pharmacology aims to explore the effects of a novel drug by incorporating mathematical models, pharmacological information, and biological systems. In fact, this research discipline reveals numerous insights into drug design, biomarkers, and dosage. One of the challenges which quantitative systems pharmacology faces, though, is associated with data collection. Experts need to understand how to validate and incorporate real-world quantitative data. Since wearable devices and mHealth apps are among the most popular tools in health research and practice, data that comes directly from patients and their electronic devices can be abundant and unstructured. Security concerns and data privacy should be considered as well. Interestingly, recent studies show that when it comes to drug discovery, patients are willing to share their data and contribute to science.

Digital health is becoming more and more personalized, an approach known as precision medicine. In fact, real-world and genomic data can revolutionize precision medicine (Agarwala et al., 2018). Researchers have started to take into account individual preferences, genetic differences, and histories to analyze drug effectiveness. Modeling and simulation techniques can also benefit precision medicine and help researchers identify patients who would respond to therapy before the actual clinical trial. To set an example, a novel study used genetic sequencing to identify suitable participants with gastrointestinal cancer. With the newest advancements in health technology, tech solutions foster precision medicine and drug discovery. For instance, the mining of electronic health records can be supported by the use of controlled vocabularies and natural language processing. Wearables, on the other hand, can capture a wide range of subjective and objective data off-site. Hence, artificial intelligence can help researchers extract, validate, and exchange data across diverse datasets. To support interoperability and drug discovery, standards for data exchange and safety should be implemented in practice.

Social media plays a crucial role in people’s lives, including well-being. Social media channels (e.g., Facebook, Twitter) are a powerful source of real-world data. To be more precise, the mining of social media sources has numerous benefits (Limaye & Saraogi, 2018). Such information can provide insights into pharmacovigilance, post-marketing practices, and people’s lives. For instance, data collected via social listening can give insights into important medical questions (e.g., why patients switch between therapies), which can foster drug discovery and development. Yet, researchers need to tackle factors, such as data validation, unstructured data, grammar mistakes, and privacy concerns, which become obstacles in social listening. Note that specialized health channels and online platforms are highly valuable as they provide structured and tailored medical information.

Real-world data can benefit all areas of the entire drug lifecycle, particularly drug discovery. Interestingly, FDA has already started to employ real-world data regarding approval decisions about new health treatments devices. Nevertheless, as real-world sources are often unstructured and unregulated, artificial intelligence and other digital solutions should be employed. Digital tools can help researchers access structured information across a wide range of sources. Experts agree that regulatory guidance is also needed to provide effective ways of standardizing real-world evidence. Legal issues of data ownership and technical concerns about data security need to be considered, with cybersecurity being one of the leading factors to address in research. Interestingly, it’s been proven that blockchain technology can facilitate data exchange and integrity. In fact, interoperability is another challenge which researchers need to tackle. Since real-world data sources fluctuate, with information being difficult to access or exchange, there’s a need for a global catalog or marketplace (Wise et al., 2018). Consequently, researchers have started to develop a global catalog for data, as well as a common data model for noisy and unstructured data.

The most important aspect in drug discovery is to embrace the influence digital health technology has on medical research. A common platform, for instance, can eliminate the need for data wrangling or the need for organizing a wide range of existing real-world data sources. In addition, any cloud-based platform for biomedical data can be regulated by clear business models on how to purchase data once and reuse it across disciplines. Note that incompatibility between platforms can be addressed by the integration of standards, such as Health Level Seven (HL7) and Fast Healthcare Interoperability Resources (FHIR), as well as clear roles and methodologies. These aspects of research, of course, require specific research skills for the exploitation of real-world data and Mendelian randomization. Most of all, researchers need to embrace the fact that participants have become the center of medical research and drug discovery. Real-word data should recruit people whose needs haven’t been met by current treatments, speed recruitment, and development, and optimize relevant results at an affordable cost.

From drug discovery to marketing authorization, real-world data has numerous applications in drug discovery. Biopharmaceutical companies admit that real-world data is becoming an integrated part of the development lifecycle of their products. It can benefit disease taxonomies, drug effectiveness, and post-marketing practices. Real-world data can also establish endpoints and outcomes that are patient-centered, which is a leading focus of digital health research. In fact, patients can authorize data sharing from apps and wearables. Note that studies show that patients are eager to share their medical information. The implementation of technology allows the collection of high-quality data and licensing  (Wise et al., 2018). At the same time, as explained above, real-data implementation comes with numerous practical challenges. Stakeholders should reveal a medicine’s real-world value. Pharmaceutical companies should increase patient well-being, decrease costs, and focus on long-term benefits. Regulatory bodies, health technology assessment agencies, and biopharmaceutical companies should develop a common language and standard practices. Simply because investing in real-world data will lead to better health outcomes.

In the end, real-world data can complement scientific findings, foster drug discovery, and save people’s lives.

Agarwala, V., Khozin, S., Singal, G., O’Connell, C., Kuk, D., Li, G., Gossai, A., Miller, V., & Abernethy, A. (2018). Real-World Evidence in Support of Precision Medicine: Clinico-Genomic Cancer Data as A Case Study. Health Affairs, 37 (5).

Geerts, H., & Spiros, A. (2015). From Big Data to Smart Data. How Quantitative Systems Pharmacology Can Support CNS Pharmaceutical R&D. Journal of Pharmacokinetics and Pharmacodynamics.

Limaye, N., & Saraogi, A. (2018). How Social Media Is Transforming Pharma and Healthcare. Applied Clinical Trials, 27 (2).

Toh, S. (2017). Pharmacoepidemiology in the Era of Real-World Evidence. Current Epidemiology Reports, 4 (4), p. 262-265.

Wise, J., Moller, A., Christie, D., Kalra, D., Brodsky, E., Georgieva, E., Jones, G., Smith, I., Greiffenberg, L., McCarthy, M., Arend, M., Lutteringer, O., Kloss, S., &Arlington, S. (2018). The positive impacts of Real-World Data on the challenges facing the evolution of biopharma. Drug Discovery Today, 23 (4), p. 788-801.

Repeatability is a paramount factor which reveals the consistency of any clinical method. In other words, repeatability unveils if the same instrument used in the same subject more than once will lead to the same results (Peat, 2011). Note that the term repeatability was introduced by Bland and Altman, after which various terminology was created in a similar way.

Although terms, such as repeatability, reproducibility, reliability, consistency and test-retest variability, can be used interchangeably – there are some slight differences. Repeatability, for instance, requires the same location, the same tool, the same observer, and the same subject. Consequently, the repeatability coefficient reveals the precision of a test and the difference between the two repeated tests findings over a short period of time. To test repeatability in continuous data – statistics, such as the intraclass correlation coefficient and Levine’s test of equal variance, can be utilized. For categorical data – kappa and proportion in the agreement can support research. Reproducibility, on the other side, refers to the ability to replicate medical studies. In other words, it reveals the agreement between results – obtained from different subjects, via different tools, and at different locations (“What is the difference between repeatability and reproducibility?” 2014).

Data types also matter. As explained above, in case of continuously distributed measures, measurement error (or standard error of the measurement (SEM)) and intraclass correlation coefficient (ICC) are the two most effective indicators of repeatability (regarding the reliability of a measure). The measurement error reveals any within-subject test-retest variation. Note the measurement error is an absolute estimate of the absolute range in which the true value can be found. On the other hand, the ICC is defined as a relative estimate of repeatability. To be more precise, it reveals any between-subject variance to the total variance for continuous measures. When it comes to interpretation, a high ICC means that only a small proportion of the variance is due to within-subject variance. In fact, ICC close to one means there’s no within-subject variance.

For categorical data, there are also various methods – with kappa being one of the sufficient statistics. Basically, kappa is similar to ICC but applicable to categorical data. Thus, kappa values close to one indicate total agreement (Peat, 2011). Note that repeatability in categorical measurements is also called misclassification error.

Medical research is a curious niche full of unexpected outcomes. Variations and errors are quite common. Variations may occur even when the same subject is tested twice via the same tool. Such discrepancies might be a result of various factors: within-observer variation (intra-observer error), between-observer variation (inter-observer error), within-subject variation (test-retest error), and actual changes in the subject after a certain intervention (responsiveness). To be more precise, variations may occur due to changes: in the observer, the subject or the equipment.

Consequently, it’s hard to analyze true values. To guarantee the accuracy, any good study design should ensure that more than one measurement will be taken from each subject to assess estimate repeatability (Peat, 2011).

Selection bias affects the repeatability scores. Therefore, studies that have different subject selection criteria cannot be compared. At the same time, estimates of studies with three or four repeated measures cannot be compared to studies with two repeated measures.

Note that estimates of ICC may be higher and estimates of measurement error lower if the inclusion criteria lead to variations. To set an example, usually, ICC will be higher when subjects are selected randomly. Researchers should recruit a sample of minimum 30 subjects to guarantee adequate measurements of variance.

Paired data is vital in research. Paired data should be used to measure within-subject variances. The mean values and the standard deviation (SD) of the differences also must be computed. The measurement error can be later transformed into a 95% range. In fact, this is the so-called limits of agreement – or the 95% certainty that the true value for a subject lies within the calculated range (Peat, 2011).

• Paired t-tests are beneficial in assessing systematic bias between observers.
• A test of equal variance (e.g., Levene’s test of equal variance) can be helpful to assess repeatability in two different groups.
• A plot of the mean value is also crucial in order to assess the difference between various measures for each subject. This is an effective method as usually, the mean-vs-difference plot (or Bland-&-Altman plot) is clearer than any scatter plots.
• Note that Kendall’s correlation can add more valuable insights to the study. Kendall’s tau-b correlation coefficient indicates the strength of association that exists between two variables (“Kendall’s Tau-b using SPSS Statistics”).

Nothing is only black and white in medical research. Often, more than two measures are required per subject. In case there are more than two measurements taken per subject, experts should calculate the variance for each subject and after that, any within-subject variances. Note that such values can be calculated via ANOVA.  A mean-vs-standard deviation plot can visualize the results. In addition, Kendall’s coefficient can indicate if there’s any systematic error.

Note that deciding on a reliable measure out of a selection of different measures is a difficult task. In clinical settings, this is a vital process as it may affect patients’ outcomes. Assessing measurement errors is also fundamental. The measurement error indicates the range of normal and abnormal values from the baseline. In fact, these values can reveal either a positive or a negative effect of a treatment (Peat, 2011). Let’s say that previously abnormal values have come close to normal values. One interpretation of this phenomenon can be that a disease has been affected by a treatment in a positive direction.

The ICC is an essential indicator in order to show the extent to which multiple measures taken from the same subjects are related to each other. This form of correlation is also known as a reliability coefficient. As explained above, a high ICC value means that variances are due to true differences between subjects, while the rest due to measurement errors (within-subject variance).

Unlike other correlation coefficients (Pearson’s correlation, for example), ICC is relatively easy to calculate. There are a few methods that can help experts calculate ICC, along with other sufficient computer programs. The first method employs a one-way analysis of variance – it is used when the difference between observers is fixed. The second method that can be beneficial refers to cases when there are many observers – it is based on two-way analysis of variance. There is also a third method which is simplified – based on only two measures per subject.

• In fact, P values can be computed from ICC, which eliminates the need for a test of significance. However, note that the Pearson’s correlation coefficient (R) is often used to describe repeatability or agreement, which could lead to false interpretations. To set an example, in cases when there’s a systematic difference (e.g., the second set of measures which is larger than the first), the correlation can be perfect but yet, the repeatability poor.
• A coefficient of variation, which is the within-subject SD divided by the mean of the measures, may be employed. Still, ICC is a more accurate indicator.
• F tests to show if ICC differs from zero can be performed. Generally speaking, an F test is computed after methods, such as ANOVA and regression, to assess if the mean values of two populations differ (“F Statistic / F Value: Simple Definition and Interpretation”).
• Confidence intervals should be calculated as well to support the data analysis.

To sum up, measurement error and ICC are two paramount indicators in medical research. Since both give different statistics, they should be reported together.

Note that while the ICC is related to the ration of the measurement error to the total SD, the measurement error is an absolute value related to the total SD (Peat, 2011).

The repeatability of categorical data (e.g., the presence of illnesses) collected via surveys and questionnaires is also vital. As explained above, when it comes to categorical data, measurement error is called misclassification error. Note that there are a few requirements which are mandatory: 1) the questions, the mode of administration and the settings must be identical on each occasion, 2) subjects and observers must be blinded to the results, and 3) the time between the test-retest processes must be appropriate. When it comes to the community, the repeatability values should be established in similar community settings (not extreme subsamples). On top of that, patients who are tested quite often should be excluded as they might have all answers well-rehearsed.

Kappa is the most popular statistics, which can reveal the observed proportion in agreement and the estimate correct classification value. Usually, kappa is beneficial for measuring test-retest repeatability of self-administered surveys and between-observer agreement in interviews. Note that a kappa value of zero reveals the chance agreement, while a value of one the perfect agreement. In addition, 0.5 shows moderate agreement, above 0.7 – good agreement, above 0.8 – very good agreement. We should mention that the average correct classification rate is an alternative to kappa: it’s higher than the observed proportion in agreement, and it represents the probability of a consistent reply.

In the end, repeatability and validity go hand in hand. Basically, poor repeatability leads to the poor validity of an instrument and limits the accuracy of results. Thus, many indicators and statistics can be employed. Since causes and outcomes are all interconnected, it’s not recommended to use ICC in isolation – simply because ICC is not a very responsive factor and it is not powerful enough to describe the consistency of an instrument.

We should mention that even a valid instrument may reveal some measurement error. In fact, measurement error has a simple clinical interpretation, which makes it a good statistic to use.

Apart from repeatability, the agreement is another paramount aspect of medical research. The agreement is defined as the extent to which two different methods used to measure a particular variable can be compared or substituted with one another. To set an example, experts should know when measurements taken from the same subject via different tools can be used interchangeably (Peat, 2011). Note that agreement, or comparability of the tests, mainly assesses the criterion and the construct validity of a test. Nevertheless, results can never be identical.

There are numerous statistics which can be explored to measure agreement. There are tables that can guide experts how to employ several effective methods in various situations. For example, just like with repeatability, measurement error, ICC, and paired tests are among the most powerful statistics for continuous data and units the same. Also, Kappa is the main indicator that can help researchers with the analysis of categorical data. On the other hand, in situations when one measure is continuous, and the other one categorical – Receiver Operating Curve (ROC) curve can be employed (“What is a ROC curve?”).

As mentioned earlier, different measures rarely give identical results. Even if experts measure weight via two different scales, figures won’t be exactly the same. Thus, when two measurements have to be used interchangeably or converted from one another, it must be clear how much error there will be after the conversion.

When figures are expressed in the same units, the agreement can be assessed via the measurement error or the mean value of the within-subject variance. Since these measures are calculated within the same group of subjects, methods are similar to the ones for repeatability.

Drawing a mean-vs-differences plot is also an effective method, which can be used along with calculating the 95% of agreement. Note that when two tools do not agree well, this effect might be because they measure different variables or because one of the instruments is unprecise.

Note that when it comes to a poor agreement, consistent bias can be assessed by computing the rank correlation coefficient of a plot. Kendall’s correlation coefficient, for instance, can indicate if the agreement and the size of measurement are related. If the correlation is high, then there’s a systematic bias which varies with the size of the measurement. In case such a relationship occurs, a regression equation can help experts convert measurements. Usually, a regression equation can help researchers explore the connections between sets of data and predict future events. Note that in linear regression, there’s a perfectly straight line.

Calculating the 95%-of-agreement is also essential. As a matter of fact, Bland and Altman defined the limits of agreement as the range in which 95% of the difference can be found (Peat, 2011). Note that a measure with poor repeatability will never agree well with another tool.

Variances are common phenomena in research. As described above, it’s normal for two instruments to express some differences. However, such instruments can be used interchangeably in practice only when this range of differences is not clinically important. In the end, patients’ well-being is the main goal of science.

Medical research is a challenging process, which involves the use of numerous statistical procedures. In fact, even different units should be compared from time to time. Measuring the extent to which different instruments can be used is vital to estimate if one measurement predicts the other.

When it comes to continuous data and units different, linear regression and correlation coefficients are the most accurate statistics, which can help experts check what extent of the variation in one measure is explained by the other measure (Peat, 2011).

Continuous data is crucial, so is categorical information. Categorical measurements and the level of agreement between them can reveal the utility of a test. To be more precise, the ability of a test to predict the presence or the absence of disease is paramount in medicine (Peat, 2011). In clinical settings, methods, such as self-reports, observation of clinical symptoms and diagnostic tests (e.g., X-rays), can help experts classify patients according to a presence of a disease or an absence of a disease (e.g., tuberculosis).

In this case, sensitivity and specificity are two essential statistics as they can be applied to different populations. Sensitivity is defined as the proportion of ill subjects who are correctly diagnosed by a positive test result. Specificity, on the other hand, is the proportion of disease negative patients who are correctly diagnosed by a negative test result. What’s more, these indicators can be compared between different studies, which employ different selection criteria and different testing methods.

Yet, the probability that the measure will reveal the correct diagnosis is the most important aspect, which can be achieved by the positive predictive value (PPV) and the negative predictive value (NPV) of a test (Peat, 2011). Note that PPV and NPV depend on the prevalence of a disease, so they cannot be applied to studies with different levels of prevalence of illness. We should mention that in rare diseases, the PPV will be closer to zero. This effect is because experts cannot be certain if a positive result can actually reveal an existing illness.

The likelihood ratio is the most effective statistic used to compare different populations and clinical settings. The likelihood ratio is defined as the likelihood that certain findings would be expected in subjects with the disorder of interest, compared to subjects without that disease. This statistic incorporates both sensitivity and specificity and reveals how good a test is (e.g., the higher the value, the more effective the test will be). The likelihood ratio can be used to calculate pre-test and post-test odds, which can provide valuable clinical information (“Likelihood ratios”).

Note that all statistics described above, including likelihood ratio, reveal a certain degree of error (Peat, 2011). Therefore, their 95% confidence intervals should be calculated.

Continuously distributed information (such as blood tests) is often needed in practice as it can predict the presence of a disease. Also, in order to predict the presence or the absence of a condition, experts need cut-off values, which can indicate normal and abnormal results. The ROC curve is the most effective method to obtain such information.

Note that in order to draw a ROC curve, the first step is to calculate the sensitivity and the specificity of the measure – for different cut-off points of the variable. The bigger the area under the curve is, the more effective the test is. Also, if experts want to check if one test can differentiate between two conditions, they can plot two ROC curves on the same graph and compare them (Peat, 2011).

Measures of association are also vital in reporting the results. The relative risk (RR), odds ratio (OR) and number needed to treat (NNT) can reveal if there’s a risk of a disease in patients exposed to a certain factor or a treatment (Peat, 2011).

For prospective cohort and cross-sectional studies, relative risk is the most effective statistic to present associations between exposures and outcomes. Note that exposures can be related to personal choice (e.g., drinking) or occupational and environmental risks (e.g., pollutants).

Consequently, relative risk is calculated by comparing the prevalence of an illness in the exposed and non-exposed group. Note that relative risk depends on the time needed for an illness to develop.

Odds ratios are another essential characteristic. It can be employed in case-control studies in which is impossible to calculate the relative risk due to the sampling method. The odds ratios represent the odds of exposure in both cases and controls. Note that in such studies, the prevalence of a disease does not represent the prevalence in the community. Nevertheless, statistical procedures like multiple regression allow experts to apply odds ratios to cohort and cross-sectional studies. When confounders occur, experts may employ adjusted odds ratios – or when confounders have been removed from the association between risks and outcomes.

Here we should mention that both statistics, relative risk, and odds ratios, are hard to interpret. Due to their complicity, when it comes to 95% confidence intervals, it is recommended to use a statistical program. Note that sometimes both statistics may differ when in practice, the absolute effect of the exposure is the same. On the other hand, they may be statistically the same when in reality, the absolute effect is actually different. Such differences may mislead experts and lead to type I or type II errors. Thus, odds ratios are recommended only for case-control studies and rare diseases (Peat, 2011).

Analysis of medical data might be tricky. While odds ratios are difficult to explore, the number needed to treat is a statistic, which is extremely beneficial in practice. The number needed to treat is defined as the estimate of patients who need to undergo treatment for one additional subject to benefit (Peat, 2011). In other words, this is the number of subjects which experts need to treat to prevent one bad outcome (“Number Needed to Treat”).

Note that the number needed to treat represents the clinical effect of a new treatment. It should balance the costs of treatment and possible negative effects for the controls. The number needed to treat can be calculated from meta-analyses and findings from different studies. There are also formulas to convert odds ratios to a number needed to treat. To set an example, if the number needed to treat for a new intervention equals four (NNT=4), that means that experts need to treat four people to prevent one bad outcome. It’s not surprising that it’s better to save one life for four patients, instead of one life for ten patients (Peat, 2011).

There are different study designs, and in fact, case-control studies are a popular method. Basically, in case-control studies, the matching process of cases and controls is based on confounders. Note that removing any possible effects in the study design is a more effective technique than analyzing confounders at a later stage of the study. Also, we should mention that analyses may employ paired data and methods such as conditional logistic regression.

Another crucial characteristic of matched and paired analyses is the fact that the number of units is the number of matches or pairs – not the total number of subjects. The effect of pairing also has an effect on associations, confidence intervals and odds ratios (Peat, 2011). Usually, matched analyses reduce bias and improve the precision of confidence intervals.

In some situations, more than one control can be used for each case. This technique improves precision. Note that the number of controls is considered for an effective sample size.

Let’s say we have 30 cases and 60 controls; the number of matched pairs will be 60.

When there are non-matched data, experts can use logistic regression and calculate adjusted odds ratios. Note that logistic regression is used when more than one independent variable determines an outcome. The outcome, on the other hand, is a dichotomous variable (e.g., data can be coded as 1 (e.g., pregnant) and 0 (non-pregnant)).

In addition, the differences in outcomes between cases and controls can be calculated via a paired t-test. This type of testing assesses if the mean difference between two sets of observations (each subject is measured twice) is zero. Multiple regression is also beneficial.

Data analysis should be based on accurate statistical methods. It’s unethical to violate information to obtain statistically significant results, which are not clinically important. In cases when the prevalence of a disease is not common, exact methods can be employed. Exact methods can also be used for small sample size and small groups in stratified analyses (Peat, 2011).

The differences between normal methods and exact method are:

• Normal methods rely on big samples
• Normal methods utilize normally distributed data
• The variable of interest in normal methods is not rare
• Exact methods require more complex statistical packages

To explore rare diseases, experts may investigate the incidence or the rate of occurrence of a disease. The incidence reveals any new cases within a defined group and a defined period of time. Usually, since some diseases are rare, this number is expressed per 10,000 or 100,000 subjects (e.g., children less than five years old).

Prevalence, on the other hand, is estimated from the total number of cases – regarding a specific illness, a given population, and a clear time period (e.g., 10% of the population in 2017). Prevalence is affected by factors, such as the number of deaths (Peat, 2011).

Confidence intervals are also paramount in exact methods. As explained above, the 95% confidence intervals are defined as the range in which experts are 95% confident that the true value lies. Usually, the exact confidence intervals are based on the Poisson distribution. This type of distribution helps researchers investigate the probability of events in a certain period.

When we need to explore the association between a disease and other factors, such as age, we can employ a contingency table. This helps the cross-classification of data into a table in order to visualize the total number of participants in all subgroups. Note that chi-square is a statistic that is of great help. Usually, Pearson’s chi-square is used for large samples (more than 1000, with five subjects in each cell of the table). Continuity adjusted chi-square, on the other hand, can be adjusted for samples under 1000. Last but not the least, Fischer’s chi-square is applicable when there are less than five subjects in each case. Chi-square tests are also used to analyze subsets of information (Peat, 2011).

Reporting the results is a complicated process. Data types and statistical procedures may challenge scientific findings. Nevertheless, experts should always aim for accuracy – with the sole purpose to improve patients’ well-being.

Bartlett, J., & Frost, C. (2008). Reliability, repeatability and reproducibility: analysis of measurement errors in continuous variables. Ultrasound in Obstetrics and Gynecology, 4, p. 466-75.

Kendall’s Tau-b using SPSS Statistics. Retrieved from https://statistics.laerd.com/spss-tutorials/kendalls-tau-b-using-spss-statistics.php

Likelihood Ratios. Retrieved from https://www.cebm.net/2014/02/likelihood-ratios/

Number needed to treat. Retrieved from https://www.cebm.net/2014/03/number-needed-to-treat-nnt/

Peat, J. (2011). Reporting the Results. Health Science Research: SAGE Publications, Ltd.

Statistic / F Value: Simple Definition and Interpretation. Retrieved from http://www.statisticshowto.com/probability-and-statistics/f-statistic-value-test/

What is a ROC curve? Retrieved from https://www.theanalysisfactor.com/what-is-an-roc-curve/

What is the Difference Between Repeatability and Reproducibility? (2014, June 27). Retrieved from https://www.labmate-online.com/news/news-and-views/5/breaking-news/what-is-the-difference-between-repeatability-and-reproducibility/30638

Request

When it comes to calculating the In general, the sample needs to be big enough to guarantee the generalizability of results, and small enough to answer the research questions via the research sources available (Peat, 2011). However, calculating the sample size is always prone to errors, as explained above. In fact, calculating the sample size is a subjective process. For example, in large samples, some outcomes may appear statistically significant, while in clinical settings, they are unimportant. On the other hand, small samples may reveal some important clinical differences, which due to the small sample size do not show any statistical significance.

Experts need to be familiar with such issues to avoid them. In fact, the problems presented above are known as oversized and undersized studies and clinical trials. What’s more, when the study is oversized, type I error may occur. Type I error is defined as the wrong rejection of a true null hypothesis. To be more precise, this happens when the null hypothesis is true, but researchers reject it and accept the alternate one, which is the hypothesis explored by their team. Thus, oversized studies may waste resources and become unethical due to any excessive enrollment of subjects. On the other side, when the study is undersized, both type I and II errors may occur. Type II error is defined as the inability to reject a false null hypothesis. In other words, researchers may fail to reject the null hypothesis, which is untrue when compared to the alternate hypothesis. In fact, a small sample will often lead to inadequate statistical analyses. Undersized studies may also become unethical – simply because they won’t be able to fulfill the research goals of the study (Peat, 2011). Note that when sampling errors occur, it’s better to terminate a study rather than waste resources or mislead subjects.

Since the main goal in medicine is to provide clear information about medical practices and patient care, medical literature is a source of knowledge and expertise. At the same time, the load of articles is increasing rapidly, so it’s hard to track accurate and relevant literature. Thus, critical appraisal of literature can help experts distinguish quality data from flawed experiments, which can mislead and harm evidence-based practice (Umesh et al., 2016).

Therefore, apart from designing a sophisticated study design and applying accurate research methods, critical appraisal skills are crucial for success.

Critical appraisal is defined as the application of various scientific rules to assess the validity of a study and its findings. In other words, critical appraisal is the evaluation of the scientific merit of a medical study (Peat, 2011).

The most important step during any critical appraisal is to identify if the relationship between variables is causal or due to other effects, such as confounding, bias or chance. What’s more, the following steps are paramount to decide: 1) if a medical article is a valuable source of information; 2) if the employed research methods are valid, and 3) if the literature evidence is enough in order to implement changes in clinical practice (Peat, 2011):

• Identify goals and hypotheses
• Identity study designs and research methods
• Assess the criteria for inclusion, exclusion, and sample size
• Assess sources of bias and confounding
• Appraise statistical methods and results
• List strengths and weaknesses and draw a conclusion

A critical appraisal can help researchers prioritize medical research and justify the need for a new study or intervention. In case experts identify poor practices with poor efficacy and effectiveness, the gaps in knowledge should be addressed. Consequently, more rigorous studies should be designed and conducted.

Note that peer review is also a valuable aspect for any successful critical appraisal.

Systematic reviews are also fundamental in medicine. They are defined as the procedure of selecting and combining the evidence from the most rigorous studies available (Peat, 2011). Often narrative reviews and editorials offer literature that supports the researcher’s point. However, systematic reviews that include all relevant studies and meaningful results should be presented.

Just as with critical appraisal, a checklist is also available to help experts. After articles have been selected, their results can be combined via a meta-analysis and a combination of odds ratios. Note that systematic reviews are not necessarily limited to randomized controlled trials.

• Define outcome variables and interventions of interest
• Define search strategies and literature databases
• Define inclusion and exclusion criteria for studies
• Conduct a literature search
• Review of studies by two observers and consensus are a must
• Conduct a review
• Get data and conduct a meta-analysis

Submit and publish the final review

The Cochrane Collaboration is an important part of the assessment of systematic reviews. In fact, this is the gold standard for assessing evidence for healthcare and practice. Up-to-date reviews of trials can save lives. It’s not surprising that the Cochrane collaboration helps volunteers submit reviews and promote high standards in systematic reviews and practice. This approach also aims to: address specific health problems, train experts to avoid duplication, teach efficient search strategies, and support meta-analyses (Peat, 2011).

Thus, the Cochrane collaboration has become an international network, with centers, books, and programs all over the world. Note that reviews are incorporated into the Cochrane Database of Systematic Reviews and the Cochrane Library. There are strict guidelines, of course, to establish rules and avoid duplication. The Cochrane structure compromises of review groups, methods working groups, and centers. We should mention that the review group is a network of experts interested in a topic, that provide their own funding. Anyone can reach a review group, submit a topic for approval and register their title. Then, a protocol should be submitted; after approval, the actual review can be conducted. Consequently, the group critically appraises the review and publishes the results. An editorial team and method working groups are designed to support the review groups. On the other hand, Cochrane centers manage and coordinate the collaboration of researchers worldwide (Peat, 2011).

Evidence-based practice is another important key to medical success and patients’ well-being. It is defined as patient care which is based on evidence of the best studies available. In other words, evidence-based practice uses the best evidence available to deliver patient care. From assessment to clinical trials, evidence-based practice emphasizes the importance of knowledge and data. Simply because inaccuracy and ‘maybe’s’ can put patients at risk. Thus, there are strict rules this approach must follow (Peat, 2011):

• Define the problem and break it down into questions that can be answered
• Select relevant studies and choose the most accurate ones
• Appraise the evidence and focus on reliability, validity, results, generalizability, etc.
• Make clinical decisions and implement changes
• If any information is missing, conduct a new study
• Evaluate the outcome of any changes that have been implemented

Note that the Cochrane collaboration can also support evidence-based practice. In fact, journals like Evidence-Based Medicine can provide appraisals of studies and facilitate research. The evidence-based approach can benefit medical prognoses, interventions, hospitalization rates, testing procedures, and new health-related topics. At the same time, experts must consider costs and risks; and most of all, patients’ well-being.

Reviewing the literature is a challenging task which can benefit medical care. To set an example, critical appraisal combats information overload to improve healthcare practices (Al-Jundi et al., 2017). Although reviewing the literature is a time-consuming process, it’s worth it.

Al-Jundi, A., & Sakka, S. (2017). Critical appraisal of clinical research. Journal of clinical and diagnostic research, 11(5).

Peat, J. (2011). Reviewing the literature. Health science research. SAGE Publications, Ltd.

Reviewing the literature: A short guide for research students. Retrieved from https://uq.edu.au/student-services/pdf/learning/lit-reviews-for-rx-students-v7.pdf

Types of medical literature (2018, April 5). Retrieved from http://research.library.gsu.edu/c.php?g=115556&p=752623

Umesh, G., Karippacheril, J., & Magazine, R. (2016). Critical appraisal of published literature. Indian Journal of Anaesthesia, 60 (9), p. 670-673.

When it comes to calculating the In general, the sample needs to be big enough to guarantee the generalizability of results, and small enough to answer the research questions via the research sources available (Peat, 2011). However, calculating the sample size is always prone to errors, as explained above. In fact, calculating the sample size is a subjective process. For example, in large samples, some outcomes may appear statistically significant, while in clinical settings, they are unimportant. On the other hand, small samples may reveal some important clinical differences, which due to the small sample size do not show any statistical significance.

Experts need to be familiar with such issues to avoid them. In fact, the problems presented above are known as oversized and undersized studies and clinical trials. What’s more, when the study is oversized, type I error may occur. Type I error is defined as the wrong rejection of a true null hypothesis. To be more precise, this happens when the null hypothesis is true, but researchers reject it and accept the alternate one, which is the hypothesis explored by their team. Thus, oversized studies may waste resources and become unethical due to any excessive enrollment of subjects. On the other side, when the study is undersized, both type I and II errors may occur. Type II error is defined as the inability to reject a false null hypothesis. In other words, researchers may fail to reject the null hypothesis, which is untrue when compared to the alternate hypothesis. In fact, a small sample will often lead to inadequate statistical analyses. Undersized studies may also become unethical – simply because they won’t be able to fulfill the research goals of the study (Peat, 2011). Note that when sampling errors occur, it’s better to terminate a study rather than waste resources or mislead subjects.

Before calculating the sample size, researchers need to consider essential factors, such as the power and the probability of their study. The power of the study is the probability of rejecting a false null hypothesis. In other words, the power of the study reveals if researchers can detect significant changes and reduce the probability of making type II error.

As a matter of fact, this is a vital practical issue. In clinical settings, type I and type II error can lead to different consequences (Peat, 2011). Let’s explore a study about a new cancer treatment. The null hypothesis will be that both the new and the existing treatment are the same. Type I error will mean that the existing treatment will be rejected, and the new intervention accepted. When a new treatment is less effective and more expensive, type I error can cause further damage to patients. Type II error, on the other hand, will mean that the new treatment won’t be accepted, even though it will be more effective than the existing one. Thus, due to type II error, many patients will be denied the new treatment.

Technological advancements support research and medicine. Nevertheless, although many computer programs can help researchers calculate the sample size, the best way to perform calculations is to use a table. Tables are clear, accurate, and simple. As a matter of fact, using a table can be extremely helpful for chi-square tests and McNemar’s tests. Note that the McNemar’s test is used mainly for paired nominal data or 2×2 tables (e.g., smokers and non-smokers).

Perhaps one of the major factors that experts need to consider is the type of the measured variables. For instance, for categorical values, such as gender (e.g., male and female), the sample size should be doubled in order to provide clinically and statistically powerful results.

Another important aspect when calculating the sample size is determining the confidence interval and prevalence. As explained above, choosing a sample helps experts find a mean that represents the population of interest – without testing the whole population or wasting resources. However, it’s uncertain if the sample would represent the population. Therefore, experts need to determine a confidence interval. Confidence intervals are likely to show the parameters that would apply to the population. They are based on a confidence level, which often is 95%. That means that, hypothetically, after any consequent sampling procedure within the same population, 95% of the cases will represent the true parameters of interest.

Estimating prevalence is also crucial. Prevalence is defined as the burden of a disease in a population (Ward, 2013). For instance, researchers may need to consider the prevalence of Alzheimer’s in the community (Beckett et al., 1992). Thus, research plans need to ensure an adequate number of cases and controls. Describing the process of calculating the sample size is also an important aspect of research (Peat, 2011). Documentation is crucial.

Considering prevalence is a complicated process. This task is even more complicated for rare events. For example, in a new surgical trial, serious adverse effects may not appear at all. However, that does not mean that there are no risks.

In rare events, to calculate the sample size, the upper limit of risk should be agreed upon. The following formula can help; 3/n (n being the sample size). What do these numbers mean? Let’s say that the upper limit of risk is one in ten, which is 10%. Thus, experts would need a sample, which equals 3/n, or in this case, 3 divided by 10% or 3 divided by 0.1. This makes 30 subjects. So, 30 subjects will be required to help experts fulfill the goals of their research.

There are many factors researchers need to consider. Practical limitations, such as compliance with intervention, often become an obstacle to clinical trials. In general, if non-compliance is high, the size of the intervention groups needs to be doubled (Peat, 2011).

Although tables are beneficial, run-in periods are also a technique which can help experts ensure compliance. To be more precise, the method of eliminating non-compliant subjects during the run-in phase (zation) can help experts maintain the power of the study, especially in studies that measure efficacy. Nevertheless, in studies that measure effectiveness, this approach may reduce generalizability. In the end, goals and practice need to be balanced.

Medical research is a complicated process. Deciding on the primary outcome variable is crucial. However, as stated earlier, all factors that affect the topic of interest should be considered. Tables should always be consulted. They can help experts calculate the sample size for continuous outcome variables, for both paired and unpaired variables (Peat, 2011). Note that the effect size is defined as the smallest significant difference between groups. Also, the sample size depends on the standard deviation. Standard deviation is defined as the measure of variability in the collected data. Let’s say that experts need to assess subjects that weigh between 45 kg and 100 kg (Kadam & Bhalerao, 2010). This is a large variability, so a large sample will be needed.

However, if the variables are not normally distributed or if they are non-parametric, standard deviation cannot be calculated (in case there are more than two categories, and when data is collected via Borg and Likert scales). Again, describing the goals and the statistical procedures employed is vital.

Another research method is to balance the number of cases and controls. In rare events, populations, and diseases, the power of the study can be improved by enrolling more controls; for instance, more than two controls for each case. This is extremely helpful for testing the efficacy of new or expensive interventions (Peat, 2011).

Note that trade-off should also be considered. In simple words, the trade-off effect is defined as the decision of losing one quality to gain another.

Odds ratio or the association between an exposure and an outcome is also Szumilas, 2010). Odds ratio (OR) can reveal if an outcome occurs as a result of exposure of interest. In other words, the odds ratio can reveal if a particular exposure is a risk factor for a disease. When the odds ratio is calculated and equals one, then the exposure does not affect the outcome. If this number is higher, the risk is also higher. Interestingly, logistic regression and confidence intervals can also be employed to determine the odds ratio (Peat, 2011).

When stratification based on confounders or matched case-control studies are performed, we should mention that overmatching is not a good approach. Overmatching can lead to low efficacy.

Correlation coefficients are also helpful to calculate the sample size. Again, tables are vital. In general, correlations show how strong the relationship between the two variables is. Usually, in linear regression analysis, Pearson’s correlation is used as a valuable indicator.

Note that the p-value should differ from zero to be significant (Peat, 2011). The p-values are described as the level of significance. Usually, p<0.05 is accepted as significant. That means that the probability of observing changes due to chance (not intervention) is 5%. As a matter of fact, as explained earlier, statistically significant associations do not always indicate clinically important differences.

When calculating the sample size, no matter what variables or procedures have been employed, repeatability and agreement are two factors that should be considered.

To ensure repeatability, experts can increase the sample size. For studies with insufficient subjects, the measurements employed for each subject can be increased. Usually, a sample of 30 is the minimum. On the other hand, to ensure agreement between two continuously distributed measurements, a sample of 100 is acceptable (Peat, 2011). Of course, more subjects are needed for categorical data.

Analysis of variance (ANOVA) and multivariate analysis of variance (MANOVA) are two popular statistical procedures that play a significant role in the calculation of the sample size. ANOVA can be used to test the differences between the mean values of various groups, e.g., different treatments. While ANOVA is used to test only one dependent variable at a time, MANOVA can be used for various variables at the same time. When it comes to results, note that a size effect of 0.1-0.2 is considered to be small, 0.25-0.4 medium, and 0.5 large.

For MANOVA, an ad-hoc method is needed. Usually, when researchers use ad hoc tests, that means that the method employed works only for the specific purpose it was designed for. In fact, ad hoc means “for a particular purpose only” (“Ad Hoc Analysis and Testing,” 2015).

The time to event or survival time can vary between days and years.

Since the number of deaths is the focus of interest, experts can either increase the number of subjects or the length of the follow-up period. As explained above describing sample size calculations is vital – including factors, such as the levels of significance, the power of the study, the expected effect size, and the standard deviation in the population.

Clinical trials are incredibly complex and involve numerous ethical issues. Therefore, experts can conduct a statistical analysis before the actual recruitment of participants. This method is known as interim analysis (Peat, 2011).

Interim analyses can be employed to help experts decide to continue a clinical trial or not. This can prevent failures at later stages and cut costs. Also, interim analyses can be used to check and reassess the planned sample size and recruitment process (Kumar & Chakraborty, 2016). Nevertheless, the number of interim analyses should be limited and decided prior the actual study. What’s more, since such analyses must unbiased, an independent monitoring committee can be asked to perform them.

Internal pilot studies can be performed to calculate the sample size as well. Such studies involve the first patients enrolled. By analyzing the results obtained from the first subjects, experts can calculate the variance in the reference group and recalculate the sample size. Note that experts need to be blinded to the results. Also, it’s important to understand that these results should not be used separately to test the study hypothesis, but they should be included in the final analysis (Peat, 2011). By recalculating the sample size, the power and the efficacy of the study increase, which spares lots of efforts and sources. Depending on the study goals, a preliminary analysis can be done with 20 subjects for a study with a total number of 40 participants. At the same time, it can include 100 subjects for a study of 1000 participants.

Also, professionals should differentiate classical pilot studies from internal pilot studies. Usually, pilot studies are conducted before the actual study to test if the recruitment procedure and the equipment employed are effective. While results obtained from a classical pilot study are not included in the analysis, results from internal pilot studies are used in the final analysis.

Clinical trials aim to improve medicine and find the best treatment. However, new medications may have various unknown side effects. To tackle the problem of possible adverse effects, safety analysis is paramount (Peat, 2011).

Usually, after the recruitment of the sample size, experts need to perform a safety analysis, with results being interpreted by an external monitoring committee. In the end, risk-benefit assessment is crucial in medicine.

Equipoise is another principle that is vital and in favor of patients’ well-being. Equipoise shows the uncertainty in the minds of the researchers. In fact, clinical equipoise has been proposed as a solution to randomization and clinical merits of an intervention (Hey et al., 2017). Ethical considerations should always come first, and patients who enroll should not worry about receiving inferior treatment or insufficient care.

However, stopping a study should follow established rules. Sometimes, by continuing, adjusting confounders, and using subgroups, further analyses can reveal potential benefits.

In conclusion, calculating the sample size is a complex process. In the end, patients are not only numbers but human beings.

Since preliminary analyses can reveal some valuable results, experts may decide to stop a study. This decision can be based on both statistical and ethical issues. For instance, if an interim analysis shows some toxicity of a new treatment, researchers will not recruit any more subjects. Although a larger sample is needed to answer all questions about efficacy, subjects cannot be exposed to risks. Apart from possible risks, clinical trials can be stopped prematurely if obvious differences are revealed, or respectively, non-significant results are obtained early.

Stopping a study is a delicate topic, and an external committee needs to be consulted. In fact, some studies have been stopped prematurely without a real reason behind it. Thus, clear rules should be established. In general, to avoid a false positive result, the decision to stop a study should be based on a high level of significance or small p-values.

Ad Hoc Analysis and Testing (2015, September, 27). Retrieved from http://www.statisticshowto.com/probability-and-statistics/find-sample-size/

Beckett, L., Scherr, P., & Evands, D.  (1992). Population prevalence estimates from complex samples. Journal of Clinical Epidemiology, 45(4), p. 393-402.

Hey, S., London, A., Weijer, C., Rid, A., & Miller, F. (2017). Is the concept of clinical equipoise still relevant to research? BMJ.

Kadam, P., & Bhalerao, S. (2010). Sample size calculation. International Journal of Ayurveda Research, 1(1), p. 55-57.

Kumar, A., & Chakraborty., B. (2016). Interim analysis: A rational approach of decision making in clinical trial. Journal of Advanced Pharmaceutical Technology & Research, 7(4), p. 118-122.

Peat, J. (2011). Calculating the Sample Size. Health Science Research, SAGE Publications, Ltd.

Sample Size in Statistics (How to Find it): Excel, Cochran’s Formula, General Tips (2018, January 15). Retrieved from http://www.statisticshowto.com/probability-and-statistics/find-sample-size/

Szumilas, M. (2010).  Explaining Odds Ratios. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 19(3), p. 227–229.

Ward, M. (2013). Estimating Disease Prevalence and Incidence using Administrative Data. The Journal of Rheumatology, 40(8), p. 1241–1243.

Aims and hypotheses: Research ideas should be clear, and hypotheses should be testable. The very first section of the protocol must be elegant as it sets the direction of the whole document. When it comes to hypotheses, three is the maximum number to ensure clarity. Note that hypotheses which are complex, or which contain multiple causes, reflect unclear thinking. On top of that, hypotheses should differ for experimental designs and descriptive studies (Peat, 2011). Experts can employ a null hypothesis, which states there aren’t any significant differences between variables, or a priori/alternate hypothesis, which states the direction of the relationship between variables. Hypotheses can be numbered to ensure readability. In the end, the significance of the study (e.g., prevention) should be explained.

Background: The background section should be intriguing and sophisticated – just like an introduction in a study journal. As a matter of fact, this section should sell the study. Most of all, experts must explain in the protocol what the study will achieve and why – with an emphasize on any new information the study may acquire. Stating personal experience in the field is also recommended. Note that topic sentences can be used to clarify short paragraphs in order to foster readability.

Research methods: Research methods should match the aims of the study, and the process should follow certain time periods. Aspects, such as recruitment, sample size, generalizability, and confounders, should be described in detail. The inclusion and exclusion criteria should be listed, which may foster repeatability. Interim analyses, stopping rules, and methods to control for confounders are also vital.

Conducting the study: Another important aspect, which tackles all details of data collection, location, observers, training, documentation, and statistics. Research steps should fulfill the aim of the study and answer the stated research questions.

Statistical procedures: Data should be collected to answer specific research questions. It’s unethical to adjust data only to get statistically powerful results. Data types, variables, and missing data should be considered and documented. The interpretation of data should also be explained in the study protocol.

Budget and staff requirements: Costs should be precise, and various research aspects, such as training, should be considered. Requests and budget should be justified – a factor which can be supported by the actual significance of the study.

Often to conduct a meaningful study, researchers have to develop a new testing tool. To establish the validity of new questionnaires and equipment and to ensure repeatability, methodological studies should be conducted. Note that many measures lack validity and lead to errors and bias. Thus, experts should not rely only on established and convenient instruments. Methodological studies focus on such issues and help researchers access high-quality data. In addition to the main checklist presented above, the following specific requirements should be considered in any methodological study protocol:

• Study objectives: The protocol should focus on repeatability, sensitivity, specificity, agreement, and responsiveness.
• Methods: Risks and pilot studies should be considered. The development of surveys and a timetable for data collection should be explained in detail.
• Reducing bias: This is another important aspect that the study protocol should consider. From blinding procedures to randomization practices – methods should be clear and easy to understand.
• Statistical methods: All statistical steps should be presented. This includes how the results of each data analysis will be used to fulfill the purposes of the study.

In clinical settings, studies that explore the effects of a new treatment are paramount. Experimental clinical studies aim to answer questions related to efficacy and effectiveness. Thus, such studies require high levels of precision. Note that in clinical trials and case-control studies, the selection criteria and allocation process play a crucial role in the generalizability of the results (Peat, 2011). Below are some additional steps to the core checklist, which clinical studies and protocols should follow:

• Study design: All characteristics and matching criteria should be presented.
• Treatment or intervention: Aspects, such as placebo and compliance, should be stated in the study protocol.
• Methods: Sampling methods, recruitment procedures, outcome variables, measurements, pilot studies, and feedback to subjects need to be included in the protocol. Note that the validity of the methods is fundamental.
• Reducing bias and confounders: All factors that may lead to bias and errors should be reported, including procedures to reduce bias and eliminate confounders.
• Statistical methods: Here experts should consider if statistically important differences mean clinically important variations. The type of analysis also matters (e.g., intention to treat). Threshold or dose-response effects should be considered.

The relationship between exposures and outcomes is paramount in healthcare. Thus, epidemiological studies are vital. They are used to measure incidence and prevalence and to assess the effects of all environmental interventions. Here, the sample size is essential as many studies compare populations or subgroups over time (Peat, 2011). Precision is needed to ensure generalizability. When it comes to epidemiological studies, in addition to the main checklist, experts should consider the following specifics:

• Study design: Experts should describe if the study is ecological, cross-sectional, cohort study, or intervention.
• Recruitment: The selection criteria and sampling methods should be discussed.
• Methods: Since medical terms vary, the actual definition used to describe the disease of interest should be stated in a clear manner.
• Measurements: For any exposure and outcome measures aspects, such as repeatability, validity, and applicability, are crucial.
• Reducing bias: The study protocol should explain how experts will increase response rates and improve follow-up procedures.
• Statistical methods: Each method should be explained in detail, including any implications for possible causations.

While ambitious ideas make the world of science spin, funding is the main aspect that can bring innovations into practice. Obtaining funding is a challenging task, though. In order to get funding, research projects should be clear and clinically beneficial. In addition, application protocols should be beautifully prepared and presented (Peat, 2011).

Preparing a good application requires lots of patience, teamwork, and persistence. Paperwork can be burdensome but vital. Having a reliable team can help researchers prepare a grant application. Front-and-back pages (e.g., budget, bibliographies, ethics, signatures) should not be underestimated, and deadlines should not be taken lightly. Note that peer review and editing can be time-consuming.

When it comes to peer review, internal and external peer review can only benefit a grant application. The best option is to consult people who have been involved in research and grantsmanship. On top of that, the application should be presented to people who are not experts. If an application can be understood by people who are not familiar with the research topic, then the application will be easily understood by the people involved in the actual granting process. Although it might be frustrating to receive negative feedback and rewrite time-consuming aspects, always listen to your peers. Yet, try to differentiate useful advice from unscientific and personal comments (Peat, 2011).

Most of all, the study should be beautifully presented. The hypotheses should be clear, the aim of the study should be relevant to clinical practices, and the application should be organized and visually appealing. In fact, good presentation is recommended not only to receive funding but to contribute to the reputation of the research team.

It’s not only about novel ideas and good science. The committee panel receives numerous applications, so papers that are beautifully arranged have a better chance to succeed. There must be a logical flow, charts, and timelines. A topic sentence at the beginning of each paragraph can support readability. Large font, simple language, and sufficient white space are recommended.

The committee panel may consist of people who are not familiar with the research topic. As a matter of fact, only a small number of readers, such as the content experts and the spokesperson, will read the application in detail. They are the ones who’ll influence the decision of the whole granting committee. Thus, any research limitations should be tackled, pilot data presented, and budget justified.

Note that budget is often limited. Thus, justifying the budget (e.g., training, equipment, rewards) is vital. A rounded budget shows inaccuracy and lack of precision. On top of that, when it comes to more expensive tools and staff, the budget should be precisely explained (Peat, 2011).

In the end, conducting a study is one of the most rewarding events for any research team or educational institution. Just like having an article published in a scientific journal, obtaining a funding grant is one of the best rewards for medical experts. Simply because that shows that your grant application has incorporated the best science available, with the sole purpose to improve medicine and patients’ well-being.

Apart from funding, research ethics are another paramount factor which researchers must consider. Ethical principles also play a crucial role in the funding process. Research ethics come before any other interests; therefore, they are clearly defined by governments and committees:

• Medical studies should be approved by ethics committees
• Research staff should be professional and motivated
• The aims of the study should justify any potential risk for the subjects
• Participants should be able to withdraw freely without a risk to their health
• If a treatment proves to be harmful, research should be terminated
• Participants should be informed, and consent sought
• The well-being and feelings of subjects should always come first

Note that medical studies which include vulnerable people, unconscious patients, and children add some additional challenges, which experts should consider (Peat, 2011).

Last but not the least, when it comes to recruitment, consent is essential. It’s unethical to recruit family members or people who cannot refuse to participate. On top of that, reimbursement should not be the only motive for subjects to participate.

To ensure good research ethics, any medical study should be approved by an appropriate ethics committee (Peat, 2011). A committee may consist of ministers of religion, lawyers, clinicians, etc. The committee must ensure that:

• Patients are informed
• Consent obtained
• Any possible risks are justified
• Unethical research prevented

Note that studies are ethical only when researchers are uncertain which treatment is more beneficial for patients. On the other hand, unethical situations may include:

• Conducting studies that have no practical implications
• Starting a new study without considering previous data and findings
• Not following the study protocol
• Conducting a study without a control group or exposing the control group to placebo (instead of standard treatment)
• Testing children or vulnerable people when questions can be answered by adults
• Including measures not approved by the ethics committee
• Enrolling subjects only to get statistically powerful results
• Stopping a study inadequately
• Failing to analyze data and failing to report results

To sum up, planning and conducting medical studies, appraising research protocols and applying for funding, following research ethics and improving patients’ well-being are only a few of the most challenging aspects of research. Nevertheless, science is rewarding.

Peat, J. (2011). Appraising research protocols. Health Science Research. SAGE Publications, Ltd.