developing research questions ppt

• My presentations

Auth with social network:

Download presentation

We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!

Presentation is loading. Please wait.

How to Develop the Right Research Questions for Program Evaluation

Published by Doreen Welch Modified over 8 years ago

Similar presentations

Presentation on theme: "How to Develop the Right Research Questions for Program Evaluation"— Presentation transcript:

Program Evaluation: What is it?

REGIONAL CONFERENCES Planning, Doing, and Using Evaluation.

Proposal Development Guidelines for Signature Grantee Semi-Finalists The Covenant Foundation.

Using Logic Models in Program Planning and Grant Proposals The Covenant Foundation.

Campus Improvement Plans

How to Develop a Program Logic Model. Learning objectives By the end of this presentation, you will be able to: Describe what a logic model is, and how.

How to Evaluate Your Health Literacy Project Jill Lucht, MS Project Director, Center for Health Policy

2014 AmeriCorps External Reviewer Training Assessing Need, Theory of Change, and Logic Model.

Program Evaluation. Lecture Overview  Program evaluation and program development  Logic of program evaluation (Program theory)  Four-Step Model  Comprehensive.

Program Evaluation and Measurement Janet Myers. Objectives for today… To define and explain concepts and terms used in program evaluation. To understand.

Does It Work? Evaluating Your Program

Evaluation.

Outreach Evaluation Series: Community Assessment Susan Barnes and Alan Carr National Network of Libraries of Medicine Outreach Evaluation Resource Center.

Developing a Logic Model

The Lumina Center Grantseeking Workshop Series Presents Outcomes & Evaluations April 20, 2006.

1 Minority SA/HIV Initiative MAI Training SPF Step 3 – Planning Presented By: Tracy Johnson, CSAP’s Central CAPT Janer Hernandez, CSAP’s Northeast CAPT.

2014 AmeriCorps State and National Symposium How to Develop a Program Logic Model.

Evaluation. Practical Evaluation Michael Quinn Patton.

How to Write Goals and Objectives

About project

© 2024 SlidePlayer.com Inc. All rights reserved.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Indian Assoc Pediatr Surg
• v.24(1); Jan-Mar 2019

Formulation of Research Question – Stepwise Approach

Simmi k. ratan.

Department of Pediatric Surgery, Maulana Azad Medical College, New Delhi, India

1 Department of Community Medicine, North Delhi Municipal Corporation Medical College, New Delhi, India

2 Department of Pediatric Surgery, Batra Hospital and Research Centre, New Delhi, India

Formulation of research question (RQ) is an essentiality before starting any research. It aims to explore an existing uncertainty in an area of concern and points to a need for deliberate investigation. It is, therefore, pertinent to formulate a good RQ. The present paper aims to discuss the process of formulation of RQ with stepwise approach. The characteristics of good RQ are expressed by acronym “FINERMAPS” expanded as feasible, interesting, novel, ethical, relevant, manageable, appropriate, potential value, publishability, and systematic. A RQ can address different formats depending on the aspect to be evaluated. Based on this, there can be different types of RQ such as based on the existence of the phenomenon, description and classification, composition, relationship, comparative, and causality. To develop a RQ, one needs to begin by identifying the subject of interest and then do preliminary research on that subject. The researcher then defines what still needs to be known in that particular subject and assesses the implied questions. After narrowing the focus and scope of the research subject, researcher frames a RQ and then evaluates it. Thus, conception to formulation of RQ is very systematic process and has to be performed meticulously as research guided by such question can have wider impact in the field of social and health research by leading to formulation of policies for the benefit of larger population.

I NTRODUCTION

A good research question (RQ) forms backbone of a good research, which in turn is vital in unraveling mysteries of nature and giving insight into a problem.[ 1 , 2 , 3 , 4 ] RQ identifies the problem to be studied and guides to the methodology. It leads to building up of an appropriate hypothesis (Hs). Hence, RQ aims to explore an existing uncertainty in an area of concern and points to a need for deliberate investigation. A good RQ helps support a focused arguable thesis and construction of a logical argument. Hence, formulation of a good RQ is undoubtedly one of the first critical steps in the research process, especially in the field of social and health research, where the systematic generation of knowledge that can be used to promote, restore, maintain, and/or protect health of individuals and populations.[ 1 , 3 , 4 ] Basically, the research can be classified as action, applied, basic, clinical, empirical, administrative, theoretical, or qualitative or quantitative research, depending on its purpose.[ 2 ]

Research plays an important role in developing clinical practices and instituting new health policies. Hence, there is a need for a logical scientific approach as research has an important goal of generating new claims.[ 1 ]

C HARACTERISTICS OF G OOD R ESEARCH Q UESTION

“The most successful research topics are narrowly focused and carefully defined but are important parts of a broad-ranging, complex problem.”

A good RQ is an asset as it:

• Details the problem statement
• Further describes and refines the issue under study
• Adds focus to the problem statement
• Guides data collection and analysis
• Sets context of research.

Hence, while writing RQ, it is important to see if it is relevant to the existing time frame and conditions. For example, the impact of “odd-even” vehicle formula in decreasing the level of air particulate pollution in various districts of Delhi.

A good research is represented by acronym FINERMAPS[ 5 ]

Interesting.

• Appropriate
• Potential value and publishability
• Systematic.

Feasibility means that it is within the ability of the investigator to carry out. It should be backed by an appropriate number of subjects and methodology as well as time and funds to reach the conclusions. One needs to be realistic about the scope and scale of the project. One has to have access to the people, gadgets, documents, statistics, etc. One should be able to relate the concepts of the RQ to the observations, phenomena, indicators, or variables that one can access. One should be clear that the collection of data and the proceedings of project can be completed within the limited time and resources available to the investigator. Sometimes, a RQ appears feasible, but when fieldwork or study gets started, it proves otherwise. In this situation, it is important to write up the problems honestly and to reflect on what has been learned. One should try to discuss with more experienced colleagues or the supervisor so as to develop a contingency plan to anticipate possible problems while working on a RQ and find possible solutions in such situations.

This is essential that one has a real grounded interest in one's RQ and one can explore this and back it up with academic and intellectual debate. This interest will motivate one to keep going with RQ.

The question should not simply copy questions investigated by other workers but should have scope to be investigated. It may aim at confirming or refuting the already established findings, establish new facts, or find new aspects of the established facts. It should show imagination of the researcher. Above all, the question has to be simple and clear. The complexity of a question can frequently hide unclear thoughts and lead to a confused research process. A very elaborate RQ, or a question which is not differentiated into different parts, may hide concepts that are contradictory or not relevant. This needs to be clear and thought-through. Having one key question with several subcomponents will guide your research.

This is the foremost requirement of any RQ and is mandatory to get clearance from appropriate authorities before stating research on the question. Further, the RQ should be such that it minimizes the risk of harm to the participants in the research, protect the privacy and maintain their confidentiality, and provide the participants right to withdraw from research. It should also guide in avoiding deceptive practices in research.

The question should of academic and intellectual interest to people in the field you have chosen to study. The question preferably should arise from issues raised in the current situation, literature, or in practice. It should establish a clear purpose for the research in relation to the chosen field. For example, filling a gap in knowledge, analyzing academic assumptions or professional practice, monitoring a development in practice, comparing different approaches, or testing theories within a specific population are some of the relevant RQs.

Manageable (M): It has the similar essence as of feasibility but mainly means that the following research can be managed by the researcher.

Appropriate (A): RQ should be appropriate logically and scientifically for the community and institution.

Potential value and publishability (P): The study can make significant health impact in clinical and community practices. Therefore, research should aim for significant economic impact to reduce unnecessary or excessive costs. Furthermore, the proposed study should exist within a clinical, consumer, or policy-making context that is amenable to evidence-based change. Above all, a good RQ must address a topic that has clear implications for resolving important dilemmas in health and health-care decisions made by one or more stakeholder groups.

Systematic (S): Research is structured with specified steps to be taken in a specified sequence in accordance with the well-defined set of rules though it does not rule out creative thinking.

Example of RQ: Would the topical skin application of oil as a skin barrier reduces hypothermia in preterm infants? This question fulfills the criteria of a good RQ, that is, feasible, interesting, novel, ethical, and relevant.

Types of research question

A RQ can address different formats depending on the aspect to be evaluated.[ 6 ] For example:

• Existence: This is designed to uphold the existence of a particular phenomenon or to rule out rival explanation, for example, can neonates perceive pain?
• Description and classification: This type of question encompasses statement of uniqueness, for example, what are characteristics and types of neuropathic bladders?
• Composition: It calls for breakdown of whole into components, for example, what are stages of reflux nephropathy?
• Relationship: Evaluate relation between variables, for example, association between tumor rupture and recurrence rates in Wilm's tumor
• Descriptive—comparative: Expected that researcher will ensure that all is same between groups except issue in question, for example, Are germ cell tumors occurring in gonads more aggressive than those occurring in extragonadal sites?
• Causality: Does deletion of p53 leads to worse outcome in patients with neuroblastoma?
• Causality—comparative: Such questions frequently aim to see effect of two rival treatments, for example, does adding surgical resection improves survival rate outcome in children with neuroblastoma than with chemotherapy alone?
• Causality–Comparative interactions: Does immunotherapy leads to better survival outcome in neuroblastoma Stage IV S than with chemotherapy in the setting of adverse genetic profile than without it? (Does X cause more changes in Y than those caused by Z under certain condition and not under other conditions).

How to develop a research question

• Begin by identifying a broader subject of interest that lends itself to investigate, for example, hormone levels among hypospadias
• Do preliminary research on the general topic to find out what research has already been done and what literature already exists.[ 7 ] Therefore, one should begin with “information gaps” (What do you already know about the problem? For example, studies with results on testosterone levels among hypospadias
• What do you still need to know? (e.g., levels of other reproductive hormones among hypospadias)
• What are the implied questions: The need to know about a problem will lead to few implied questions. Each general question should lead to more specific questions (e.g., how hormone levels differ among isolated hypospadias with respect to that in normal population)
• Narrow the scope and focus of research (e.g., assessment of reproductive hormone levels among isolated hypospadias and hypospadias those with associated anomalies)
• Is RQ clear? With so much research available on any given topic, RQs must be as clear as possible in order to be effective in helping the writer direct his or her research
• Is the RQ focused? RQs must be specific enough to be well covered in the space available
• Is the RQ complex? RQs should not be answerable with a simple “yes” or “no” or by easily found facts. They should, instead, require both research and analysis on the part of the writer
• Is the RQ one that is of interest to the researcher and potentially useful to others? Is it a new issue or problem that needs to be solved or is it attempting to shed light on previously researched topic
• Is the RQ researchable? Consider the available time frame and the required resources. Is the methodology to conduct the research feasible?
• Is the RQ measurable and will the process produce data that can be supported or contradicted?
• Is the RQ too broad or too narrow?
• Create Hs: After formulating RQ, think where research is likely to be progressing? What kind of argument is likely to be made/supported? What would it mean if the research disputed the planned argument? At this step, one can well be on the way to have a focus for the research and construction of a thesis. Hs consists of more specific predictions about the nature and direction of the relationship between two variables. It is a predictive statement about the outcome of the research, dictate the method, and design of the research[ 1 ]
• Understand implications of your research: This is important for application: whether one achieves to fill gap in knowledge and how the results of the research have practical implications, for example, to develop health policies or improve educational policies.[ 1 , 8 ]

Brainstorm/Concept map for formulating research question

• First, identify what types of studies have been done in the past?
• Is there a unique area that is yet to be investigated or is there a particular question that may be worth replicating?
• Begin to narrow the topic by asking open-ended “how” and “why” questions
• Evaluate the question
• Develop a Hypothesis (Hs)
• Write down the RQ.

Writing down the research question

• State the question in your own words
• Write down the RQ as completely as possible.

For example, Evaluation of reproductive hormonal profile in children presenting with isolated hypospadias)

• Divide your question into concepts. Narrow to two or three concepts (reproductive hormonal profile, isolated hypospadias, compare with normal/not isolated hypospadias–implied)
• Specify the population to be studied (children with isolated hypospadias)
• Refer to the exposure or intervention to be investigated, if any
• Reflect the outcome of interest (hormonal profile).

Another example of a research question

Would the topical skin application of oil as a skin barrier reduces hypothermia in preterm infants? Apart from fulfilling the criteria of a good RQ, that is, feasible, interesting, novel, ethical, and relevant, it also details about the intervention done (topical skin application of oil), rationale of intervention (as a skin barrier), population to be studied (preterm infants), and outcome (reduces hypothermia).

Other important points to be heeded to while framing research question

• Make reference to a population when a relationship is expected among a certain type of subjects
• RQs and Hs should be made as specific as possible
• Avoid words or terms that do not add to the meaning of RQs and Hs
• Stick to what will be studied, not implications
• Name the variables in the order in which they occur/will be measured
• Avoid the words significant/”prove”
• Avoid using two different terms to refer to the same variable.

Some of the other problems and their possible solutions have been discussed in Table 1 .

Potential problems and solutions while making research question

G OING B EYOND F ORMULATION OF R ESEARCH Q UESTION–THE P ATH A HEAD

Once RQ is formulated, a Hs can be developed. Hs means transformation of a RQ into an operational analog.[ 1 ] It means a statement as to what prediction one makes about the phenomenon to be examined.[ 4 ] More often, for case–control trial, null Hs is generated which is later accepted or refuted.

A strong Hs should have following characteristics:

• Give insight into a RQ
• Are testable and measurable by the proposed experiments
• Have logical basis
• Follows the most likely outcome, not the exceptional outcome.

E XAMPLES OF R ESEARCH Q UESTION AND H YPOTHESIS

Research question-1.

• Does reduced gap between the two segments of the esophagus in patients of esophageal atresia reduces the mortality and morbidity of such patients?

Hypothesis-1

• Reduced gap between the two segments of the esophagus in patients of esophageal atresia reduces the mortality and morbidity of such patients
• In pediatric patients with esophageal atresia, gap of <2 cm between two segments of the esophagus and proper mobilization of proximal pouch reduces the morbidity and mortality among such patients.

Research question-2

• Does application of mitomycin C improves the outcome in patient of corrosive esophageal strictures?

Hypothesis-2

In patients aged 2–9 years with corrosive esophageal strictures, 34 applications of mitomycin C in dosage of 0.4 mg/ml for 5 min over a period of 6 months improve the outcome in terms of symptomatic and radiological relief. Some other examples of good and bad RQs have been shown in Table 2 .

Examples of few bad (left-hand side column) and few good (right-hand side) research questions

R ESEARCH Q UESTION AND S TUDY D ESIGN

RQ determines study design, for example, the question aimed to find the incidence of a disease in population will lead to conducting a survey; to find risk factors for a disease will need case–control study or a cohort study. RQ may also culminate into clinical trial.[ 9 , 10 ] For example, effect of administration of folic acid tablet in the perinatal period in decreasing incidence of neural tube defect. Accordingly, Hs is framed.

Appropriate statistical calculations are instituted to generate sample size. The subject inclusion, exclusion criteria and time frame of research are carefully defined. The detailed subject information sheet and pro forma are carefully defined. Moreover, research is set off few examples of research methodology guided by RQ:

• Incidence of anorectal malformations among adolescent females (hospital-based survey)
• Risk factors for the development of spontaneous pneumoperitoneum in pediatric patients (case–control design and cohort study)
• Effect of technique of extramucosal ureteric reimplantation without the creation of submucosal tunnel for the preservation of upper tract in bladder exstrophy (clinical trial).

The results of the research are then be available for wider applications for health and social life

C ONCLUSION

A good RQ needs thorough literature search and deep insight into the specific area/problem to be investigated. A RQ has to be focused yet simple. Research guided by such question can have wider impact in the field of social and health research by leading to formulation of policies for the benefit of larger population.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

R EFERENCES

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Developing a research question

Your research questions are perhaps the most important part of your study. They guide your choice of methodology and underpin each chapter in your thesis or dissertation. It’s worth investing time to develop robust questions that will guide your research.

Identifying an effective research question

While the criteria for an effective research question vary considerably, generally, a research question should be:

• focused – its scope should be adequately narrow to allow you to carry out your research within the available timeframe and using available resources.
• analytical (as opposed to descriptive) – your research question needs to display enough complexity so that the answers to it cannot be easily obtained. For example, it cannot be answered through a simple internet search or fact check. An effective research question would require answers which are subject to interpretation, analysis and synthesis.
• effectively expressed – the research question needs to use clear, specific and concise language so that it is accessbile to the reader.

Strategies for developing effective research questions

To develop effective research questions, you may like to try one of the following two key strategies:

I) Convert your topic into one or more research questions by:

• breaking down your topic into its different features
• choosing a feature that interests you to narrow down your topic scope
• brainstorming and reading literature around this feature to focus it further
• convert this focused feature into question form.

II) Formulate a problem statement and then convert it into question form. Use the following template as a guide to writing your problem statement:

• I am examining …
• It matters because…

The following presentation shows how to use the above discussed strategies for developing robust research questions. Work through each section of the webinar. Feel free to pause the video at suitable points and complete the included activities.

Presentation on developing research questions (12:54 min)

Developing your research questions (12:54 min) by RMIT University ( YouTube )

Further resources

For more information on research questions, consult the following resources:

• How Research Questions Can Make or Break Your Project (8:36 min) by Professor James Arvanitakis ( YouTube )
• Research Questions Hypothesis and Variables: Connecting the Dots (7:55 min) by Associate Professor Ron Wallace  ( YouTube )

Research and Writing Skills for Academic and Graduate Researchers Copyright © 2022 by RMIT University is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

MERRIMACK COLLEGE MCQUADE LIBRARY

• Get Started

What is a Research Question?

Developing a research question (northern kentucky university), developing a research question (indiana university).

• Find Books / E-books / DVDs
• Find Articles
• Find Video / Film
• Database Search Strategies
• Types of Resources
• Literature Review
• Evaluate Sources
• Cite Sources
• Annotated Bibliography
• PSY Websites
• PSY Organizations
• Government Resources
• Open Educational Resources (OER)

Contact a Librarian

A research question is a statement that defines what is to be studied. It is the core of the research project, study, or literature review. Your research question  focuses the study, determines the methodology, and guides all stages of inquiry, analysis, and reporting.

Your research question should...

• Be focused 
• Identify the problem you're writing about
• Establish significance 

• << Previous: Get Started
• Next: Keywords >>
• Last Updated: Apr 9, 2024 12:15 PM
• URL: https://libguides.merrimack.edu/psychology

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

How to formulate research questions

Related Papers

Dr. Jayanta Ghosh

Rebekka Tunombili

Writing for Publication

Christian Dueñas

A group of higher education faculty members from different colleges and departments were participating in a 3-day professional development institute on writing for professional publication. The pressure to publish was on at their institution, newly categorized as a university. Prior to the mid-morning break on the fi rst day, the presenter asked the participants to write their concerns about publishing on Post-it notes and then read and categorized them before the group reconvened. The great majority of the participants were worried about their ability to fulfi ll the scalating expectations for faculty. Only a few had published previously and they ondered if they were capable of writing well enough to publish their work. As away to allay their fears, the presenter offered to assess a short writing sample from each participant that evening and return it the next day.

Pavel Collado

Sebastian Garcia

Cristian Perez

Tola Boeurt

COURSE DURATION: 63 Hours, 42 sessions, 21 weeks LECTURER: BOEURT CHANTOLA I. COURSE DESCRIPTION This course is designed for advanced level English Learners who wish to effectively find information on every aspect. This course will provide all learners the knowledge of how to collect the information from the different resources. Specifically, the research methodology will seek to enable all learners the comprehensive knowledge of research such as; processing of research, formulating the research problem, conceptualizing a research design, constructing on instrument for collecting the data, writing a research proposal, processing the data, and writing a research report. II. STRATEGIC OBJECTIVES By the end of the course, all the students will be able to:  Work independently to develop complex research project  Analyze critically the structure, argumentation and quality of the thesis or practical project  Identify the methodological approaches employed in the a spirit of intellectual openness and generosity  Present the outline of specific questions developing out their research III. INTENDED LEARNING OUTCOMES Upon completion of the course, students will be able:

Dr. Krishnanaik Vankdoth

Research methods and presentation is a broad term. While methods of data collection and data analysis represent the core of research methods. The most important elements of research methodology expected to be covered in business dissertation at Bachelor’s, Master’s and PhD levels include research philosophy

Davida Scharf

See review in Library Journal. http://reviews.libraryjournal.com/2013/11/reference/wiki-literacy/

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Francisco Javier Leal López

Chitu Okoli

Malehe pourkazemi

Ágnes Szokolszky

Jazmín Guillén

Penny Beile

mnkeni africanus

Proceedings of the International Conference …

Hiroaki Ogata

Aksha Memon

Educational Technology & Society

Lung-Hsiang Wong , Chee Lay Tan , Chee-Kuen Chin

Gwo-dong Chen

Prof.Dr. Bahadır Erişti

Sarada Dhakal

Tùng McLeonard

Xianzhong He

Tejaswini Kotian

glenda crosling

Donald J. Kochan

Veiko Locos

Greg Stoner

US-China Education Review A & B

Dr Dare E Ajayi

Xerxes Daji

Ridwan Osman

Abdilahi Adam Mohamoud

Zahori Conferencista Internacional

Roohullah Nawandish

Dondon B. Buensuceso

Sadman Sakib

Susan Searing

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Home Collections Education Research Questions PowerPoint

Free - Effective Research Questions PPT And Google Slides

Research Questions Presentation Slides

Features of the templates:.

• 100% customizable slides and easy to download
• The slides contained 16:9 and 4:3 formats.
• Easy to change the slide colors quickly.
• Planets animation inserted template.
• Ready-made nodes are given to you.
• Research Questions
• Developing Research Questions
• Research Questions Model
• Research Questions Management
• Research Questions Theme
• Google Slides

43+ Templates

177+ Templates

1295+ Templates

Animals and birds

268+ Templates

Country Flags

46+ Templates

417+ Templates

179+ Templates

Galaxy or Space

124+ Templates

30+ Templates

You May Also Like These PowerPoint Templates

Our approach

• Responsibility
• Infrastructure
• Try Meta AI

RECOMMENDED READS

• 5 Steps to Getting Started with Llama 2
• The Llama Ecosystem: Past, Present, and Future
• Introducing Code Llama, a state-of-the-art large language model for coding
• Meta and Microsoft Introduce the Next Generation of Llama
• Today, we’re introducing Meta Llama 3, the next generation of our state-of-the-art open source large language model.
• Llama 3 models will soon be available on AWS, Databricks, Google Cloud, Hugging Face, Kaggle, IBM WatsonX, Microsoft Azure, NVIDIA NIM, and Snowflake, and with support from hardware platforms offered by AMD, AWS, Dell, Intel, NVIDIA, and Qualcomm.
• We’re dedicated to developing Llama 3 in a responsible way, and we’re offering various resources to help others use it responsibly as well. This includes introducing new trust and safety tools with Llama Guard 2, Code Shield, and CyberSec Eval 2.
• In the coming months, we expect to introduce new capabilities, longer context windows, additional model sizes, and enhanced performance, and we’ll share the Llama 3 research paper.
• Meta AI, built with Llama 3 technology, is now one of the world’s leading AI assistants that can boost your intelligence and lighten your load—helping you learn, get things done, create content, and connect to make the most out of every moment. You can try Meta AI here .

Today, we’re excited to share the first two models of the next generation of Llama, Meta Llama 3, available for broad use. This release features pretrained and instruction-fine-tuned language models with 8B and 70B parameters that can support a broad range of use cases. This next generation of Llama demonstrates state-of-the-art performance on a wide range of industry benchmarks and offers new capabilities, including improved reasoning. We believe these are the best open source models of their class, period. In support of our longstanding open approach, we’re putting Llama 3 in the hands of the community. We want to kickstart the next wave of innovation in AI across the stack—from applications to developer tools to evals to inference optimizations and more. We can’t wait to see what you build and look forward to your feedback.

Our goals for Llama 3

With Llama 3, we set out to build the best open models that are on par with the best proprietary models available today. We wanted to address developer feedback to increase the overall helpfulness of Llama 3 and are doing so while continuing to play a leading role on responsible use and deployment of LLMs. We are embracing the open source ethos of releasing early and often to enable the community to get access to these models while they are still in development. The text-based models we are releasing today are the first in the Llama 3 collection of models. Our goal in the near future is to make Llama 3 multilingual and multimodal, have longer context, and continue to improve overall performance across core LLM capabilities such as reasoning and coding.

State-of-the-art performance

Our new 8B and 70B parameter Llama 3 models are a major leap over Llama 2 and establish a new state-of-the-art for LLM models at those scales. Thanks to improvements in pretraining and post-training, our pretrained and instruction-fine-tuned models are the best models existing today at the 8B and 70B parameter scale. Improvements in our post-training procedures substantially reduced false refusal rates, improved alignment, and increased diversity in model responses. We also saw greatly improved capabilities like reasoning, code generation, and instruction following making Llama 3 more steerable.

*Please see evaluation details for setting and parameters with which these evaluations are calculated.

In the development of Llama 3, we looked at model performance on standard benchmarks and also sought to optimize for performance for real-world scenarios. To this end, we developed a new high-quality human evaluation set. This evaluation set contains 1,800 prompts that cover 12 key use cases: asking for advice, brainstorming, classification, closed question answering, coding, creative writing, extraction, inhabiting a character/persona, open question answering, reasoning, rewriting, and summarization. To prevent accidental overfitting of our models on this evaluation set, even our own modeling teams do not have access to it. The chart below shows aggregated results of our human evaluations across of these categories and prompts against Claude Sonnet, Mistral Medium, and GPT-3.5.

Preference rankings by human annotators based on this evaluation set highlight the strong performance of our 70B instruction-following model compared to competing models of comparable size in real-world scenarios.

Our pretrained model also establishes a new state-of-the-art for LLM models at those scales.

To develop a great language model, we believe it’s important to innovate, scale, and optimize for simplicity. We adopted this design philosophy throughout the Llama 3 project with a focus on four key ingredients: the model architecture, the pretraining data, scaling up pretraining, and instruction fine-tuning.

Model architecture

In line with our design philosophy, we opted for a relatively standard decoder-only transformer architecture in Llama 3. Compared to Llama 2, we made several key improvements. Llama 3 uses a tokenizer with a vocabulary of 128K tokens that encodes language much more efficiently, which leads to substantially improved model performance. To improve the inference efficiency of Llama 3 models, we’ve adopted grouped query attention (GQA) across both the 8B and 70B sizes. We trained the models on sequences of 8,192 tokens, using a mask to ensure self-attention does not cross document boundaries.

Training data

To train the best language model, the curation of a large, high-quality training dataset is paramount. In line with our design principles, we invested heavily in pretraining data. Llama 3 is pretrained on over 15T tokens that were all collected from publicly available sources. Our training dataset is seven times larger than that used for Llama 2, and it includes four times more code. To prepare for upcoming multilingual use cases, over 5% of the Llama 3 pretraining dataset consists of high-quality non-English data that covers over 30 languages. However, we do not expect the same level of performance in these languages as in English.

To ensure Llama 3 is trained on data of the highest quality, we developed a series of data-filtering pipelines. These pipelines include using heuristic filters, NSFW filters, semantic deduplication approaches, and text classifiers to predict data quality. We found that previous generations of Llama are surprisingly good at identifying high-quality data, hence we used Llama 2 to generate the training data for the text-quality classifiers that are powering Llama 3.

We also performed extensive experiments to evaluate the best ways of mixing data from different sources in our final pretraining dataset. These experiments enabled us to select a data mix that ensures that Llama 3 performs well across use cases including trivia questions, STEM, coding, historical knowledge, etc.

Scaling up pretraining

To effectively leverage our pretraining data in Llama 3 models, we put substantial effort into scaling up pretraining. Specifically, we have developed a series of detailed scaling laws for downstream benchmark evaluations. These scaling laws enable us to select an optimal data mix and to make informed decisions on how to best use our training compute. Importantly, scaling laws allow us to predict the performance of our largest models on key tasks (for example, code generation as evaluated on the HumanEval benchmark—see above) before we actually train the models. This helps us ensure strong performance of our final models across a variety of use cases and capabilities.

We made several new observations on scaling behavior during the development of Llama 3. For example, while the Chinchilla-optimal amount of training compute for an 8B parameter model corresponds to ~200B tokens, we found that model performance continues to improve even after the model is trained on two orders of magnitude more data. Both our 8B and 70B parameter models continued to improve log-linearly after we trained them on up to 15T tokens. Larger models can match the performance of these smaller models with less training compute, but smaller models are generally preferred because they are much more efficient during inference.

To train our largest Llama 3 models, we combined three types of parallelization: data parallelization, model parallelization, and pipeline parallelization. Our most efficient implementation achieves a compute utilization of over 400 TFLOPS per GPU when trained on 16K GPUs simultaneously. We performed training runs on two custom-built 24K GPU clusters . To maximize GPU uptime, we developed an advanced new training stack that automates error detection, handling, and maintenance. We also greatly improved our hardware reliability and detection mechanisms for silent data corruption, and we developed new scalable storage systems that reduce overheads of checkpointing and rollback. Those improvements resulted in an overall effective training time of more than 95%. Combined, these improvements increased the efficiency of Llama 3 training by ~three times compared to Llama 2.

Instruction fine-tuning

To fully unlock the potential of our pretrained models in chat use cases, we innovated on our approach to instruction-tuning as well. Our approach to post-training is a combination of supervised fine-tuning (SFT), rejection sampling, proximal policy optimization (PPO), and direct preference optimization (DPO). The quality of the prompts that are used in SFT and the preference rankings that are used in PPO and DPO has an outsized influence on the performance of aligned models. Some of our biggest improvements in model quality came from carefully curating this data and performing multiple rounds of quality assurance on annotations provided by human annotators.

Learning from preference rankings via PPO and DPO also greatly improved the performance of Llama 3 on reasoning and coding tasks. We found that if you ask a model a reasoning question that it struggles to answer, the model will sometimes produce the right reasoning trace: The model knows how to produce the right answer, but it does not know how to select it. Training on preference rankings enables the model to learn how to select it.

Building with Llama 3

Our vision is to enable developers to customize Llama 3 to support relevant use cases and to make it easier to adopt best practices and improve the open ecosystem. With this release, we’re providing new trust and safety tools including updated components with both Llama Guard 2 and Cybersec Eval 2, and the introduction of Code Shield—an inference time guardrail for filtering insecure code produced by LLMs.

We’ve also co-developed Llama 3 with torchtune , the new PyTorch-native library for easily authoring, fine-tuning, and experimenting with LLMs. torchtune provides memory efficient and hackable training recipes written entirely in PyTorch. The library is integrated with popular platforms such as Hugging Face, Weights & Biases, and EleutherAI and even supports Executorch for enabling efficient inference to be run on a wide variety of mobile and edge devices. For everything from prompt engineering to using Llama 3 with LangChain we have a comprehensive getting started guide and takes you from downloading Llama 3 all the way to deployment at scale within your generative AI application.

A system-level approach to responsibility

We have designed Llama 3 models to be maximally helpful while ensuring an industry leading approach to responsibly deploying them. To achieve this, we have adopted a new, system-level approach to the responsible development and deployment of Llama. We envision Llama models as part of a broader system that puts the developer in the driver’s seat. Llama models will serve as a foundational piece of a system that developers design with their unique end goals in mind.

Instruction fine-tuning also plays a major role in ensuring the safety of our models. Our instruction-fine-tuned models have been red-teamed (tested) for safety through internal and external efforts. ​​Our red teaming approach leverages human experts and automation methods to generate adversarial prompts that try to elicit problematic responses. For instance, we apply comprehensive testing to assess risks of misuse related to Chemical, Biological, Cyber Security, and other risk areas. All of these efforts are iterative and used to inform safety fine-tuning of the models being released. You can read more about our efforts in the model card .

Llama Guard models are meant to be a foundation for prompt and response safety and can easily be fine-tuned to create a new taxonomy depending on application needs. As a starting point, the new Llama Guard 2 uses the recently announced MLCommons taxonomy, in an effort to support the emergence of industry standards in this important area. Additionally, CyberSecEval 2 expands on its predecessor by adding measures of an LLM’s propensity to allow for abuse of its code interpreter, offensive cybersecurity capabilities, and susceptibility to prompt injection attacks (learn more in our technical paper ). Finally, we’re introducing Code Shield which adds support for inference-time filtering of insecure code produced by LLMs. This offers mitigation of risks around insecure code suggestions, code interpreter abuse prevention, and secure command execution.

With the speed at which the generative AI space is moving, we believe an open approach is an important way to bring the ecosystem together and mitigate these potential harms. As part of that, we’re updating our Responsible Use Guide (RUG) that provides a comprehensive guide to responsible development with LLMs. As we outlined in the RUG, we recommend that all inputs and outputs be checked and filtered in accordance with content guidelines appropriate to the application. Additionally, many cloud service providers offer content moderation APIs and other tools for responsible deployment, and we encourage developers to also consider using these options.

Deploying Llama 3 at scale

Llama 3 will soon be available on all major platforms including cloud providers, model API providers, and much more. Llama 3 will be everywhere .

Our benchmarks show the tokenizer offers improved token efficiency, yielding up to 15% fewer tokens compared to Llama 2. Also, Group Query Attention (GQA) now has been added to Llama 3 8B as well. As a result, we observed that despite the model having 1B more parameters compared to Llama 2 7B, the improved tokenizer efficiency and GQA contribute to maintaining the inference efficiency on par with Llama 2 7B.

For examples of how to leverage all of these capabilities, check out Llama Recipes which contains all of our open source code that can be leveraged for everything from fine-tuning to deployment to model evaluation.

What’s next for Llama 3?

The Llama 3 8B and 70B models mark the beginning of what we plan to release for Llama 3. And there’s a lot more to come.

Our largest models are over 400B parameters and, while these models are still training, our team is excited about how they’re trending. Over the coming months, we’ll release multiple models with new capabilities including multimodality, the ability to converse in multiple languages, a much longer context window, and stronger overall capabilities. We will also publish a detailed research paper once we are done training Llama 3.

To give you a sneak preview for where these models are today as they continue training, we thought we could share some snapshots of how our largest LLM model is trending. Please note that this data is based on an early checkpoint of Llama 3 that is still training and these capabilities are not supported as part of the models released today.

We’re committed to the continued growth and development of an open AI ecosystem for releasing our models responsibly. We have long believed that openness leads to better, safer products, faster innovation, and a healthier overall market. This is good for Meta, and it is good for society. We’re taking a community-first approach with Llama 3, and starting today, these models are available on the leading cloud, hosting, and hardware platforms with many more to come.

Try Meta Llama 3 today

We’ve integrated our latest models into Meta AI, which we believe is the world’s leading AI assistant. It’s now built with Llama 3 technology and it’s available in more countries across our apps.

You can use Meta AI on Facebook, Instagram, WhatsApp, Messenger, and the web to get things done, learn, create, and connect with the things that matter to you. You can read more about the Meta AI experience here .

Visit the Llama 3 website to download the models and reference the Getting Started Guide for the latest list of all available platforms.

You’ll also soon be able to test multimodal Meta AI on our Ray-Ban Meta smart glasses.

As always, we look forward to seeing all the amazing products and experiences you will build with Meta Llama 3.

Our latest updates delivered to your inbox

Subscribe to our newsletter to keep up with Meta AI news, events, research breakthroughs, and more.

Join us in the pursuit of what’s possible with AI.

Product experiences

Foundational models

Latest news

Meta © 2024

Developing A Research Question and Hypothesis

Nov 08, 2019

140 likes | 209 Views

Developing A Research Question and Hypothesis. Research question. How to create a good research question. A research question guides and centers your research It should be clear and focused It should ideally be something that you are interested in or care about.

Share Presentation

• research question
• dependent variable
• independent variable
• good research question
• research question researchable

Presentation Transcript

Research question How to create a good research question

A research question guides and centers your research • It should be clear and focused • It should ideally be something that you are interested in or care about

How do you formulate a good research question? • Choose a general topic of interest • Conduct preliminary research on this topic in current periodicals and journals to see what research has already been done • This will help determine what kinds of questions the topic generates • Once you have conducted preliminary research, consider: • Who is the audience? • Is its an academic essay, or will it be read by a more general public? • Start asking open-ended “How?” “What?” and “Why?” questions • Then evaluate the possible responses to those questions

Checklist of Potential Research Questions • Is the research question something I/others care about? Is it arguable? • Is the research question a new spin on an old idea, or does it solve a problem? • Is it too broad or too narrow? • Is the research question researchable within the given time frame and location? • What information is needed?

Independent and dependent variables • A variable is an object, event, idea, feeling, time period, or any other type of category you are trying to measure • There are two types of variables, independent and dependent • An independent variable stands alone and is not changed by other variables you are trying to measure • When looking for the relationship between variables, you are trying to see if the independent variable causes some kind of change in other variables, or dependent variables • A dependent variable is a variable that depends on other factors • Example: a test score could depend on how much you studied, how much sleep you got the night before the test, or even how hungry you were when you took the test • When looking for the relationship between variables, you are trying to find out what makes the dependent variable change the way it does • The independent variable causes a change in the dependent variable and it is not possible that the dependent variable could cause a change in the independent variable

hypothesis How to create a good scientific answer to the question

A strong hypothesis should meet three fundamental criteria • Needs to state the hypothesis in proper phrasing (i.e. grammar, punctuation • Needs to clearly establish the relationship between the variables • Needs to establish that said relationship is scientifically provable • Always make sure the hypothesis being tested matches your project and the preliminary research conducted • Remember, a hypothesis is not a question, but a statement

Examples of a hypothesis • Water levels affect the amount of lice suffered by rainbow trout • Good general statement, but it does not guide how to design the experiment or research • Rainbow trout suffer more lice when water levels are low • Provides some directionality, but statement is still not testable • Rainbow trout suffer more lice in low water conditions because there is less oxygen in the water • Statement is testable; It established variables to see if there is a correlation between water levels and the number of lice on fish

• More by User

Developing Your Research Question

Developing Your Research Question. Your research question provides a starting point for investigating a conversation. What is a research question and how does it shape my research writing project? What is the difference between a research question and a thesis statement?

759 views • 9 slides

Developing a Research Question

Developing a Research Question. “Developing a Research Question” by Annie Knight and Luis Pedroza, Santa Ana College, is licensed under CC BY SA 4.0. Introduction. This tutorial will help you develop a research question in order to make your research more meaningful and focused.

228 views • 12 slides

 Choosing and Developing a Research Question

 Choosing and Developing a Research Question. What is research?. OED Definition Wikipedia Definition MLA Format Guidelines Office of Research Integrity The Research Whisperer. Google Image Search. So are we any closer to understanding what research is?. What is research in English?.

243 views • 10 slides

Developing a Research Question or Thesis Statement

Developing a Research Question or Thesis Statement. Objective: Students will be able to brainstorm potential topics to create a research question or thesis statement. . A good research question or thesis statement is essential when beginning research on a topic.

841 views • 19 slides

Developing Your Research Question. I know what general area, but I’m not sure of my research question?. The Importance of Good Questions. A good research question: Defines the investigation Sets boundaries Provides direction. Defining Your Topic.

476 views • 16 slides

Developing Your Research Question. I know what general area, but I’m not sure of my research question?. The Importance of Good Questions. A good research question: Defines the investigation Sets boundaries Provides direction Helps produce good research. Defining Your Topic.

395 views • 14 slides

Developing a research question and research topic

Developing a research question and research topic. Hans P. Binswanger- Mkhize Visiting Professor China Agricultural University May 3, 2013. Start with a policy-relevant question. Directly relevant: it answers a specific question how much to invest in agricultural research?

286 views • 12 slides

Developing Your Research Question. Do you know what general area you’re interested in, but not sure of your research question?. The Importance of Good Questions. A good research question: Defines the investigation Sets boundaries Provides direction. Defining Your Topic.

563 views • 24 slides

Variables and Developing a Hypothesis

Variables and Developing a Hypothesis. Variables. What is a variable?  When you vary something, you change it.  Variables are things that can change. Variables. There are three main types of variables: I ndependent D ependent C ontrolled 

351 views • 10 slides

Research Question &amp; Hypothesis

Effects of Physical Attractiveness on Evaluations of a Male Employee’s Allegation of Sexual Harassment by His Female Employer By Karl L. Wuensch &amp; Charles H. Moore. Research Question &amp; Hypothesis. Research Question What effects of the sex of the juror and the

196 views • 8 slides

Steps to Creating A Research Question (Hypothesis):

Steps to Creating A Research Question (Hypothesis):. Brainstorm possible risk factors &amp; protective factors Choose one (or a few) ‘factors’ to analyze Predict the type of relationship between factor s &amp; outcome Write your research question Quantify your prediction in step 3 by…

524 views • 17 slides

Developing a good research question

Developing a good research question. Adapted from O'Farrell, Finbar . Approach your assessment the IB Way: Extended Essay. pg. 13. What does a Good research question Look like?. Should be an INTERESTING and ENGAGING question

228 views • 7 slides

RESEARCH PROPOSAL: THEORY, RESEARCH QUESTION &amp; HYPOTHESIS

RESEARCH PROPOSAL: THEORY, RESEARCH QUESTION &amp; HYPOTHESIS. MNGT 583 – Özge Can. Research Proposal Format:. Brief introduction Research question Theoretical background Literature review: A critical review of the theories, concepts, debates and major viewpoints around your research question

840 views • 20 slides

Research Problem Research Question Research Hypothesis

Research Problem Research Question Research Hypothesis. โดย ผู้ช่วยศาสตราจารย์ดร.มาเรียม นิลพันธุ์ คณะศึกษาศาสตร์ มหาวิทยาลัยศิลปากร. มาเรียม นิลพันธุ์. Research Problem. เป็นข้อสงสัย ที่ต้องการคำตอบ สิ่งที่อยากรู้ คำถามที่ต้องการคำตอบ

547 views • 20 slides

Research problem , Purpose, question and hypothesis

Research problem , Purpose, question and hypothesis. P repared by Dr. Manal Moussa. ILO’S O Define of the research problem. O Identify sources of research problem. O Formulate a research problem. O Evaluate a research problem. O Identify criteria of good problem statement.

702 views • 45 slides

Developing a Research Question. Mrs. Bovino Library. Research Question. The answer should not be in your notes or in any one source Reflect upon what you learned this year Requires critical thinking to answer You must evaluate the facts and draw your own conclusions!

395 views • 11 slides

Developing a Research Question. Checklist for a good research question: Defines the investigation. Sets boundaries. Provides direction. Interests the researcher enough to spark his/her own thoughts and opinions. Requires higher order thinking to answer.

272 views • 11 slides

Research Curriculum Session I – Developing A Research Question

Research Curriculum Session I – Developing A Research Question. Jim Quinn MD MS Research Director , Division of Emergency Medicine Stanford University. Overview. Review the purpose of the curriculum Review 5 page protocol concept Discuss the components of a good research question

165 views • 16 slides

Developing a research question

Developing a research question. Where to begin?.

150 views • 14 slides

2. Developing the Research Hypothesis

2. Developing the Research Hypothesis. How do behavioral scientists get ideas for their research projects?. What is a literature search? Why is a literature search necessary, and how it is conducted?. What computer databases contain listings of behavioral research reports?.

113 views • 11 slides