• Digital Manufacturing
• Smart Factory
Clearly defined research question(s) are the key elements which set the focus for study identification and data extraction [21] . These questions are formulated based on the PICOC criteria as presented in the example in Table 2 (PICOC keywords are underlined).
Research questions examples.
Research Questions examples |
---|
• : What are the current challenges of context-aware systems that support the decision-making of business processes in smart manufacturing? • : Which technique is most appropriate to support decision-making for business process management in smart factories? • : In which scenarios are semantic web and machine learning used to provide context-awareness in business process management for smart manufacturing? |
The validity of a study will depend on the proper selection of a database since it must adequately cover the area under investigation [19] . The Web of Science (WoS) is an international and multidisciplinary tool for accessing literature in science, technology, biomedicine, and other disciplines. Scopus is a database that today indexes 40,562 peer-reviewed journals, compared to 24,831 for WoS. Thus, Scopus is currently the largest existing multidisciplinary database. However, it may also be necessary to include sources relevant to computer science, such as EI Compendex, IEEE Xplore, and ACM. Table 3 compares the area of expertise of a selection of databases.
Planning Step 3 “Select digital libraries”. Description of digital libraries in computer science and software engineering.
Database | Description | URL | Area | Advanced Search Y/N |
---|---|---|---|---|
Scopus | From Elsevier. sOne of the largest databases. Very user-friendly interface | Interdisciplinary | Y | |
Web of Science | From Clarivate. Multidisciplinary database with wide ranging content. | Interdisciplinary | Y | |
EI Compendex | From Elsevier. Focused on engineering literature. | Engineering | Y (Query view not available) | |
IEEE Digital Library | Contains scientific and technical articles published by IEEE and its publishing partners. | Engineering and Technology | Y | |
ACM Digital Library | Complete collection of ACM publications. | Computing and information technology | Y |
Authors should define the inclusion and exclusion criteria before conducting the review to prevent bias, although these can be adjusted later, if necessary. The selection of primary studies will depend on these criteria. Articles are included or excluded in this first selection based on abstract and primary bibliographic data. When unsure, the article is skimmed to further decide the relevance for the review. Table 4 sets out some criteria types with descriptions and examples.
Planning Step 4 “Define inclusion and exclusion criteria”. Examples of criteria type.
Criteria Type | Description | Example |
---|---|---|
Period | Articles can be selected based on the time period to review, e.g., reviewing the technology under study from the year it emerged, or reviewing progress in the field since the publication of a prior literature review. | : From 2015 to 2021 Articles prior 2015 |
Language | Articles can be excluded based on language. | : Articles not in English |
Type of Literature | Articles can be excluded if they are fall into the category of grey literature. | Reports, policy literature, working papers, newsletters, government documents, speeches |
Type of source | Articles can be included or excluded by the type of origin, i.e., conference or journal articles or books. | : Articles from Conferences or Journals Articles from books |
Impact Source | Articles can be excluded if the author limits the impact factor or quartile of the source. | Articles from Q1, and Q2 sources : Articles with a Journal Impact Score (JIS) lower than |
Accessibility | Not accessible in specific databases. | : Not accessible |
Relevance to research questions | Articles can be excluded if they are not relevant to a particular question or to “ ” number of research questions. | Not relevant to at least 2 research questions |
Assessing the quality of an article requires an artifact which describes how to perform a detailed assessment. A typical quality assessment is a checklist that contains multiple factors to evaluate. A numerical scale is used to assess the criteria and quantify the QA [22] . Zhou et al. [25] presented a detailed description of assessment criteria in software engineering, classified into four main aspects of study quality: Reporting, Rigor, Credibility, and Relevance. Each of these criteria can be evaluated using, for instance, a Likert-type scale [17] , as shown in Table 5 . It is essential to select the same scale for all criteria established on the quality assessment.
Planning Step 5 “Define QA assessment checklist”. Examples of QA scales and questions.
Do the researchers discuss any problems (limitations, threats) with the validity of their results (reliability)? | 1 – No, and not considered (Score: 0) 2 – Partially (Score: 0.5) 3 – Yes (Score: 1) |
Is there a clear definition/ description/ statement of the aims/ goals/ purposes/ motivations/ objectives/ questions of the research? | 1 – Disagree (Score: 1) 2 – Somewhat disagree (Score: 2) 3 – Neither agree nor disagree (Score: 3) 4 – Somewhat agree (Score: 4) 5 – Agree (Score: 5) |
The data extraction form represents the information necessary to answer the research questions established for the review. Synthesizing the articles is a crucial step when conducting research. Ramesh et al. [15] presented a classification scheme for computer science research, based on topics, research methods, and levels of analysis that can be used to categorize the articles selected. Classification methods and fields to consider when conducting a review are presented in Table 6 .
Planning Step 6 “Define data extraction form”. Examples of fields.
Classification and fields to consider for data extraction | Description and examples |
---|---|
Research type | • focuses on abstract ideas, concepts, and theories built on literature reviews . • uses scientific data or case studies for explorative, descriptive, explanatory, or measurable findings . an SLR on context-awareness for S-PSS and categorized the articles in theoretical and empirical research. |
By process phases, stages | When analyzing a process or series of processes, an effective way to structure the data is to find a well-established framework of reference or architecture. : • an SLR on self-adaptive systems uses the MAPE-K model to understand how the authors tackle each module stage. • presented a context-awareness survey using the stages of context-aware lifecycle to review different methods. |
By technology, framework, or platform | When analyzing a computer science topic, it is important to know the technology currently employed to understand trends, benefits, or limitations. : • an SLR on the big data ecosystem in the manufacturing field that includes frameworks, tools, and platforms for each stage of the big data ecosystem. |
By application field and/or industry domain | If the review is not limited to a specific “Context” or “Population" (industry domain), it can be useful to identify the field of application : • an SLR on adaptive training using virtual reality (VR). The review presents an extensive description of multiple application domains and examines related work. |
Gaps and challenges | Identifying gaps and challenges is important in reviews to determine the research needs and further establish research directions that can help scholars act on the topic. |
Findings in research | Research in computer science can deliver multiple types of findings, e.g.: |
Evaluation method | Case studies, experiments, surveys, mathematical demonstrations, and performance indicators. |
The data extraction must be relevant to the research questions, and the relationship to each of the questions should be included in the form. Kitchenham & Charters [6] presented more pertinent data that can be captured, such as conclusions, recommendations, strengths, and weaknesses. Although the data extraction form can be updated if more information is needed, this should be treated with caution since it can be time-consuming. It can therefore be helpful to first have a general background in the research topic to determine better data extraction criteria.
After defining the protocol, conducting the review requires following each of the steps previously described. Using tools can help simplify the performance of this task. Standard tools such as Excel or Google sheets allow multiple researchers to work collaboratively. Another online tool specifically designed for performing SLRs is Parsif.al 1 . This tool allows researchers, especially in the context of software engineering, to define goals and objectives, import articles using BibTeX files, eliminate duplicates, define selection criteria, and generate reports.
Search strings are built considering the PICOC elements and synonyms to execute the search in each database library. A search string should separate the synonyms with the boolean operator OR. In comparison, the PICOC elements are separated with parentheses and the boolean operator AND. An example is presented next:
(“Smart Manufacturing” OR “Digital Manufacturing” OR “Smart Factory”) AND (“Business Process Management” OR “BPEL” OR “BPM” OR “BPMN”) AND (“Semantic Web” OR “Ontology” OR “Semantic” OR “Semantic Web Service”) AND (“Framework” OR “Extension” OR “Plugin” OR “Tool”
Databases that feature advanced searches enable researchers to perform search queries based on titles, abstracts, and keywords, as well as for years or areas of research. Fig. 1 presents the example of an advanced search in Scopus, using titles, abstracts, and keywords (TITLE-ABS-KEY). Most of the databases allow the use of logical operators (i.e., AND, OR). In the example, the search is for “BIG DATA” and “USER EXPERIENCE” or “UX” as a synonym.
Example of Advanced search on Scopus.
In general, bibliometric data of articles can be exported from the databases as a comma-separated-value file (CSV) or BibTeX file, which is helpful for data extraction and quantitative and qualitative analysis. In addition, researchers should take advantage of reference-management software such as Zotero, Mendeley, Endnote, or Jabref, which import bibliographic information onto the software easily.
The first step in this stage is to identify any duplicates that appear in the different searches in the selected databases. Some automatic procedures, tools like Excel formulas, or programming languages (i.e., Python) can be convenient here.
In the second step, articles are included or excluded according to the selection criteria, mainly by reading titles and abstracts. Finally, the quality is assessed using the predefined scale. Fig. 2 shows an example of an article QA evaluation in Parsif.al, using a simple scale. In this scenario, the scoring procedure is the following YES= 1, PARTIALLY= 0.5, and NO or UNKNOWN = 0 . A cut-off score should be defined to filter those articles that do not pass the QA. The QA will require a light review of the full text of the article.
Performing quality assessment (QA) in Parsif.al.
Those articles that pass the study selection are then thoroughly and critically read. Next, the researcher completes the information required using the “data extraction” form, as illustrated in Fig. 3 , in this scenario using Parsif.al tool.
Example of data extraction form using Parsif.al.
The information required (study characteristics and findings) from each included study must be acquired and documented through careful reading. Data extraction is valuable, especially if the data requires manipulation or assumptions and inferences. Thus, information can be synthesized from the extracted data for qualitative or quantitative analysis [16] . This documentation supports clarity, precise reporting, and the ability to scrutinize and replicate the examination.
The analysis phase examines the synthesized data and extracts meaningful information from the selected articles [10] . There are two main goals in this phase.
The first goal is to analyze the literature in terms of leading authors, journals, countries, and organizations. Furthermore, it helps identify correlations among topic s . Even when not mandatory, this activity can be constructive for researchers to position their work, find trends, and find collaboration opportunities. Next, data from the selected articles can be analyzed using bibliometric analysis (BA). BA summarizes large amounts of bibliometric data to present the state of intellectual structure and emerging trends in a topic or field of research [4] . Table 7 sets out some of the most common bibliometric analysis representations.
Techniques for bibliometric analysis and examples.
Publication-related analysis | Description | Example |
---|---|---|
Years of publications | Determine interest in the research topic by years or the period established by the SLR, by quantifying the number of papers published. Using this information, it is also possible to forecast the growth rate of research interest. | [ ] identified the growth rate of research interest and the yearly publication trend. |
Top contribution journals/conferences | Identify the leading journals and conferences in which authors can share their current and future work. | , |
Top countries' or affiliation contributions | Examine the impacts of countries or affiliations leading the research topic. | [ , ] identified the most influential countries. |
Leading authors | Identify the most significant authors in a research field. | - |
Keyword correlation analysis | Explore existing relationships between topics in a research field based on the written content of the publication or related keywords established in the articles. | using keyword clustering analysis ( ). using frequency analysis. |
Total and average citation | Identify the most relevant publications in a research field. | Scatter plot citation scores and journal factor impact |
Several tools can perform this type of analysis, such as Excel and Google Sheets for statistical graphs or using programming languages such as Python that has available multiple data visualization libraries (i.e. Matplotlib, Seaborn). Cluster maps based on bibliographic data(i.e keywords, authors) can be developed in VosViewer which makes it easy to identify clusters of related items [18] . In Fig. 4 , node size is representative of the number of papers related to the keyword, and lines represent the links among keyword terms.
[1] Keyword co-relationship analysis using clusterization in vos viewer.
This second and most important goal is to answer the formulated research questions, which should include a quantitative and qualitative analysis. The quantitative analysis can make use of data categorized, labelled, or coded in the extraction form (see Section 1.6). This data can be transformed into numerical values to perform statistical analysis. One of the most widely employed method is frequency analysis, which shows the recurrence of an event, and can also represent the percental distribution of the population (i.e., percentage by technology type, frequency of use of different frameworks, etc.). Q ualitative analysis includes the narration of the results, the discussion indicating the way forward in future research work, and inferring a conclusion.
Finally, the literature review report should state the protocol to ensure others researchers can replicate the process and understand how the analysis was performed. In the protocol, it is essential to present the inclusion and exclusion criteria, quality assessment, and rationality beyond these aspects.
The presentation and reporting of results will depend on the structure of the review given by the researchers conducting the SLR, there is no one answer. This structure should tie the studies together into key themes, characteristics, or subgroups [ 28 ].
SLR can be an extensive and demanding task, however the results are beneficial in providing a comprehensive overview of the available evidence on a given topic. For this reason, researchers should keep in mind that the entire process of the SLR is tailored to answer the research question(s). This article has detailed a practical guide with the essential steps to conducting an SLR in the context of computer science and software engineering while citing multiple helpful examples and tools. It is envisaged that this method will assist researchers, and particularly early-stage researchers, in following an algorithmic approach to fulfill this task. Finally, a quick checklist is presented in Appendix A as a companion of this article.
Angela Carrera-Rivera: Conceptualization, Methodology, Writing-Original. William Ochoa-Agurto : Methodology, Writing-Original. Felix Larrinaga : Reviewing and Supervision Ganix Lasa: Reviewing and Supervision.
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Funding : This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Grant No. 814078.
Carrera-Rivera, A., Larrinaga, F., & Lasa, G. (2022). Context-awareness for the design of Smart-product service systems: Literature review. Computers in Industry, 142, 103730.
1 https://parsif.al/
A literature review involves researching, reading, analyzing, evaluating, and summarizing scholarly literature (typically journals and articles) about a specific topic. The results of a literature review may be an entire report or article OR may be part of a article, thesis, dissertation, or grant proposal. A literature review helps the author learn about the history and nature of their topic, and identify research gaps and problems.
Problem formulation
Elements of a Literature Review
1100 Lawrence Street Denver, CO 80204 303-315-7700 Ask Us Directions
Steps in the literature review process.
Note: The first four steps are the best points at which to contact a librarian. Your librarian can help you determine the best databases to use for your topic, assess scope, and formulate a search strategy.
This 4.5 minute video from Academic Education Materials has a Creative Commons License and a British narrator.
This paper is part of a series of methodological guidance from the Cochrane Rapid Reviews Methods Group. Rapid reviews (RR) use modified systematic review methods to accelerate the review process while maintaining systematic, transparent and reproducible methods. In this paper, we address considerations for RR searches. We cover the main areas relevant to the search process: preparation and planning, information sources and search methods, search strategy development, quality assurance, reporting, and record management. Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches and (2) reducing the size of the search result. Because screening search results is usually more resource-intensive than conducting the search, we suggest investing time upfront in planning and optimising the search to save time by reducing the literature screening workload. To achieve this goal, RR teams should work with an information specialist. They should select a small number of relevant information sources (eg, databases) and use search methods that are highly likely to identify relevant literature for their topic. Database search strategies should aim to optimise both precision and sensitivity, and quality assurance measures (peer review and validation of search strategies) should be applied to minimise errors.
No data are available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ .
https://doi.org/10.1136/bmjebm-2022-112079
Request permissions.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Compared with systematic reviews, rapid reviews (RR) often abbreviate or limit the literature search in some way to accelerate review production. However, RR guidance rarely specifies how to select topic-appropriate search approaches.
This paper presents an overview of considerations and recommendations for RR searching, covering the complete search process from the planning stage to record management. We also provide extensive appendices with practical examples, useful sources and a glossary of terms.
There is no one-size-fits-all solution for RR literature searching: review teams should consider what search approaches best fit their RR project.
This paper is part of a series from the Cochrane Rapid Reviews Methods Group (RRMG) providing methodological guidance for rapid reviews (RRs). 1–3 While the RRMG’s guidance 4 5 on Cochrane RR production includes brief advice on literature searching, we aim to provide in-depth recommendations for the entire search process.
Literature searching is the foundation for all reviews; therefore, it is important to understand the goals of a specific RR. The scope of RRs varies considerably (from focused questions to overviews of broad topics). 6 As with conventional systematic reviews (SRs), there is not a one-size-fits-all approach for RR literature searches. We aim to support RR teams in choosing methods that best fit their project while understanding the limitations of modified search methods. Our recommendations derive from current systematic search guidance, evidence on modified search methods and practical experience conducting RRs.
This paper presents considerations and recommendations, described briefly in table 1 . The table also includes a comparison to the SR search process based on common recommendations. 7–10 We provide supplemental materials, including a list of additional resources, further details of our recommendations, practical examples, and a glossary (explaining the terms written in italics) in online supplemental appendices A–C .
Recommendations for rapid review literature searching
Given that the results of systematic literature searches underpin a review, planning the searches is integral to the overall RR preparation. The RR search process follows the same steps as an SR search; therefore, RR teams must be familiar with the general standards of systematic searching . Templates (see online supplemental appendix B ) and reporting guidance 11 for SR searches can also be adapted to structure the RR search process.
Developing a plan for the literature search forms part of protocol development and should involve an information specialist (eg, librarian). Information specialists can assist in refining the research question, selecting appropriate search methods and resources, designing and executing search strategies, and reporting the search methods. At minimum, specialist input should include assessing information sources and methods and providing feedback on the primary database search strategy.
Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches (eg, using automation tools, reusing existing search strategies, omitting planning or quality assurance steps) and (2) reducing the size of the search result (eg, limiting the number of information sources, increasing the precision of search strategies, using study design filters). Study selection (ie, screening search results) is usually more resource-intensive than searching, 12 particularly for topics with complex or broad concepts or diffuse terminology; thus, the second option may be more efficient for the entire RR. Investing time upfront in optimising search sensitivity (ie, completeness) and precision (ie, positive predictive value) can save time in the long run by reducing the screening and selection workload.
Preliminary or scoping searches are critical to this process. They inform the choice of search methods and identify potentially relevant literature. Texts identified through preliminary searching serve as known relevant records that can be used throughout the search development process (see sections on database selection, development and validation of search strategies).
In addition to planning the search itself, the review team should factor in time for quality assurance steps (eg, search strategy peer review) and the management of search results (eg, deduplication, full-text retrieval).
To optimise the balance of search sensitivity and precision, RR teams should prioritise the most relevant information sources for the topic and the type of evidence required. These can include bibliographic databases (eg, MEDLINE/PubMed), grey literature sources and targeted supplementary search methods. Note that this approach differs from the Methodological Expectations of Cochrane Intervention Reviews Standards 9 where the same core set of information sources is required for every review and further supplemented by additional topic-specific and evidence-specific sources.
For many review topics, most evidence is found in peer-reviewed journal articles, making bibliographic databases the main resource of systematic searching. Limiting the number of databases searched can be a viable option in RRs, but it is important to prioritise topic-appropriate databases.
MEDLINE has been found to have high coverage for studies included in SRs 13 14 and is an appealing database choice because access is free via PubMed. However, coverage varies depending on topics and relevant study designs. 15 16 Additionally, even if all eligible studies for a topic were available in MEDLINE, search strategies will usually miss some eligible studies because search sensitivity is lower than database coverage. 13 17 This means searching MEDLINE alone is not a viable option, and additional information sources or search methods are required. Known relevant records can be used to help assess the coverage of selected databases (see also online supplemental appendix C ).
Supplementary systematic search methods have three main goals, to identify (1) grey literature, (2) published literature not covered by the selected bibliographic databases and (3) database-covered literature that was not retrieved by the database searches.
When RRs search only a small number of databases, supplementary searches can be particularly important to pick up eligible studies not identified via database searching. While supplementary methods might increase the time spent on searching, they sometimes better optimise search sensitivity and precision, saving time in the long run. 18 Depending on the topic and relevant evidence, such methods can offer an alternative to adding additional specialised database searches. To decide if and what supplementary searches are helpful, it is important to evaluate what literature might be missed by the database searches and how this might affect the specific RR.
Some studies indicate that the omission of grey literature searches rarely affects review conclusions. 17 19 However, the relevance of study registries and other grey literature sources is topic-dependent. 16 19–21 For example, randomised controlled trials (RCTs) on newly approved drugs are typically identified in ClinicalTrials.gov. 20 For rapidly evolving topics such as COVID-19, preprints are an important source. 21 For public health interventions, various types of grey literature may be important (eg, evaluations conducted by local public health agencies). 22
Other supplementary techniques (eg, checking reference lists, reviewing specific websites or electronic table of contents, contacting experts) may identify additional studies not retrieved by database searches. 23 One of the most common approaches involves checking reference lists of included studies and relevant reviews. This method may identify studies missed by limited database searches. 12 Another promising citation-based approach is using the ‘similar articles’ option in PubMed, although research has focused on updating existing SRs. 24 25
Databases and search methods to identify RCTs have been particularly well researched. 17 20 24 26 27 For this reason, it is possible to give more precise recommendations for RRs based on RCTs than for other types of review. Table 2 provides an overview of the most important considerations; additional information can be found in online supplemental appendix C .
Information sources for identification of randomised controlled trials (RCTs)
We define ‘search strategy’ as a Boolean search query in a specific database (eg, MEDLINE) using a specific interface (eg, Ovid). When several databases are searched, this query is usually developed in a primary database and interface (eg, Ovid MEDLINE) and translated to other databases.
Optimising search strategy precision while aiming for high sensitivity is critical in reducing the number of records retrieved. Preliminary searches provide crucial information to aid efficient search strategy development. Reviewing the abstracts and subject headings used in known relevant records will assist in identifying appropriate search terms. Text analysis tools can also be used to support this process, 28 29 for example, to develop ‘objectively derived’ search strategies. 30
Reusing or adapting complete search strategies (eg, from SRs identified by the preliminary searches) or selecting elements of search strategies for reuse can accelerate search strategy development. Additionally, validated search filters (eg, for study design) can be used to reduce the size of the search result without compromising the sensitivity of a search strategy. 31 However, quality assurance measures are necessary whether the search strategy is purpose-built, reused or adapted (see the ‘Quality assurance’ section.)
Database-specific and interface-specific functionalities can also be used to improve searches’ precision and reduce the search result’s size. Some options are: restricting to records where subject terms have been assigned as the major focus of an article (eg, major descriptors in MeSH), using proximity operators (ie, terms adjacent or within a set number of words), frequency operators (ie, terms have to appear a minimum number of times in an abstract) or restricting search terms to the article title. 32–34
Automated syntax translation can save time and reduce errors when translating a primary search strategy to different databases. 35 36 However, manual adjustments will usually be necessary.
The time taken to learn how to use supporting technologies (eg, text analysis, syntax translation) proficiently should not be underestimated. The time investment is most likely to pay off for frequent searchers. A later paper in this series will address supporting software for the entire review process.
Limits and restrictions (eg, publication dates, language) are another way to reduce the number of records retrieved but should be tailored to the topic and applied with caution. For example, if most studies about an intervention were published 10 years ago, then an arbitrary cut-off of ‘the last 5 years’ will miss many relevant studies. 37 Similarly, limiting to ‘English only’ is acceptable for most cases, but early in the COVID-19 pandemic, a quarter of available research articles were written in Chinese. 38 Depending on the RR topic, certain document types (eg, conference abstracts, dissertations) might be excluded if not considered relevant to the research question.
Note also that preset limiting functions in search interfaces (eg, limit to humans) often rely on subject headings (eg, MeSH) alone. They will miss eligible studies that lack or have incomplete subject indexing. Using (validated) search filters 31 is preferable.
One approach to RR production involves updating an existing SR. In this case, preliminary searches should be used to check if new evidence is available. If the review team decide to update the review, they should assess the original search methods and adapt these as necessary.
One option is to identify the minimum set of databases required to retrieve all the original included studies. 39 Any reused search strategies should be validated and peer-reviewed (see below) and optimised for precision and/or sensitivity.
Additionally, it is important to assess whether the topic terminology or the relevant databases have changed since the original SR search.
In some cases, designing a new search process may be more efficient than reproducing the original search.
Errors in search strategies are common and can impact the sensitivity and comprehensiveness of the search result. 40 If an RR search uses a small number of information sources, such errors could affect the identification of relevant studies.
The primary database search strategy should be validated using known relevant records (if available). This means testing if the primary search strategy retrieves eligible studies found through preliminary searching. If some known studies are not identified, the searcher assesses the reasons and decides if revisions are necessary. Even a precision-focused systematic search should identify the majority—we suggest at least 80%–90%—of known studies. This is based on benchmarks for sensitivity-precision-maximising search filters 41 and assumes that the set of known studies is representative of the whole of relevant studies.
Ideally, an information specialist should review the planned information sources and search methods and use the PRESS (Peer Review of Electronic Search Strategies) checklist 42 to assess the primary search strategy. Turnaround time has to be factored into the process from the outset (eg, waiting for feedback, revising the search strategy). PRESS recommends a maximum turnaround time of five working days for feedback, but in-house peer review often takes only a few hours.
If the overall RR time plan does not allow for a full peer review of the search strategy, a review team member with search experience should check the search strategy for spelling errors and correct use of Boolean operators (AND, OR, NOT) at a minimum.
Record management requirements of RRs are largely identical to SRs and have to be factored into the time plan. Teams should develop a data management plan and review the relevant reporting standards at the project’s outset. PRISMA-S (Preferred Reporting Items for Systematic Reviews and Meta-Analyses literature search extension) 11 is a reporting standard for SR searches that can be adapted for RRs.
Reference management software (eg, EndNote, 43 Zotero 44 ) should be used to track search results, including deduplication. Note that record management for database searches is less time-consuming than for many supplementary or grey literature searches, which often require manual entry into reference management software. 12
Additionally, software platforms for SR production (eg, Covidence, 45 EPPI-Reviewer, 46 Systematic Review Data Repository Plus 47 ) can provide a unified way to keep track of records throughout the whole review process, which can improve management and save time. These platforms and other dedicated tools (eg, SRA Deduplicator) 48 also offer automated deduplication. However, the time and cost investment necessary to appropriately use these tools have to be considered.
Decisions about search methods for an RR need to consider where time can be most usefully invested and processes accelerated. The literature search should be considered in the context of the entire review process, for example, protocol development and literature screening: Findings of preliminary searches often affect the development and refinement of the research question and the review’s eligibility criteria . In turn, they affect the number of records retrieved by the searches and therefore the time needed for literature selection.
For this reason, focusing only on reducing time spent on designing and conducting searches can be a false economy when seeking to speed up review production. While some approaches (eg, text analysis, automated syntax translation) may save time without negatively affecting search validity, others (eg, skipping quality assurance steps, using convenient information sources without considering their topic appropriateness) may harm the entire review. Information specialists can provide crucial aid concerning the appropriate design of search strategies, choice of methods and information sources.
For this reason, we consider that investing time at the outset of the review to carefully choose a small number of highly appropriate search methods and optimise search sensitivity and precision likely leads to better and more manageable results.
Patient consent for publication.
Not applicable.
Supplementary data.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Twitter @micaelaescb
Collaborators On behalf of the Cochrane Rapid Reviews Methods Group: Declan Devane, Gerald Gartlehner, Isolde Sommer.
Contributors IK, SR, AB, CME-L and SW contributed to the conceptualisation of this paper. IK, AB and CME-L wrote the first draft of the manuscript. All authors critically reviewed and revised the manuscript. IK is responsible for the overall content.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests AB is co-convenor of the Cochrane Qualitative and Implementation Methods Group. In the last 36 months, he received royalties from Systematic Approaches To a Successful Literature Review (Sage 3rd edn), payment or honoraria form the Agency for Healthcare Research and Quality, and travel support from the WHO. DD works part time for Cochrane Ireland and Evidence Synthesis Ireland, which are funded within the University of Ireland Galway (Ireland) by the Health Research Board (HRB) and the Health and Social Care, Research and Development (HSC R&D) Division of the Public Health Agency in Northern Ireland.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.
All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .
Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.
Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.
Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.
Original Submission Date Received: .
Find support for a specific problem in the support section of our website.
Please let us know what you think of our products and services.
Visit our dedicated information section to learn more about MDPI.
Integrating blockchain, iot, and xbrl in accounting information systems: a systematic literature review.
2. methodology, 2.1. definition of the research question, 2.2. search for literature, 2.3. applying inclusion and exclusion criteria, 2.4. quality assessment, 3.1. journal analysis, 3.2. blockchain studies, 3.2.1. overview analysis, 3.2.2. blockchain as an accounting system, blockchain as a triple-entry accounting system, blockchain as a single accounting system, 3.2.3. blockchain and the quality of accounting information, 3.3. internet of things technology studies, 3.3.1. overview analysis, 3.3.2. using the internet of things (iot) in the accounting field, 3.4. extensible business reporting language (xbrl) studies, 3.4.1. overview analysis, 3.4.2. the benefits of extensible business reporting language (xbrl) in accounting, 3.5. the integration of blockchain, the internet of things, and xbrl, 4. discussion, 5. the limitations of the study, 6. future research directions, 7. conclusions, supplementary materials, author contributions, data availability statement, acknowledgments, conflicts of interest.
Click here to enlarge figure
Inclusion Criteria | Exclusion Criteria |
---|---|
Studies within the fields of business, management, and accounting. | Studies out of the fields of business, management, and accounting |
English studies | Studies in languages other than English |
Peer-reviewed articles, book chapters, and books | Conference papers, notes |
Accessible studies | Inaccessible studies |
Studies related to the variables of the study | Studies not related to the variables of the study |
From 2013 to 2023 | Duplicated studies |
Journal | Number of Papers |
---|---|
Journal of Information Systems | 34 |
Journal of Emerging Technologies in Accounting | 25 |
International Journal of Accounting Information Systems | 12 |
International Journal of Digital Accounting Research | 9 |
International Journal of Accounting and Information Management | 9 |
Financial and Credit Activity: Problems of Theory and Practice | 8 |
Accounting, Auditing and Accountability Journal | 8 |
Australian Accounting Review | 6 |
Journal of Accounting and Public Policy | 6 |
Accounting Perspectives | 6 |
Journal of Financial Reporting and Accounting | 6 |
Academy of Accounting and Financial Studies Journal | 5 |
Decision Support Systems | 5 |
Intelligent Systems in Accounting, Finance and Management | 5 |
Accounting and Finance | 4 |
Quality—Access to Success | 4 |
Accounting Research Journal | 3 |
Journal of Business Research | 3 |
Journal of Risk and Financial Management | 3 |
Title | Authors | Citation |
---|---|---|
Toward Blockchain-Based Accounting and Assurance | ( ) | 377 |
The Role of Internet-Related Technologies in Shaping the Work of Accountants: New Directions for Accounting Research | ( ) | 187 |
Accounting and Auditing at the Time of Blockchain Technology: A Research Agenda | ( ) | 165 |
Configuring Blockchain Architectures for Transaction Information in Blockchain Consortiums: The Case of Accounting and Supply Chain Systems | ( ) | 163 |
Blockchain: Emergent Industry Adoption and Implications for Accounting | ( ) | 163 |
Title | Authors | Citation |
---|---|---|
Machine Learning-Based Digital Twin Framework for Production Optimization in the Petrochemical Industry | ( ) | 245 |
The Internet of Things and Corporate Business Models: A Systematic Literature Review | ( ) | 46 |
The Internet of Things and Economic Growth in a Panel of Countries | ( ) | 34 |
Demand Effects of the Internet-of-Things Sales Channel: Evidence from Automating the Purchase Process | ( ) | 11 |
Integrated Billing Mechanisms in the Internet of Things to Support Information Sharing and Enable New Business Opportunities | ( ) | 7 |
Title | Authors | Citation |
---|---|---|
Does Search-Facilitating Technology Improve the Transparency of Financial Reporting? | ( ) | 298 |
The Production and Use of Semantically Rich Accounting Reports on the Internet: XML and XBRL | ( ) | 170 |
Does It Add Up? Early Evidence on the Data Quality of XBRL Filings to the SEC | ( ) | 140 |
Does XBRL Adoption Reduce Information Asymmetry? | ( ) | 133 |
Measuring Accounting Reporting Complexity with XBRL | ( ) | 113 |
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
Nofel, M.; Marzouk, M.; Elbardan, H.; Saleh, R.; Mogahed, A. Integrating Blockchain, IoT, and XBRL in Accounting Information Systems: A Systematic Literature Review. J. Risk Financial Manag. 2024 , 17 , 372. https://doi.org/10.3390/jrfm17080372
Nofel M, Marzouk M, Elbardan H, Saleh R, Mogahed A. Integrating Blockchain, IoT, and XBRL in Accounting Information Systems: A Systematic Literature Review. Journal of Risk and Financial Management . 2024; 17(8):372. https://doi.org/10.3390/jrfm17080372
Nofel, Mohamed, Mahmoud Marzouk, Hany Elbardan, Reda Saleh, and Aly Mogahed. 2024. "Integrating Blockchain, IoT, and XBRL in Accounting Information Systems: A Systematic Literature Review" Journal of Risk and Financial Management 17, no. 8: 372. https://doi.org/10.3390/jrfm17080372
Article access statistics, supplementary material.
ZIP-Document (ZIP, 1249 KiB)
Mdpi initiatives, follow mdpi.
Subscribe to receive issue release notifications and newsletters from MDPI journals
Laboratory experiments have long been used to guide predictions of organismal stress in response to our rapidly changing climate. However, the ability to simulate real world conditions in the laboratory can be a major barrier to prediction accuracy, creating obstacles to efforts informing ecosystem conservation and management. Capitalizing on an extensive experimental literature of coral bleaching physiology, we performed a systematic review of the literature and assembled a database to identify the methods being used to measure coral bleaching in heating experiments and assess how closely heating experiments resembled marine heatwaves (MHWs) on coral reefs. Observations of the maximum photochemical yield of Photosystem II (FV/FM), though not a direct measure of bleaching, vastly outnumbered Symbiodiniaceae density and chlorophyll (ug cm-2, pg cell-1) observations in the available literature, indicating the widespread misuse of FV/FM as a proxy for coral bleaching. Laboratory studies in our database used significantly higher maximum temperatures, degree heating times (~ 1.7 x) and heating rates (~ 7.3 x), and significantly shorter durations (~ 1.5 x) than MHWs on coral reefs. We then asked whether exposure differences between lab and reef altered the relationship between coral bleaching and heating metrics using the example of hormesis, the biphasic dose response wherein low to moderate doses elicit some benefit, while high doses are deleterious. We fit curves on the data both with and without ecologically relevant heating metrics and found hormetic curves in some response variables were altered with the exclusion of exposures that fell outside of the bounds of MHWs on coral reefs. Differences between lab exposures and real-world MHWs were large enough to alter the relationships, indicating a high likelihood of prediction error. We recommend laboratory-based studies of coral bleaching use ecologically relevant exposures to improve our predictions of the coral physiological response to our rapidly warming oceans.
The authors have declared no competing interest.
View the discussion thread.
Thank you for your interest in spreading the word about bioRxiv.
NOTE: Your email address is requested solely to identify you as the sender of this article.
IMAGES
COMMENTS
A Review of the Theoretical Literature" (Theoretical literature review about the development of economic migration theory from the 1950s to today.) Example literature review #2: "Literature review as a research methodology: An overview and guidelines" ( Methodological literature review about interdisciplinary knowledge acquisition and ...
A typology of literature reviews. A methodological review is a type of systematic secondary research (i.e., research synthesis) which focuses on summarising the state-of-the-art methodological practices of research in a substantive field or topic" (Chong et al, 2021).
As mentioned previously, there are a number of existing guidelines for literature reviews. Depending on the methodology needed to achieve the purpose of the review, all types can be helpful and appropriate to reach a specific goal (for examples, please see Table 1).These approaches can be qualitative, quantitative, or have a mixed design depending on the phase of the review.
Therefore, the CLR is a meta-framework. For example, in Step 1: Exploring Beliefs and Topics, we provide many parts of the belief system, such as worldview, field/discipline-specific beliefs, and topic-specific beliefs. We imagine that if a person holds many beliefs on one issue, he/she might have a meta-belief system.
This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis.
A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research. There are five key steps to writing a literature review: Search for relevant literature. Evaluate sources. Identify themes, debates and gaps.
9.3. Types of Review Articles and Brief Illustrations. EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic.
Demonstrate your knowledge of the research topic. Identify the gaps in the literature and show how your research links to these. Provide the foundation for your conceptual framework (if you have one) Inform your own methodology and research design. To achieve this, your literature review needs a well-thought-out structure.
As mentioned above, writing your literature review is a process, which I'll break down into three steps: Finding the most suitable literature. Understanding, distilling and organising the literature. Planning and writing up your literature review chapter. Importantly, you must complete steps one and two before you start writing up your chapter.
A third objective for a methods review is to offer clarity and enhance collective understanding regarding a specific methods topic that may be characterized by ambiguity, inconsistency, or a lack of comprehensiveness within the available methods literature. An example of this is a overview whose objective was to review the inconsistent ...
Included SRs evaluated 24 unique methodological approaches used for defining the review scope and eligibility, literature search, screening, data extraction, and quality appraisal in the SR process. Limited evidence supports the following (a) searching multiple resources (electronic databases, handsearching, and reference lists) to identify ...
Literature reviews establish the foundation of academic inquires. However, in the planning field, we lack rigorous systematic reviews. In this article, through a systematic search on the methodology of literature review, we categorize a typology of literature reviews, discuss steps in conducting a systematic literature review, and provide suggestions on how to enhance rigor in literature ...
If you're working on a dissertation or thesis and are looking for an example of a strong literature review chapter, you've come to the right place.. In this video, we walk you through an A-grade literature review from a dissertation that earned full distinction.We start off by discussing the five core sections of a literature review chapter by unpacking our free literature review template.
Steps for Conducting a Lit Review; Finding "The Literature" Organizing/Writing; APA Style This link opens in a new window; Chicago: Notes Bibliography This link opens in a new window; MLA Style This link opens in a new window; Sample Literature Reviews. Sample Lit Reviews from Communication Arts; Have an exemplary literature review? Get Help!
Literature review plays a significant role as a guideline for everyone's investigative. forms. They may be the foundationforthegrowth of awareness, establish policies. and practicesprotocols ...
A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship ...
Types of Literature Review are as follows: Narrative literature review: This type of review involves a comprehensive summary and critical analysis of the available literature on a particular topic or research question. It is often used as an introductory section of a research paper. Systematic literature review: This is a rigorous and ...
Overview. A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure .An SLR updates the reader with current literature about a subject .The goal is to review critical points of current knowledge on a topic about research ...
A literature review involves researching, reading, analyzing, evaluating, and summarizing scholarly literature (typically journals and articles) about a specific topic. The results of a literature review may be an entire report or article OR may be part of a article, thesis, dissertation, or grant proposal.
Method details Overview. A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure [12].An SLR updates the reader with current literature about a subject [6].The goal is to review critical points of current knowledge on a ...
Sage Research Methods community highlighted this useful collection with a series of interviews and related resources in 2023. Review research is an broad term that describes various types of review articles.Kunisch et al. (2023) define it as . a class of research inquiries that employ scientific methods to analyze and synthesize prior research to develop new knowledge for academia, practice ...
The present methodological literature review (cf. Aguinis et al., 2020) addresses this void and aims to identify the dominant approaches to sample selection and provide insights into essential choices in this step of systematic reviews, with a particular focus on management research.To follow these objectives, I have critically reviewed systematic reviews published in the two most prominent ...
Systematic Approaches to a Successful Literature Review by Andrew Booth; Anthea Sutton; Diana Papaioannou Showing you how to take a structured and organized approach to a wide range of literature review types, this book helps you to choose which approach is right for your research. Packed with constructive tools, examples, case studies and hands-on exercises, the book covers the full range of ...
This paper is part of a series of methodological guidance from the Cochrane Rapid Reviews Methods Group. Rapid reviews (RR) use modified systematic review methods to accelerate the review process while maintaining systematic, transparent and reproducible methods. In this paper, we address considerations for RR searches. We cover the main areas relevant to the search process: preparation and ...
This paper employs a systematic literature review (SLR) methodology, specifically, by adopting the widely accepted PRISMA technique. The final sample of this study included 309 related studies from 2013 to 2023. Our findings highlight the lack of literature related to the integration of these three types of technologies within a unified AIS ...
Laboratory experiments have long been used to guide predictions of organismal stress in response to our rapidly changing climate. However, the ability to simulate real world conditions in the laboratory can be a major barrier to prediction accuracy, creating obstacles to efforts informing ecosystem conservation and management. Capitalizing on an extensive experimental literature of coral ...