Writing a literature review

The literature review is a very important component of the analysis.  You will identify articles and books relating to your project topic, examine the relevant ideas in a critical way, relate them to the subject of your project, and of course, reference your sources.  Whether you are writing the brief review on a specific topic required for a Software Engineering Project, or the more extensive review of literature in a General Computing or Investigative Project, there are some things that you need to bear in mind.  This section give some indicators: further support is given in one of the project lectures.

The literature review will normally be directly related to one or more problems or decisions, or provide the underlying principles that are necessary to understanding the problem area.  It may help you to refine a research question.  Take care to ensure that the material presented relates to the subject of the project.  Your review of the material should be related to your specific problem.  Plain “bookwork” descriptions, where sources are simply paraphrased or information presented in an uncritical way, will attract lower marks.  In the viva, you may be questioned about what you have written, and asked to show your understanding of the ideas and their application in the project.

The literature review is intended to be useful.  Think carefully about what topics you will investigate, and be sure that you know what they will contribute to your project.  There is no point in a literature review that just confirms that your product will be useful to someone, or simply reiterates basic material that you already know.  The technical and other problems that you will have to solve are often a good focus for a review: make sure you know what they are!  It is helpful to conclude a literature review by summarising what you will take forward to the rest of the project.

It is neither necessary nor desirable to include descriptions of basic material.  For example, students sometimes decide to describe the principles of relational databases, common Java syntax, or the different types of UML diagrams, simply because these are going to be used.  This is not a literature review: it is basic material that is not appropriate to a final year project.

The review should not simply be a collection of quotes from other authors. Nor should it simply describe each source in turn: it must be given a structure. Identify several themes and give a succinct, careful discussion of each.  For each theme, you should not just summarise material; you should approach it in a critical way.  This means that you should appraise the work or ideas that you have found, give your perspective on their strengths and weaknesses, and make comparisons.  You may wish to form conclusions, for example, about which of a number of possible approaches is the best to use.  Write as if you are addressing a computer professional, but one who is not an expert in the specific topic.

Finding literature relevant to your topic may require persistence, and you are advised to start as early as possible. You will usually refine your searches and search objectives as you become more familiar with the subject and the available material. If you need to order material through the Inter-Library Loan scheme, it may take several weeks to arrive.

Throughout your project, you should take care to use reliable sources of information, such as textbooks and papers from academic journals and good conferences.  Your supervisor can advise on suitable types of source.  Internet sources may also be of use, but you should carefully consider their reliability.  For example, electronic versions of journals and conference proceeding, technical literature from suppliers whose software or hardware you will be using, and published papers on the websites of university departments, are usually safe sources.  Articles and blogs whose authors cannot be identified as experts in the field, sales literature, and any items whose origin is uncertain, are best avoided.  If you are investigating specific products, you may need to consult suppliers’ literature, but look also for independent accounts.  Evaluate your information sources carefully: web sources in particular are often of poor quality and carry little weight.

Thorough referencing, as described in the ‘Project Report Overview’ section of this handbook, is essential.  Direct quotations should also be clearly shown as quoted material.  Failure to do this will be treated as plagiarism. Remember that you must write the review in your own words.

Project Report – General Computing Project

You should read the following discussion in conjunction with the marking documents.  They are provided at the end of this handbook.  The main elements of this report are as follows:

• Abstract
• Introduction – always Chapter 1
• Analysis – two or more chapters
• Synthesis – one or more chapters
• Evaluation – one or two chapters
• Conclusions and Recommendations – one chapter.

Introduction

The introduction provides a fuller overview of the work to be done than is given in the abstract and sets the scene for the detail provided in subsequent chapters.  You may wish to draw on the background section written for your Terms of Reference, but the introduction should not simply reproduce parts of the TOR: you will develop a fuller understanding during the course of the project.  The introduction is normally written (or at least finalised) at the end of the project, and is written retrospectively, i.e. it says what you did, not what you are going to do.  This section is quite straightforward, but ensure that you have all the elements listed in the marking scheme.  It is quite common for students to lose a mark or two by missing something out.  Typically, the introduction will be about 1500 words.

 Analysis

The analysis section may consist of several chapters. The exact number will depend upon the nature of the work you are undertaking. Typically this part of the report should provide the reader with information they will need to know in order to appreciate and understand the work you have done in the rest of the project. You should assume a computer literate reader.

The analysis follows on from the background given in the introduction.  The marking scheme asks for the following:

• Clear identification and analysis of the problem to be investigated, identifying the key technical or other problems to be solved. Necessary background material that goes beyond the scope of the introduction may be included.
• A critical review of literature related to the topic; this will normally address some combination of the underlying principles of the problem area and possible approaches to solving the problem. The relevance of these ideas to the project should be clear.
• Discussion of approaches appropriate for the solution of the problem;
• A discussion and justification of the product requirements;
• Explanation and justification of the tools and techniques to be used in the project work.

Note that while all these elements should be present, they do not necessarily map on to five separate sections or chapters of your report.  For example, a discussion of approaches to solving the problem may fit naturally at the conclusion of your literature survey.   “Tools and techniques” here refers to what you use to build the product, while “approaches” is used to include higher-level issues such as an overall strategy or architecture, the choice of implementing one algorithm rather than another to carry out a task, a general type of solution, etc.

Your analysis should include a discussion of the wider issues, and critically examine the methods that might be used in solving the problem and any constraints that apply.  Beware of presenting a shallow treatment of the subject which might be obtained from standard texts.  You are expected to support your argument by exploring academic literature which is seminal and up to date.

The background and problem description should describe the actual problem you are looking at, and set it in context.   You should tell the reader about the product, the computing problem(s) to be investigated, etc. as relevant, building on the brief overview given in the introduction. Bring out features of the problem you would not expect the reader to know about. Do not state the obvious.

The literature review will be a very important component of the analysis.  Its length should be guided by the needs of the specific project, but they are typically 4000 – 6000 words for General projects. A good literature review will greatly strengthen your analysis and provide a sound foundation for your later work.  In producing it you will have to research articles and books relating to your project.   You should use respectable academic sources, such as refereed journals or conferences, or other reliable and authoritative sources of information.  Evaluate your information sources carefully.  The section of this handbook on ‘Writing a literature review’ gives further guidance on this.

Once you have described your problem and put it into context by carrying out the literature review, you are in a position to identify the requirements for the product. You can produce its specification.  It is important to be clear about what is required here.  Your actual specification of requirements is part of your product and is marked under that heading.  The analysis includes your discussion and justification of those requirements.  Reasons for choosing particular requirements could relate to user needs, constraints on project scope, to the findings of your literature review, etc. – whatever is appropriate to your particular project.

You may find that you now have several possible approaches and technologies that could be used to solve the problem. If this is the case, you should provide a short analysis of the possibilities justifying your selected route. However, you need only do this in detail if the factors influencing your choices are worth discussing.  There is no point in going through a spurious decision process if there is no real decision to be made, for example, if your client mandates particular software, or your own familiarity with it is the only significant factor in your choice.  Since your reader is a computer professional they do not want to read, for example, a superficial description of the common programming languages such as Java, C++ or VB. Many students choose a programming language or database or other software tool because they are familiar with it. As long as the tool is suitable, your own familiarity with it is sufficient reason for using that technology, and it is sufficient to indicate briefly why it is appropriate.

What is useful, in some cases, is to identify the special features of the language or software that you need in order to develop your product.  As an example, if you needed some form of concurrency then it would be appropriate to note that Java provides a model of concurrent programming via threads.  However, if your project requires the use of some very specialised software or programming language, it may be useful to describe its main features as well as justifying its use, and if the focus of your project is exploring a particular tool or tool features, you will of course need to discuss the features in some depth.

Synthesis

This is a description of the work you have carried out to develop the product.

You should discuss the design, implementation and testing of your product; there are sections on each of these below.  If there were other activities involved in development, such as analysis modelling based on the requirements specification, or if your project involved other practical work such as an experiment using your product, you should also include these: say what you did and discuss any interesting decisions or problems.  If your product does not fit this model – for example, a project whose product is an information strategy – the synthesis should discuss the work needed to create that product.

The narrative should especially identify areas of the work that were particularly interesting or difficult. Assume that the readership of the report will be computer literate individuals who will appreciate the problems you have tackled.

Justify in detail the method(s) you chose to synthesise a solution to the problem.  Discuss how your reading of the literature guided you in your work.  You may wish to make reference to supporting documentation in your discussion of the solution; these will be held in appendices to the report.

In general there should be neither bookwork nor theoretical material here. You should tell the reader what you did, why you did it and how you did it. Unless you have developed a worthwhile product or solved a challenging problem there will be little for you to say (and few marks to gain).

Design

Good products, whether software or hardware, must be designed.  It is not professional to hack out a solution!  Under synthesis you must describe and provide a rationale for that design.  The artefacts produced are models (and perhaps prototypes) that form part of your product.  In the report you tell the reader about your design and discuss the design decisions. Throughout the design section you should justify your choices.  Discuss the implications of making different design choices, and the reasons for the design that you have selected.

The design chapter(s) should give a top-level view of how your product meets its requirements.  For a software product, a good starting point is to describe the architecture of the software.  (If your product is not software, what corresponds to the software architecture?)  For example, suppose you are going to produce a computer game that could be played across the web.  This will involve some software concerned with the communications across the network, some software concerned with the specific game and some with aspects that could be common to many games.  This suggests an architecture consisting of three subsystems.  You may feel there are other possible designs. If so, discuss each and then tell the reader which you decided to follow, and why.

Once you have selected a top-level design you can start to look at the details of each product component.  You will produce design models as required by the development approach that you are following, and you will need to discuss your design decisions.  For example, if you are using an object oriented approach, you will probably describe and justify the important classes in your system. Design patterns are an area of increasing popularity and usefulness. Investigate making use of some.  Explain which you have used and why.

Make careful use of figures and diagrams when describing the design. Any diagrams are there to help the reader understand what you have done. They must form a minor part of the chapter. The full design documentation will be marked under the product marking part of the module.

Another aspect of the design, which you may wish to write about, is the user interface.  There is no point in simply relating HCI theory here, and merely describing or giving pictures of your screen designs is also inadequate: what the reader wants to know is how you have applied the theory.  Justify your design choices in terms of usability principles, and illustrate them with a few carefully selected screen dumps.

You may wish to discuss the design process that you followed. Theoretical descriptions of design processes are unlikely to be interesting here.  How you applied the process and how it affected your product, might be.

Implementation

In an implementation chapter or section, you are describing and justifying how you implemented your product, which for a software product means at the code level.

Do not attempt to describe every detail.  For a software product, do not include large sections of program code. Any code presented should be to illustrate important and interesting features.  You might want to describe the data structures you elected to use, e.g. in Java a LinkedList or a Vector, and explain why you chose the one you did.  If there were any interesting low-level algorithms, you should describe these.  If you feel it is important to put a significant volume of program code into your report, put it in an appendix and reference the appendix.  (The appendix usually contains examples of your code, but the place for your full code is on your USB/CD/DVD.)

Writing about program code can sometimes cause a student problems.  You should be able to find good examples of articles that discuss implementations on the web.  Read them and learn from how they do it.  For Java a good source of examples can be found at JavaWorld.com.

Pick out the key parts of your implementation and provide a rationale for them.  During your attempts to implement your product, you may have had to face unforeseen problems.  Explain how you overcame them.  They may have caused you to modify the original design.  Discuss the implications of those changes.

Testing

Testing is not part of evaluation.  It is the last part of your synthesis activities.  It is about how you checked to make sure your product was a viable and robust piece of software.

A testing chapter of your report indicates the approach you have taken to verifying and validating your system. You should not merely list the tests performed.  Rather, you should discuss how tests were selected, why they are sufficient, why a reader should believe that no important tests were omitted, and why the reader should believe that the system will really operate as desired.

You should explain your overall strategy for testing: black box and/or white box, top down and/or bottom up, kinds of test beds or test drivers used, sources of test data, test suites, coverage metrics, compile-time checks vs. run-time assertions, reasoning about your code, etc.  You might want to use different techniques (or combinations of techniques) in different parts of the program. In each case, justify your decisions.  It is not necessary to describe the techniques; the reader knows about them. Tell the reader what you used and why in the context of your product.  If you carried out usability testing, explain your approach to this.

Explain what classes of errors you expect to find (and not to find!) with your strategy. Discuss what aspects of the design make it hard or easy to validate.

Summarise the testing that has been accomplished and what if any remains.  Which modules have been tested, and how thoroughly?  Indicate the degree of confidence in the code:  what kinds of fault have been eliminated?  What kinds might remain?  Do not include large volumes of tables purporting to be a test log here.  These should be in the product documentation.

Evaluation and Conclusions

You should present two critical evaluations of your work, in separate sections or chapters.  A further chapter gives your conclusions, with recommendations for further work.

An evaluation of your product.

Approach this from a technical point of view.  Attempt to identify the strengths and weaknesses of your product in meeting its requirements, and review the possible alternative technical approaches to its design and implementation.  Beware of the ‘anecdotal’ evaluation – you are expected to take a critical view and justify your argument.  You should try to give evidence to support your evaluation: this could include the result of testing and user trials, feedback from clients, etc.  Do not be afraid to discuss weaknesses: your evaluation will be assessed by its validity, regardless of the quality of the product.  If your product is not software, you will need to be particularly careful in planning how it will be evaluated.  Be sure that enough time is allowed for gathering necessary evidence: it is essential that this is thought about early in the project.

An evaluation of the project process.

This section is fully described in your marking scheme.   The emphasis should be on the learning process and on how well you managed your project work.  What have you learned, and what would you do differently in future?  Achievement of relevant objectives should be assessed, so look at the objectives in your terms of reference and see which ones are relevant here and which are part of the product evaluation.  You can reflect on your project plan and suggest other plans that might have worked better.  You may also be able to discuss legal, social, ethical or professional issues that have arisen and comment on your handling of them.  (If any of these seem more relevant to the product evaluation, it is fine to put them there.)

Conclusions and Recommendations.

Draw conclusions from your study and relate these to your reading of the literature.  Provide an indication of the direction(s) that future work might take.

The conclusions section should be a summary of what has been achieved, and of the main results of the project.  An effective set of conclusions should not introduce new material. Instead it should briefly draw out, summarize, combine and reiterate the main points that have been made in the body of the project report and present opinions based on them.

It is quite likely that by the end of your project you will not have achieved all that you planned at the start; and in any case, your ideas will have grown during the course of the project beyond what you could hope to do within the available time.  The recommendations will focus on further work: this is where you describe your unrealised ideas.  It is where you tell the reader what extra you wish you could have done or what other investigations would benefit this subject area.  Try to look beyond the work that you yourself have done to the subject context. A good set of recommendations can provide the basis for a future project.

Marking Scheme: General Computing Project

Overview

Report: 60%

Abstract & Introduction           5%

Analysis                                   30%

Synthesis                                30%

Evaluation & Conclusions       30%

Presentation                            5%

Product 30%

Fitness for Purpose                 50%

Build Quality                            50%

Abstract and Introduction

Abstract: a precise and concise summary of the project – the essence of the work. It represents the report in a highly condensed form, including the aims of the work, methods, results and conclusions. (Typically 250-400 words.)

Introduction: includes:

• The background to the project.  This states and briefly introduces the subject/application area, including any organisational context, and explains the purpose of the work or reasons for undertaking it.
• The aims and objectives of the project, explaining their meaning and purpose where necessary.
• The precise subject of the work, including the product that was created, areas investigated, the scope of the work and any planned limitations.
• The plan of work given in the Terms of Reference,  making clear the type of work to be done, including the main tools or techniques used, and the main stages of the project.

Analysis

The analysis as a whole is expected to clearly establish and discuss the project topic; identify important ideas and principles in the problem area; and communicate a clear vision of what sort of product is to be produced and how, in general terms, it will function.  The analysis chapters should include the following elements (not necessarily as separate sections).

• Clear identification and analysis of the problem to be investigated, identifying the key technical or other problems to be solved. Necessary background material that goes beyond the scope of the introduction may be included.
• A critical review of literature related to the topic; this will normally address some combination of the underlying principles of the problem area and possible approaches to solving the problem. The relevance of these ideas to the project should be clear.
• Discussion of approaches appropriate for the solution of the problem;
• A discussion and justification of the product requirements;
• Explanation and justification of the tools and techniques to be used in the project work.

Note: the actual specification of requirements is marked as part of the product.

Synthesis

The student is expected to describe and justify the work carried out in order to solve the problem being investigated. This must cover the design, implementation and testing of the product; it will also cover any other project activities such as analysis modelling or experimental work.  The student should describe the activities, and discuss and justify decisions made during the work, e.g. alternative design structures, reasons for chosen data representations, HCI considerations, choice of test strategy, approaches to solving problems.  The work should be supported by suitable references and illustrative examples of the actual product documentation.

This assessment is not of the product deliverables but of the discussion of how the problem was solved.  Deliverables (e.g. software, designs, test plans, etc.) are marked as part of the product.

Evaluation and Conclusions

The evaluation should take a critical approach and arguments should be justified.  Comprises 3 sections, each of which is typically 800-1500 words in length:

Evaluation of the Product.  The exact content required will be dependent on the type of product, but in all cases this section should contain a critical discussion of strengths and weaknesses of the work, including how the product measures up to its specification and / or relevant objectives, discussion of any outstanding problems; and an evaluation of the choice of tools/methods used for the work.  There may be discussion of significant technical decisions and their effects; alternative approaches that could have been taken, and solutions to outstanding problems. Evaluation should refer to any appropriate evidence, including a discussion of the results of any client feedback, user reports, etc.

Evaluation of the project process addresses the way in which the student undertook the project (the process) and how well it was managed and executed.  Achievement of relevant objectives should be assessed. Actual progress made during the project should be related to the Project Plan expressed in the Terms of Reference document.  Should emphasise the learning process: what skills were gained or lessons learned from the experience, and how it might be done better a second time round. Identifies performance problems such as missed milestones and the factors that caused them.  A discussion of alternative project plans, given the benefit of hindsight, may be presented if appropriate.   Relevant legal, social, ethical or professional issues may be discussed.

Conclusions & Recommendations should cover the full project, and should compare the main aims of the project with what was achieved.  Conclusions should be drawn from the evidence presented in the report and should agree and balance logically with the introduction in terms of results and summary. Recommendations and suggestions for further work should indicate a wider perspective than that of the project.

 Presentation

Communication.  The presentation mark is aimed at assessing how effective the report is as a means of communication.  The following aspects are to be considered in assessing this:

There will not be a mark per criterion, but rather the criteria will be used for guidance in assessment.  Note that one criterion alone – e.g. badly spelt, poorly punctuated and ungrammatical English – can wreck a report in terms of its ability to communicate, which is what is being assessed.

Length.  The presentation mark is capped at 3 for reports which exceed the maximum length of 25,000 words, or do not include an accurate word count.  (Note that individual report sections that contain unnecessary or irrelevant material may also be penalised.)

Product

In order to take into account the differences between projects, a general structure for the product marking scheme is presented below.  It is the responsibility of the student, supervisor and second marker to agree detailed criteria in the Terms of Reference.  However, the agreed scheme may be refined in the light of experience during the project and submitted in a revised Terms of Reference which appears as a project report appendix.  If no criteria have been agreed, the default marking criteria (shown below) should be used.  However, these are necessarily very general, and agreed criteria are strongly preferred.

The assessment comprises two parts: ‘Fitness for Purpose’ and ‘Build Quality’.  These are equally weighted.  Within each part, mark ranges may be defined for each criterion, to total 50% for each of Fitness for Purpose and Build Quality.

It is important to remember that the product is not only code.  It consists of all the deliverables defined in the Terms of Reference.  These may vary widely between projects.

Fitness for Purpose may be defined as “Has the student built the right kind of product?”

The main criterion for assessing this is whether the product meets its specified requirements as identified in the Analysis phase, and if not, why not?  Other commonly usable criteria are:

• usability / HCI
• adequacy / completeness (bearing in mind that the product will typically be a prototype rather than a fully-fledged, complete product of practical use or commercial value),
• performance (e.g. good response time, sufficient data capacity),
• robustness
• error handling.

Build Quality may be defined as “Has the student built the product the right kind of way?”

A variety of criteria may be used in its assessment, for example:

• design quality,
• implementation quality,
• correspondence of implementation to the design,
• adherence to standards,
• quality of testing and validation,
• quality of specific documents.

In some cases, it may be difficult to decide under which heading a criterion fits: e.g. HCI might fit under either, depending on the focus of the project.  In these cases, decide on the most appropriate and ensure that the overall marking scheme fits in with the choice.

Demonstration / Viva  

The viva mark assesses the student’s presentation of their work, not the quality of that work.  This means that a good presentation receives a good viva mark even if the work itself is poor, and a poor presentation will receive a poor viva mark even if the project work is excellent. (Students who have not achieved their objectives should present as much of the work as they have done.)  Two aspects of the viva are marked:

Presentation / demonstration of the work relates to the student’s initial presentation / demonstration of their product.  This will typically last no more than 20 minutes.  The following criteria are used:

• Good planning and preparation
• Appropriate and relevant choice of content
• Clear oral presentation: audibility, fluency etc.
• Clearly demonstrates the features of the product.
• Appropriate structure of presentation and allocation of time
• Demonstrates good understanding of the work done.
• Where used, visual aids are clear and visually pleasing.

Handling of questions and discussion of the work relates to the student’s answers to questions, reactions to comments on the product, etc. Students must be able to explain the work done, including product code.  Where the tools used to build the product do not involve traditional code, the equivalent deliverables should be discussed. Marking will take into account both the content and the clarity of answers.