Step 6.3

Test refined beta prototype with target users in the field.

Primary findings

Secondary findings

Primary findings

Barriers

Users comparing two similar products may presume that the more attractive design is functionally superior even when presented with written or verbal evidence to the contrary.
Controlled experiment findings
(View full citation)

Some solutions (cancer medications) present unintended consequential effects (drug resistance) in users.
Research conclusions from conceptual framework
(View full citation)

Field studies are often seen to be time-consuming, providing a vast amount of unstructured data that is difficult to use in development.
Case studies
(View full citation)

Limitation on Consumer contribution — Users might not be willing to contribute to radical innovation projects (barrier of NOT WANTING), due to high anticipated cost of switching from familiar, and fear that user's existing knowledge may become obsolete.
Literature review and case study.
(View full citation)

Limitations on Consumer contribution — Cognitive limitations can hinder the ability of users to delivery valuable input (barrier of NOT KNOWING), due to: 1) Users can be functionally fixed on their current use context; 2) Difficult to evaluate concepts and prototypes when no reference product exists; 3) High technological complexity can limit value of input.
Literature review and case study.
(View full citation)

Users, and in particular consumers, are often not able to articulate their need for really innovative products, because their thinking is framed by the products currently on the market. Also, reactions of potential users when confronted with highly innovative new product concepts are not always reliable, as they may find it difficult to imagine the use of these innovative products in their daily lives.
Literature review.
(View full citation)

Carriers

Firms need to establish a competence in how to systematically identify consumers who are willing and able to contribute to radical innovation in NPD, and how to effectively and efficiently interact with them: 1) Know which users are capable of providing valuable input to innovation projects; 2) Know what interaction patterns with users are appropriate in innovation projects.
Literature review and cast study.
(View full citation)

Guidelines for effective Beta Test program management: 1) Product design should be frozen prior to Beta testing. 2) Effective Beta testing programs recognize the full set of benefits possible (as shown in Figure 1). 3) Maximum utility of Beta testing is derived from recognizing the segmentation of the market, and devising ways of making the Beta tests as close approximations to real-world experiences. 4)As a general control rule, grow the number of Beta sites over time; begin with sophisticated customers who have good relations with the company.
Literature review, analysis of twenty-one programs, and four in-depth field investigations.
(View full citation)

Methods

Beta testing involves time and expense but can be planned to serve multiple purposes such as: validate the product concept, eliminate performance problems prior to market introduction, and serve as an effective sales promotion device.
Literature review, analysis of twenty-one programs, and four in-depth field investigations.
(View full citation)

Conduct market research, business analysis, prototype development and trials to reduce uncertainty, acquire information and ultimately increase new product development success.
Literature Review
(View full citation)

Distribution of collaboration during Field Testing stage was manufacturer involvement (100%), user involvement (79%) and third-party involvement (69%).
Case study of seventeen medical equipment innovations marketed by 13 Dutch firms.
(View full citation)

Field-testing is needed early in the NPD process to ensure completeness of the internal product-testing regimen. This validates product acceptance. For technology-based products it also tracks performance under a range of environments and conditions of use.
Conclusions drawn from case studies and experience.
(View full citation)

Human Factors should be treated as part of the critical path in product design, including usability testing, training of users or sales personnel, writing operating manuals, and writing labels and package inserts.
Medical Device Industry experience.
(View full citation)

Include consumers in Beta prototype testing activities
Survey.
(View full citation)

It is important to collect and assess market and competitive information in order to understand customer needs, wants and specifications for the product; to know customers' price sensitivity; to understand customers' purchase decisions; and to learn about competitors' strategies, strengths and weaknesses.
Case studies conducted on over 600 new product launches in the U.S. and in China.
(View full citation)

Performance Drivers: One is the Quality of Execution. Eight activities distinguish best from worse performers: 1) Conducting a post-launch review (8.2); 2) Assessment of product's value to business (2.1); 3) Test market or trial sell to a limited set of customers (7.13); 4) Concept testing to determine customer reaction to product and gauging purchase intent before Development begins (4.11); 5) Idea Generation (1.3); 6) Customer tests of products under real-life conditions (6.3); 7) Detailed market study/research or Voice of the Customer (4.3, 4.13); 8) Pre-launch business analysis (7.7, 7.8, 7.9).
A quantitative survey of 105 business units, supported by team's experience in NPD modeling, consultation, application and analysis.
(View full citation)

Prototype testing can be used to uncover and correct problems. However, the small samples used may not be representative of the entire population. Information may be more useful for troubleshooting than for predicting success.
Literature Review
(View full citation)

Research data from educational psychology proves people often have a very difficult time taking general concepts and translating them into specific applications. Ensuring comprehension is an important first step, but is not sufficient. One must help people apply the information, and ensure they can perform the tasks to achieve specific results. This can be done by clearly articulating what a user needs to accomplish and in what context.
Medical Device Industry experience.
(View full citation)

Six general themes for NPD best practice are described: (1) Instill a strategic, long-term orientation toward NPD. (2) Have a formal portfolio management process. (3) Implement a formal NPD process supported by a discipline to adhere to this process. (4) Conduct market research proactively. (5) Use cross-functional teams. (6) Utilize standardized criteria and metrics.
Generated from published studies on benchmarking data.
(View full citation)

Techniques for gathering voice of the customer information include: use of customer complaints, internal market research, focus groups, one-to-one interviews, phone interviews, contextual inquiry, customer behavior studies, and perceptual mapping.
Literature review and case studies
(View full citation)

Two-thirds of companies surveyed begin planning for customer support requirements during the latter stages of product development.
Survey of 66 companies plus one case study.
(View full citation)

User input involved an initial focus group to establish the development plan, consumer testing of the alpha prototype, and then again field testing of the beta prototype. All to ensure the device fully met the needs of the target audience.
Paper describes device design, prototype construction and testing with consumers involved at all levels.
(View full citation)

Usually with beta testing, working prototypes are placed with selected customers in order to test the influence of 'environmental factors', as well as the level of customer satisfaction. The results from these tests are used in order to refine the product further and to eliminate 'bugs'.
Literature review.
(View full citation)

Utilization of market research early in the new product development process, and continuing throughout the entire development phase is critical to ensuring success.
Survey data.
(View full citation)

Voice of the Customer Information as a Best Practice for the NPD process: 1) Market and buyer behavior studies are a valuable source of information for planning the market launch. 2) Market research as a tool to help define the product. 3) The customer or user ought to be an integral part of the Development process. 4) Identification of customers or users real or un-articulated needs and their problems, is considered fundamental to voice-of-the-customer research, and should be a key input to product design. 5) Working with highly innovative users or customers.
A quantitative survey of 105 business units, supported by team's experience in NPD modeling, consultation, application and analysis.
(View full citation)

Measures

A range of measures for quantitatively measuring customer and product support could be used in combination to generate a more comprehensive evaluation of support requirements at the design stage and during pre-launch planning. These are: 1) Installation — time required; 2) User Training — time required to train to skill level; 3) Maintenance — mean time between and time per maintenance; 4) Repair — failure rate, fault diagnosis time and mean time to repair; 5) Upgrades — time, human resources and materials required.
Survey of 66 companies plus one case study.
(View full citation)

Only about one-half of the companies surveyed report using a formal product support plan, although most report that their product support procedures could be improved.
Survey of 66 companies plus one case study.
(View full citation)

Quality Function Deployment is a tool for bringing the voice of the customer into the NPD process from conceptual design through to manufacturing. A survey also found that QFD was most commonly used to clarify customer requirements and ensure those customer requirements are considered in the product engineering requirements.
Survey of 400 companies.
(View full citation)

Tips

By acquiring market information through direct interaction with customers, developers obtain an understanding of what customers want and are better able to integrate this information into new products intuitively.
Survey of 166 NPD firms.
(View full citation)

Consumers with the right qualifications to participate in radical innovation projects are highly motivated toward exploration of new rather than familiar solutions, are open to the application of new technology platforms, possess a diverse range of competencies at high levels of proficiency, and operate from within a very supportive environment.
Literature review and case study.
(View full citation)

Customer interaction enhances the NPD process most at the early and and late stages of product design and development. The middle stage of prototyping and bench testing should be left to the internal project staff, while customers can be kept abreast of this interim progress to keep them current. A deficit in current business practice is particularly evident in the early stages of NPD.
Field interviews and 310 survey responses from R&D managers.
(View full citation)

Involvement of customers with strong past ties will result in the development of products with higher competitive performance than those that involve customers with no or few past ties. However, this practice is most effective when developing incremental products, rather than highly innovative products.
Survey of 137 new product development projects.
(View full citation)

Overall, product quality, relative to competitors (as defined by the customer), was ranked as the most important factor in how the product performed in the market. Product "value" compared with that of competitors and the clinical effectiveness of the product were then ranked as the next most important aspects, respectively.
Survey results from medical device manufacturers.
(View full citation)

The outlined dimensions of user competence need to be differentiated according to the different stages of NPD in which the consumers are involved, such as Idea Generation activity, versus Design activity, versus Prototype testing activity.
Literature review and case study.
(View full citation)

To improve short-term competitive performance of products in the marketplace, projects to develop incremental new products should involve homogeneous groups of customers.
Survey of 137 product development projects.
(View full citation)

Secondary findings

Carriers

To mitigate power inequalities and transcend the different cognitive and cultural worlds of participating stakeholders, an independent facilitator should guide the process and fulfill an intermediary role as a knowledge broker.
Source: Sperling & Ashby (2001); Hargadon (2002). In: Klerkx, Laurens & Leeuwis, Cees (2007)

When planning, implementing or evaluating a knowledge translation (innovation) process, diffusion theory suggests that one of the factors that can influence the appeal of new knowledge to a potential knowledge user is its trialability. Generally, adoption strength increases with the knowledge user’s ability to incrementally test the viability of the new knowledge prior to its full implementation.
Source: (Rogers, 2003). In: Ashley, S.R. (2009)

Methods

Empirical studies show that users play an important role for NPD in the field of medical technologies.
Source: Biemans, 1991; Luethje, 2003. In: Lettl, C (2007)

For demand articulation, dialogue between end-users and producers of knowledge and information, as well as other relevant stakeholders, should take place throughout the innovation process.
Source: Douthwaite et al (2001). In: Klerkx, Laurens & Leeuwis, Cees (2007)

Listening to the customer's needs early in the product development process has been identified as being critical for eventual market success.
Source: Griffin & Hauser, 1993; Hauer & Clausing, 1988; Nayak & Chen, 1993. In: Datar, S., Jordan, C.C., Kekre, S., Rajiv, S. & Srinivasan, K. (1997)

Marketing-R&D interface theory suggests that without the interface between these two functions, marketing knowledge would be underused, potentially resulting in unsuccessful products.
Source: Griffin & Hauser (1992); Gupta, Raj, & Wilemon (1986); Song & Dyer (1995); Song & Parry (1997). In: Li, T. & Calantone, R. J. (1998)

The market research methods most frequently cited by Fortune 500 companies include: focus groups (used by 68% of companies surveyd); limited roll out (42%); concept tests (26%); show tests and clinics (19%); attitude and usage studies (19%); conjoint analysis (15%); Delphi (9%); quality function deployment (9%); home usage tests (9%); product life-cycle models (8%); and synectics (8%).
Source: Mahajan and Wind, (1992). In: May-Plumlee, T. & Little, T.J. (2006)

Tips

Among computer and medical equipment manufacturers, successful new products incorporated greater use of market information in the NPD process, while failed products used less.
Source: Ottum & Moore, 1997. In: Suwannapron, P., Speece, M (2003)

Companies must interact with customers when technology is experiencing rapid change.
Source: Day & Wensley (1988). In: Li, T. & Calantone, R. J. (1998)

The greater the customer demand, the greater the need for integration of marketing knowledge and R&D knowledge in order to fill the gap between customer requirements and product offerings.
Source: Gupta, Raj & Wilemon (1986). In: Li, T. & Calantone, R. J. (1998)