Article: Effect of Guided Research Experience on Product Design Performance
Designing, generally, requires a team effort. Consequently, several variables affecting team performance have been studied, such as team composition, female/male ratio in the organization, and teamwork skills training. This study furthers this effort by investigating the effect of guided external research during the concept generation phase of the design process. The premise of the study is that as resources increase in number and complexity, and time constraints pressure an overcrowded curriculum, professors are challenged to find new methods to train students in the skills needed for the constantly changing workplace. One technique to address this issue, a creative collaboration and its impact on design team performance, is presented here.
The need for information literacy is paramount for engineering students and practicing engineers. However, while students arrive on campus very comfortable with the World Wide Web, they are not trained in efficient use of electronic or traditional resources, nor are they necessarily able to evaluate resources for quality. For example, studies indicate the low use of libraries and library resources by engineering students and that engineering students are least likely to know about and use the literature in their field, instead depending on informal channels of communication [1]. This tendency continues into their professional careers where studies have shown that engineers rely first upon their own previous experience, second upon their personal store of information, third upon colleagues, fourth in house technical reports, handbooks and standards and then upon the published literature [2].
For some engineering courses, however, information gathering is an intrinsic element, particularly for engineering design. Design is a common activity for most engineers. Yet, none of the information literacy related studies in the literature assessed the impact of information literacy instruction on the design performance. There are, however, controlled experiments that looked at the information gathering and using behavior during design such as studies by Christiaans and Dorst [3], Bursic and Atman [4], and Ennis and Gyeszly [5]. In only one of these studies [4] the relation between information usage and design quality was considered. However, in none of these studies an intervention to improve the information literacy skills of students was included. Our study fills a void in the literature by investigating the impact of the information literacy training (external guided research) on the performance of design teams.
Experimental Design and Application: Two sections of the Introduction to Engineering Design (ED&G 100) were included in the study. The same instructor taught both of these sections. Each section consisted of eight, mostly four-person teams. Teams were presented with the design problem (an inverted tooth chain assembly design) along with several design requirements. Each team was given eight weeks to come up with their best solution. During this time, they were to act as companies that were competing to get the project sponsoring company’s business with their design solution. The performance was measured using team quizzes, peer design evaluations, and a blind review of the design reports.
One section had a high performing team skills training intervention while the other received external guided research experience to be used during product design task. The content and delivery issues of the high performing skills training were discussed in detail in an earlier publication [6]. The reason for including two interventions as opposed to having one intervention and one control group is to eliminate the potential difference in in-class time allotment to design teams that could impact design performance. During an 8-week design project an intervention that takes 4 hours in class time could have the potential of confounding study results unless a similar time adjustment is done to the control group. As such an intervention that could be related to improving design team performance was chosen to last a similar amount of time in the control group”>high performing team skills training. This control group intervention was implemented in the same course before, and while its impact was not significant, its direction was in a way to positively impact the design performance [7].
Guided Research Experience – Content and Delivery: The planning of the library sessions for ED&G 100 carefully considered the research done on successful information literacy programs and the specific situations in the engineering classrooms. Before any sessions were planned or web pages developed long discussions of course goals, assignments, syllabus and projects were held. Guided research experience topics and assignments were reviewed as they were created and revised as the semester progressed to address any changes that were needed.
The five library sessions were approximately fifty minutes long, included demonstration, discussion, active learning exercises, and graded assignments. The emphasis during these sessions was on practical application of these skills that were transferable from one database and subject to another. The sessions covered an introduction and motivating component, research databases, web resources, patents knowledge and database, and a cumulative research assignment related to students’ design projects. For more detail please see the full paper.
Results: Despite a higher average GPA for teams that received the high performing team skills training, their overall average project grade was 90.24, which is about 5% lower where a 100 is the perfect score. Furthermore, in an earlier study it was shown that average GPA of design teams significantly affects their design performance [7]. This result, a higher average design performance despite a lower average team GPA, is attributed to the difference in training they have received, and the positive impact of the guided research experience intervention on the performance of design teams.
When the independent variables were investigated for their potential effect on the design project performance using a multiple regression the model accounted for 58% of the variation in design performance (project grade). Training’s relative importance for its effect on the design performance is significant. The implication of this is very powerful: even when the average GPA in a team is not very high, addition of a guided research intervention to the engineering design teaching improves the design performance in engineering teams.
The regression analysis provides estimates of the dependent variable, and measures of the errors that are likely to be involved in using the regression line to estimate the dependent variable. A correlation analysis is also completed, which provides estimates of how strong the relationships are between study variables. While very close to being significant (p=0.06 > 0.05), the correlation between training and design team performance is not significant. However, it is in the direction that was hypothesized, and its magnitude is substantial. Given the fact that the experiment included two different training interventions in order not to confound the results due to potential differences in allotted in-class time for participants, and the fact that only a small sample size is included in the experiment, it is concluded that the effect of the “guided research training” on design team performance is underestimated. Nevertheless, further experimentation with a larger number of teams is planned.
Conclusion: Overall, inclusion of such an intervention to engineering design teaching is recommended for two reasons”>1) it is an avenue to improve design performance, and 2) it provides a mechanism for skill acquisition in support of life-long learning. It is believed that a valuable experience gained in such a manner will likely to transfer to other courses and into the engineering practice of our graduates.
References
[1] Leckie, G.J. and A. Fullerton, “Information Literacy in Science and Engineering Undergraduate Education: Faculty Attitudes and Pedagogical Practices,” College and Research Libraries, Vol. 60, No. 1, 1999, pp. 9-29.
[2] Poland, J., “Informal Communication Among Scientists and Engineers”>A Review of the Literature,” Science and Technology Libraries, Vol. 11, Spring, 1991, pp. 61-73.
[3] Christiaans, H.H.C.M. and K.H. Dorst, “Cognitive Models in Industrial Design Engineering,” Proceedings of the ASME Design Theory and Methodology Conference, Vol. 42, 1992, pp. 131-140.
[4] Bursic, K.M. and C.J. Atman, “Information Gathering”>A Critical Step for Quality in the Design Process,” Quality Management Journal, Vol. 4, No. 4, 1997, pp. 60-75.
[5] Ennis Jr., C.W. and W. Gyeszly, “Protocol Analysis of the Engineering Systems Design Process,” Research in Engineering Design, Vol. 3, No. 1, 1991, pp. 15-22.
[6]. Horner, D., G.E. Okudan, M. Russell, B. Bogue, “A Study on the Cohesion and the Development Stage of Product Design Teams,” Proceedings of the 2002 ASEE Zone I Conference, April 5-6, 2002, 2002, West Point, New York.
[7] Okudan, G.E., D. Horner, and M. Russell, “Achieving High Performing Engineering Design Teams”>A Curriculum Intervention Study,” International Conference on Engineering Education (ICEE2001), August 6-8, 2001, Oslo, Norway.
Author 1: Gul Okudan email: [email protected]
Author 2: Bonnie Osif email: [email protected]