Comparison of methods – an unloved duty? Examples from an ongoing bibliometric study.
Award in scientific knowledge production is given primarily to new methods, new results, in short to innovation. The increasing pace of science seems to put activities as reviewing, comparing and reproducing into disfavour. Ironically, this stands in some contradiction to the call for re-use of data. This presentation gives an overview about a discussion on comparison of methods and results conducted by a group working together on the topic “Same data, different results?”.
The knowledge domain this group operates in is scientometrics. The methods to be compared are approaches, and their computational implementation to cluster scientific literature and to identify scientific topics. This presentation focus on ways to describe the different approaches in a way that they can be compared; and to find methods to compare the different results. In particular it focus on two different approaches: using set based measures and using semantic (lexical) signals. On a more generic level, we ask how much we need to question pattern recognition, and how we ought to deal with the problem that different techniques might reveal different pattern leaving us with an evaluation and interpretation problem.
Andrea Scharnhorst is Head of Research at DANS and Scientific Coordinator at the eHumanities group.
Shenghui Wang – Research scientist in OCLC (Leiden office)
Rob Koopman – Innovation agent in OCLC (Leiden office)